Chapter 15. Adding Sounds To Virtual Worlds

You can incorporate sound into your virtual worlds by including at least one csSound node in a scene graph and by invoking a csSoundAction. The csSoundAction plays the sound file specified in the csSound node. This node also includes parameters, such as volume, for playing the sound.

This chapter describes how to set and play sound using Cosmo 3D.

These are the sections in this chapter:

Overview

A csSound node contains the location of a sound file and the parameters used for playing it. To play a sound file in a virtual world, attach one or more csSound objects to a scene graph and apply a csSoundAction to it. To associate a specific sound to a specific shape, make the csSound object and shape nodes children of the same group nodes.

A csSound node references a csAudioClip node and a csAudioClip node references a csAudioSamples node, as shown in Figure 15-1.

Figure 15-1. Sound Classes

Figure 15-1 Sound Classes

A csAudioSamples node contains the raw sound data. A csAudioClip node specifies how a sound file should be played.

For more information about csAudioClip, see “How to Play a Sound File”. For more information about csAudioSamples, see “Specifying Audio Files”.

csSound Fields

The fields in csSound specify the sound source to play by specifying a csAudioClip object. csSound can optionally specify the location, the direction of the sound, and the spatial characteristics of the sound.

void                setSource(csAudioClip* audioClip);
void                setSpatialize(csBool spatialize);

void                setControl(ControlEnum control);

void                setLocation(const csVec3f& location);
void                setLocation(csFloat v0, csFloat v1, csFloat v2);
void                setDirection(const csVec3f& direction);
void                setDirection(csFloat v0, csFloat v1, csFloat v2);

void                setIntensity(csFloat intensity);
void                setMaxIntensity(csFloat maxIntensity);
void                setCullIntensity(csFloat cullIntensity);

void                setCurrentFrame(csFloat currentFrame);
void                setPriority(csFloat priority);

void                setMinFront(csFloat minFront);
void                setMaxFront(csFloat maxFront);
void                setMinBack(csFloat minBack);
void                setMaxBack(csFloat maxBack);
const csIntArray&   getEvents();

The following sections describe these fields.

Choosing Sound Samples to Play

To specify how to play a sound file, pass a csAudioClips object to setSource(). The csAudioClips object identifies a csAudioSample object, which contains the sound file to play. For more information about csAudioClips, see “How to Play a Sound File”.

A sound file commonly contains more than one sound sample. Some samples may contain more than one sound channel per sound interval to create, for example, stereo.

To choose a starting location in a sound file, pass the starting frame to the setCurrentFrame() field. A frame is equal to (1/SampleRate) of a second. The sample rate might be, for example, 44KHz, or 44,000 Hz. If, for example, you pass 44000.0 into the setCurrentFrame() field, the sound would begin playing one second (44000 × 1/44000 = 1) into the sound file.

Sound Priority

Your application can play only a limited number of sounds at the same time. The factors that determine whether or not a sound is heard include

  • The proximity of the listener to the source.

  • The priority level of the sound.

Higher priority sounds are heard instead of lower priority sounds if too many sounds could possibly be heard by the listener at the same time. Set the priority level of a sound in the setPriority() method.

Playing the Sound File

The setControl() field provides an intuitive interface for playing the sound sample in sound files. You pass into setControl() any of the ControlEnum values, including PLAY, PAUSE, REWIND, FASTFORWARD, and STOP.

Locating and Directing the Sound

To enable all of the other effects implemented by the fields in the csSound node, covered in the next section, pass a non NULL value to setSpatialize(). If you pass a NULL value to the field, the volume is a constant value throughout the scene. This choice is appropriate, for example, for background music.

To locate the sound source in a scene, pass its coordinates to the setLocation() field.

Cosmo 3D gives you a great deal of control over how sound propagates from the source. When you supply a vector describing the direction of the sound, the sound propagates in all directions, but attenuates least in specified direction. The attenuation of the sound over distance is characterized by an ellipse, as shown in Figure 15-2.

Figure 15-2. Sound Direction

Figure 15-2 Sound Direction

In this figure, (1.0, 1.0, 0.0) is passed as the direction vector to setDirection(). The ellipse tips, accordingly, at a 45 degree angle.


Note: A transformation node can reorient the sound's location and direction.

Cosmo 3D provides the following limiting tools to fashion the attenuation of the sound over the ellipse:

  • Maximum intensity—defines the maximum possible volume regardless of how close the listener is to the sound source.

  • Minimum intensity—defines the lowest possible volume of a sound. In practice, since this value is often set to zero, the minimum intensity perimeter defines the range of the sound.

The setIntensity() field specifies the volume of the sound at its source. The setMaxIntensity() field specifies the maximum volume of a sound. If a maximum intensity is set, as is the case in Figure 15-2, the intensity of the sound within the maximum intensity perimeter does not attenuate and is equal to the volume specified by setIntensity(). The maxFront() field specifies the maximum intensity perimeter within which the listener hears the maximum volume of the sound.

Outside of the maximum intensity perimeter, the intensity of the sound attenuates over distance until it reaches the minimum intensity perimeter. Beyond the minimum intensity perimeter, the volume of the sound source is constant, defined by the setCullIntensity() field.

Reverse Direction Sound

Cosmo 3D also provides a complimentary set of fields that allow you to define the propagation and attenuation of the same sound in the opposite direction, as shown in Figure 15-3.

Figure 15-3. Forward and Reverse Sound Propagation

Figure 15-3 Forward and Reverse Sound Propagation

The minimum and maximum intensities in the reverse direction are the same as those in the forward direction. The minimum and maximum intensity perimeters, however, are specified separately with the minBack(), and maxBack() fields.

How to Play a Sound File

You use csAudioClip to specify how to play the sound files referenced in the csAudioSamples node.

csAudioClip contains the following fields:

csMFString* url() const;
void setSamples(csAudioSamples* samples);
void setPitch(csFloat pitch);
void setStartTime(csTime startTime);
void setStopTime(csTime stopTime);
void setDoppler(csBool doppler);
void setLoop(csBool loop);
void setDescription(const csString& description);
csTime getDuration();
csBool getIsActive();

These set() fields have corresponding get() fields. Table 15-1 describes how these fields are used.

Table 15-1. csAudioClip Fields

Field

Description

url

Specifies a WWW URL where the sound source file can be found.

setSamples

Attaches the csAudioClip object to a csAudioSamples object.

setPitch

Adjusts the pitch of a sound sample.

setStartTime

Specifies a beginning time for the sound sample to begin playing. Time here is an expression of clock time.

setStopTime

Specifies an ending time for the sound sample to stop playing. Time here is an expression of clock time.

setDoppler

Enables the doppler effect, which is the attenuation of a sounds pitch based on the velocity of the sound source relative to the listener, for example, when a sound source, like a train whistle, approaches a listener rapidly, the pitch sounds higher; when the same sound source passes the listener, the pitch lowers.

setLoop

Allows a sound file to keep playing.

setDescription

Provides a description in the node of the sound source.

getDuration

Returns the duration of the playing of the sound source; subtracts setStartTime from setStopTime.

getIsActive

Returns whether or not the sound should be played.

Example 15-1 sets all of the fields in a csAudioSamples node.

Example 15-1. Setting the Fields in a csAudioClip Object


// create a csAudioClip object
csAudioClip* clip = new csAudioClip();

// attach the audio clip to a csAudioSamples object
clip->setSamples(csAudioSamples truck_horn);

// Set the parameters of the csAudioClip object
clip->setPitch(1.0);
clip->setIntensity(1.0);
clip->setMaxIntensity(4.0);
clip->setLoop(TRUE);
clip->setDescription(“Truck horn”);

Specifying Audio Files

You use the csAudioSamples node to specify the source of the audio files and a variety of parameters that describe those sound samples. To use the audio files specified by this node, pass a csAudioSamples object as the argument to csAudioClip::setSamples(), for example,

csAudioSamples* truck_horn_file = new csAudioSamples;
csAudioClip* truck_horn_style = new csAudioClip;

truck_horn_style->setSamples(truck_horn_file);

In this example, the truck_horn_file is used as the audio file for the truck_horn_style object.

csAudioSamples has the following fields:

void                setFileName(const csString& fileName);
void                setNumFrames(csInt numFrames);
void                setSampleRate(csFloat sampleRate);
void                setSampleSize(csInt sampleSize);
void                setSampleType(SampleTypeEnum sampleType);
void                setNumChannels(csInt numChannels);
void                setSampleScale(csFloat scale);
void                setLoadStatus(LoadStatusEnum loadStatus);
LoadStatusEnum      getLoadStatus();
csMFByte*           samples() const;

These set() fields have corresponding get() fields. Table 15-2 describes how these fields are

Table 15-2. Fields of csSoundSamples

Field

Description

setFileName

Attaches the csAudioSample node to a specific sound source file.

setNumFrames

Is the equivalent to the sampling rate of the sound sample, for example, 44 KHz.

setSampleRate

Specifies the sampling rate of the sound sample.

setSampleSize

Specifies the size of the source sound files.

setSampleType

Specifies the format of the sampling rate. Valid values include UNSIGNED_INT_SAMPLE_TYPE or FLOAT_SAMPLE_TYPE.

setNumChannels

Specifies the number of channels for each sound, for example, stereo has two channels, quad sound has four channels.

setSampleScale

Scales the overall volume of the sound sample. If if sound sample was recorded to loud or soft compared to other sound samples, you can scale the volume of the sound file so that its volume matches that of the other sound files.

setLoadStatus

Returns the status of whether or not the sound file loaded; valid values include LOAD_NEEDED, LOAD_FAILED, LOAD_PENDING, and LOAD_COMPLETE.

samples

Returns the multivalued array field that contains the actual audio samples.


Manipulating the Audio Samples Directly

csAudioSample::samples() returns the multivalued array field that contains the actual audio samples. This handle allows you to directly manipulate the array field. For example, to set a sound value, use the following code:

samples->set(index, value);

To set the number of samples and then edit the array directly, use the following code:

samples->setCount(16*44400);
char *samps = samples->edit();
for(i=0;i<16*44400);i++)
    samps[i]=DETERMINE_SAMPLE(i);
samples->editDone();

Example Setting a csAudioSamples Node

Example 15-2 sets all of the fields in a csAudioSamples node.

Example 15-2. Setting the Fields in an csAudioSamples Object


// create an audio sample node
csAudioSamples* horn = new csAudioSamples;

// attach the audio sample node to a specific file
horn->setFileName(“truck_horn.xxx”);

// Set the parameters for the source sound file
horn->setNumFrames(44000.0);
horn->setSampleRate(44000.0);
horn->setSampleSize(2000);
horn->setSampleType(UNSIGNED_INT_SAMPLE_TYPE);
horn->setNumChannels(2);
horn->setSampleScale(1.0);
...

// Load the sound sample by setting the audiosample filename
horn->load();

// Make sure the sound loaded successfully
if (horn->getLoadStatus() == LOAD_FAILED)
    abort();

When load() is called, Cosmo 3D reads the samples in the file into the sample field directly.

Playing Sound in Immediate Mode

When a csSoundAction is invoked on a scene graph, the action traverses the scene graph and gathers a list of active csSound nodes. The action notifies csContext internally of this list of nodes. When the context is applied to the rendering pipeline, the sounds specified in the associated csAudioSamples nodes are played.

You can also play a sound file immediately. Instead of using a csSoundAction to trigger the playing of the sound file, you use a csSoundPlayer node.

All of the code used to play sounds, either using csSoundAction or csSoundPlayer, is encapsulated in csSoundPlayer. Consequently, all of the field settings discussed previously in this chapter also need to be specified in csSoundPlayer.

csSoundPlayer Methods

csSoundPlayer methods provide sophisticated control over the sound source and the “microphone” recording the sound. Some of those controls include:

  • Position of the sound source and microphone.

  • Motion of the sound source and microphone; helpful in simulating a Doppler shift.

  • Scaling of the sound source intensity and frequency.

  • Number of recording channels.