Chapter 1. Introduction

This chapter is a quick introduction to the Digital Media Software Development Kit (henceforth, the dmSDK). It includes a table of terms, followed by an example audio output program.

To get started with the dmSDK, you should read this chapter, then browse the online example programs. For an in-depth treatment, consult later chapters as you experiment with your own programs.


Note: The material in this chapter assumes that the dmSDK is installed on your workstation, and that you have access to the dmSDK example programs.


Terms

These terms are used throughout this document, and some are used in the dmSDK code. Read these first to avoid any confusion.

Term 

Definition

graphics / video 

In dmSDK, graphics and video are not synonymous: “graphics” indicates the graphical display used for the user-interface on a computer; “video” indicates the type of signal sent to a video cassette recorder, or received from a camcorder.

capability tree 

A capability tree is the hierarchy of all DM devices in the system, and contains information about each DM device. An application may search a capability tree to find suitable media devices for operations you wish to perform.

system 

The highest level in the capability tree hierarchy. It is the machine on which your application is running. This machine is given the name DM_SYSTEM_LOCALHOST. Each system contains one or more devices.

physical device 

A device that corresponds to device-dependent modules in the dmSDK. Typically, each device-dependent module supports a set of software transcoders, or a single piece of hardware. Examples of devices are audio cards on a PCI bus, DV camcorders on the 1394 bus, or software DV modules. Each device-dependent module may expose a number of logical devices: jacks, paths, or transcoders.

jack 

A logical device that is an interface in/out of the system. Examples of jacks are composite video connectors and microphones. Jacks often, but not necessarily, correspond to a physical connector -- in fact, it is possible for a single dmSDK jack to refer to several such connectors. It is also possible for a single physical connector to appear as several logical jacks.

path 

A logical device that provides logical connections between memory and jacks. For example, a video output path transports data from buffers to a video output jack. Paths are logical entities. Depending on the device, it is possible for more than one instance of a path to be open and in use at the same time.

transcoder 

A transcoder is a logical device that takes data from buffers via an input pipe or pipes, performs an operation on the data, and returns the data to another buffer via an output pipe. The connections from memory to the transcoder, and from the transcoder to memory, are called pipes. Example transcoders are DV compression, or JPEG decompression.

UST 

Unadjusted System Time. UST is a special system clock which runs continuously without adjustment. This clock is used to synchronize media streams.

MSC 

Media Stream Count. MSC is a measure of the number of media samples which have passed though a jack. This is useful to synchronize media streams.

Getting Started with the dmSDK

Before you begin, you should examine your system with the dmquery(1dm) tool. This tool prints a list of all supported dmSDK devices on the system. Here is an example dmquery on the system Linux i386 Workstation:

Example 1-1. dmquery Printout

% dmquery
SYSTEM: Linux i386 Workstation
Devices:
   Software DV_MMX Codec [0]
   OSS audio device [0]


This printout indicates that there are two installed devices: a software DV transcoder, and an audio I/O device (which in this case, is built using the Linux OSS driver). Other options to dmquery allow you to gather more information about the installed devices; but for now, just knowing their names will suffice.

See the dmquery(1dm) man page for more information.

Simple Audio Output Program

This example program outputs a short beep. To keep it simple, a few details, primarily error-checking, are skipped. This program only includes the operations required to produce the beep.


Note: Consult the online example code for more advanced programs.


Step 1: Include the dmsdk.h and dmutil.h Files

To begin, you will need the dmsdk.h and dmutil.h files. The dmSDK library provides the core functionality, and the dmUtil library provides some convenient utility functions built on that core. As an application developer, you may choose to use only the core, or you may find it convenient to utilize the simpler utility functions.

Include the files as follows:

#include <dmsdk.h>
#include <dmutil.h>

Step 2: Locate a Device

You must query the capabilities of the system to find a suitable digital media device with which to perform your audio output task. To do that, you must search the dmSDK capability tree, which contains information on every dmSDK device on the system.

In your search, you should start at the top of the tree as follows:

  1. Query the local system to find the first physical device that matches your desired device name.

  2. Look in that device to find its first output jack.

  3. Find an output path that goes through that jack.

In this case, assuming that the device name is being passed in as a command-line argument, use some of the utility functions to find a suitable output path:

DMint64 devId=0;
DMint64 jackId=0;
DMint64 pathId=0;

dmuFindDeviceByName( DM_SYSTEM_LOCALHOST, argv[1], &devId );
dmuFindFirstOutputJack( devId, &jackId );
dmuFindPathToJack( jackId, &pathId );

Step 3: Open the Device Output Path

An open device output path provides your application with a dedicated connection to the hardware, and it allocates system resources for use in subsequent operations. The device path is opened with an open call as follows:

 dmOpen( pathId, NULL, &openPath );

If the open call is successful, you will get an open path identifier. All operations using that path must use its identifier.


Note: Sometimes an open call can fail due to insufficient resources (typically because too many applications may already be using the same physical device).


Step 4: Set Up the Audio Device Path

Now you set up the path you just opened for your operation. In this case you will use signed 16-bit audio samples, with:

  • A single (mono) audio channel

  • A gain of -12dB

  • A sample rate of 44.1kHz

To make those settings, you must construct a controls message to describe them. The controls message is a list of param/value (DMpv) pairs, where the last entry in the list is DM_END.

dmpv controls[5];
DMreal64 gain = -12; // decibels

controls[0].param = DM_AUDIO_FORMAT_INT32;
controls[0].value.int32 = DM_FORMAT_S16;
controls[1].param = DM_AUDIO_CHANNELS_INT32;
controls[1].value.int32 = 1;
controls[2].param = DM_AUDIO_GAINS_REAL64_ARRAY;
controls[2].value.pReal64 = &gain
controls[2].length = 1;
controls[3].param = DM_AUDIO_SAMPLE_RATE_REAL64;
controls[3].value.real64 = 44100.0;
controls[4].param = DM_END;

Notice that this message contains both scalar parameters (for example, the number of audio channels) and an array parameter (the array of audio gains).

Step 5: Set Controls on Audio Device Path

After the controls message has been constructed, you must set the controls on the open audio path as follows:

dmSetControls(openPath, controls);

This call makes all the desired control settings and does not return until those settings have been sent to the hardware. If it returns successfully, it indicates that all of the control changes have been committed to the device (and you are free to delete or alter the controls message).


Note: All control changes within a single controls message are processed atomically. So, either the call succeeds, and they are all applied, or the call fails, and none are applied.


Assuming that the call succeeded, the path is now set up and ready to receive audio data.

Step 6: Send Buffer to Device for Processing

This example assumes that you have already allocated a buffer in memory and filled it with audio samples. To send that buffer to the device for processing, you must first construct a buffers message that describes it. That message includes both a pointer to the buffer and the length of the buffer (in bytes):

 DMpv msg[2];
  msg[0].param = DM_AUDIO_BUFFER_POINTER;
  msg[0].value.pByte = ourAudioBuffer;
  msg[0].length = sizeof(ourAudioBuffer);
  msg[1].param = DM_END;

Then, send the buffers message to the opened path:

 dmSendBuffers(openPath, msg);

When the message is sent, it is placed on a queue of messages going to the device. The send call does very little work: it gives the message a cursory look before sending it to the device for later processing.


Note: Unlike the set call, the send call does not wait for the device to process the message, it simply enqueues it and then returns.


Step 7: Begin Message Processing

You must tell the device to start processing enqueued messages. This is done with the begin transfer call as follows:

dmBeginTransfer(openPath);

You can sleep while the device is busy working on the message as follows:

sleep(5)

This is not the best way to approach this, but it is the simplest. (It will be changed in the next example.)

Step 8: Receive the Reply Message

As the device processes each message, it generates a reply message which is sent back to our application. By examining that reply we can confirm that the buffer was transferred successfully, as follows:

DMint32 messageType;
DMpv* message;

dmReceiveMessage(openPath, &messageType, &Message );

if( messageType == DM_BUFFERS_COMPLETE )
  printf("Buffer transferred!\n");

Step 9: Close the Path

Once you have verified that the buffer transferred successfully, you can close the path as follows:

dmClose(openPath);

Closing the path ends active transfer and frees any resources allocated when the path was opened.

Realistic Audio Output Program

The preceding procedure was for a single audio buffer. In this example, you will process millions of audio samples.

Step 1: Open the Device Output Path

Open the device output path just as in the previous example:

dmOpen( pathId, NULL, &openPath );

Opening the path also allocates memory for the message queues used to communicate with the device. One of those queues will hold messages sent from our application to the device, and one will hold replies sent from the device back to our application.

Step 2: Allocate Buffers

If you were only processing a short sound, you could preallocate space for the entire sound and perform the operation straight from memory. However, for a more general and efficient solution, you need to allocate space for a small number of buffers, and reuse each buffer many times to complete the whole transfer.

Here, assume that memory has been allocated for twelve audio buffers, and that those buffers have been filled with the first few seconds of audio data to be output.

Step 3: Send Buffers to the Open Path

Now send each of the twelve buffers to the open path. Here the queue of messages between application and device becomes more interesting. The dmSDK enables you to enqueue all the buffers without the device having even looked at the first one as follows:

int i;
for(i=0; i<12; i++)
  {
   DMpv msg[3];
   msg[0].param = DM_IMAGE_BUFFER_POINTER;
   msg[0].value.pByte = (DMbyte*)buffers[i];
   msg[0].maxLength = imageSize;
   msg[1].param = DM_AUDIO_UST_INT64;
   msg[1].param = DM_END;
   dmSendBuffers(openPath, msg);
  }

Notice that each audio buffer is sent in its own message, this is because each message is processed atomically, and refers to a single instant in time. In addition to the audio buffer, this message also contains space for an audio Unadjusted System Time (UST) time stamp. That time stamp will be filled in as the device processes each message. It will indicate the time at which the first audio sample in each buffer passed out of the machine.

Step 4: Begin the Transfer

Now you can tell the device to begin the transfer. It reads messages from its input queue, interprets the buffer parameters within them, and processes those buffers with the following:

dmBeginTransfer(openPath);

At this point, you can sleep as the device processes the buffers. However, a more efficient approach is to select the file descriptor for the queue of messages sent from the device back to your application. In dmSDK terminology, that file descriptor is called a wait handle on the receive queue:

 DMwaitable pathWaitHandle;
  dmGetReceiveWaitHandle(openPath, &pathWaitHandle);

Having obtained the wait handle, you can wait for it to fire by using select on IRIX/Linux, or WaitForSingleObject on Windows as follows:

On IRIX/Linux:

 fd_set fdset;
  FD_ZERO( &fdset);
  FD_SET( pathWaitHandle, &fdset);

  select( pathWaitHandle+1, &fdset, NULL, NULL, NULL );

On Windows:

WaitForSingleObject( pathWaitHandle, INFINITE );

Step 5: Receive Replies from the Device

Once the select call fires, a reply will be waiting. Retrieve the reply from the receive queue as follows:

 DMint32 messageType;
  DMpv* replyMessage;

  dmReceiveMessage(openPath, &messageType, &replyMessage );

  if( messageType == DM_BUFFERS_COMPLETE )
    printf("Buffer received!\n");

This reply has the same format and content as the buffers message that was originally enqueued, plus any blanks in the original message will have been filled in. In this case, the reply message includes the location of the audio buffer that was transferred, as well as a UST time stamp indicating when its contents started to flow out of the machine:

 DMbyte* audioBuffer = replyMessage[0].value.pByte;
  DMint64 audioUST    = replyMessage[1].value.int64;


Note: The UST time stamp is useful to synchronize several different media streams (for example, to make sure the sounds and pictures of a movie match up).


Step 6: Refill the Buffer for Further Processing

At this point you can refill the buffer with more audio data, and send it back to the device to be processed again with the following:

dmSendBuffers(openPath, replyMessage);

In this case you are making a small optimization, so rather than construct a whole new buffers message, simply reuse the reply to your original message.

At this point you have processed the reply to one buffer. If you wish, you can now go back to the select call and wait for another reply from the device. This can be repeated indefinitely.

Step 7: End the Transfer

Once enough buffers have been transferred, you can end the transfer as follows:

dmEndTransfer(openPath);

In addition to ending the transfer, this call performs the following:

  • Flushes the queue to the device.

  • Aborts any remaining unprocessed messages.

  • Returns any replies on the receive queue to the application.

The endTransfer call is a blocking call. When it returns, the queue to the device will be empty, the device will be idle, and the queue from the device to your application will contain any remaining replies.

If you wish, at this point, you can send more buffers to the path (see “Step 3: Send Buffers to the Open Path”).

Step 8: Close the Path

Use the following to close the path:

dmClose(openPath);


Note: This chapter has provided only a quick introduction to an audio output device. Through a similar interface, the dmSDK also supports audio input, video input, video output, and memory-to-memory transcoding operations.


Audio/Video Jacks

The Digital Media Library is concerned with three types of interfaces: jacks for control of external adjustments, paths for audio and video through jacks in/out of the machine and pipes to/from transcoders. All share common control, buffer, and queueing mechanisms. In this section these mechanisms are described in the context of operating on a jack and its associated path. In subsequent sections, the application of these mechanisms to transcoders and pipes is discussed.

Opening a Jack

Before setting controls to a jack, a connection must be opened. This is done by calling dmOpen.

DMstatus dmOpen(const DMint64 objectId, DMpv* options, DMopenid* openId);

A jack is usually an external connection point and most often one end of a path. Jacks may be shared by many paths or they may have other exclusivity inherent in the hardware. For example, a common video decoder may have a multiplexed input shared between composite and S-video. If only one can be in use at a given instance, then there is an implied exclusiveness between them. Many jacks do not support an input message queue since an application cannot send data to a jack (it must be sent via a path). Therefore, the dmSendControls and dmSendBuffers are not supported on a jack, so that dmSetControls must be used to adjust controls. Typically, the adjustments on a path affect hardware registers and can be changed while a data transfer is ongoing (on a path that connects the jack to memory). Examples are brightness and contrast. Some controls are not adjustable during a data transfer. For example, the timing of a jack cannot usually be changed while a data transfer is in effect. Reply messages may be sent by jacks and usually indicate some external condition, such as sync lost or gained.

Constructing a Message

Messages are arrays of parameters, where the last parameter is always DM_END. For example, the flicker and notch filters can be adjusted with a message such as the following:

DMpv message[3];
message[0].param = DM_VIDEO_FLICKER_FILTER_INT32;
message[0].value.int32 = 1;
message[1].param = DM_VIDEO_NOTCH_FILTER_INT32;
message[1].value.int32 = 1;
message[2].param = DM_END 

Setting Jack Controls

Since jack controls deal with external conditions and not processing associated with data transfers, applications use dmSetControls or dmGetControls calls to manipulate these controls. Here is an example of how the genlock vertical and horizontal phase can be obtained immediately:

DMpv message[3];
message[0].param = DM_VIDEO_H_PHASE_INT32;
message[1].param = DM_VIDEO_V_PHASE_INT32;
message[2].param = DM_END;
if( dmGetControls( aJackConnection, message)) 		handleError();
 else
   printf("Horizontal offset is %d, Vertical offset is %d\n",
      message[0].value.int32, message[1].value.int32);

dmSetControls and dmGetControls are blocking calls. If the call succeeds, the message has been successfully processed. Note that not all controls may be set via dmSetControls. The access privilege in the param capabilities can be used to verify when and how controls can be modified.

Closing a Jack

When an application has finished using a jack it may close it with dmClose:

DMstatus dmClose(DMopenid openId);

All controls previously set by this application normally remain in effect though they may be modified by other applications.