Whenever a VL path is open in continuous mode, the HD I/O Board and certain other Silicon Graphics video devices continuously try to dequeue media stream samples from the path's buffer for input, or to enqueue media stream samples onto the path's buffer for output. If the buffer between the application and each device never underflows or overflows, then the application can measure and schedule the timing of input and output signals to 100% of the accuracy of the underlying device.

Occasionally, the application is held off and audio, video, or both come out late. Buffer underflow on output and overflow on input can result if the application does not keep the buffer adequately filled for the following reasons:

To get around this problem, a patented^[1] mechanism built into the VL helps keep track of data flow into and out of buffers by providing accurate timing information for each frame of video that enters or leaves the system. This mechanism, called UST/MSC, produces matched pairs of the following two numbers:

The device keeps a counter called the device media stream count (device MSC), which increments by one every time the device attempts to enqueue or dequeue a media stream sample, whether or not the enqueue or dequeue attempt is successful. UST/MSC was designed to return timing information in a form that is valid whenever the buffer is not underflowing or overflowing.

The UST/MSC capability and the buffering that goes with it are appropriate for applications and devices such as movie players and digital video editing devices.

UST/MSC provides maximally accurate synchronization when scheduling cannot be guaranteed and some buffering is acceptable. Also, if scheduling becomes reliable at some later point, UST/MSC continues to function the same way with no code changes required; the buffers can be made smaller, and the result is a low-latency application with the same accurate synchronization.

Note that UST/MSC itself

For the HD I/O Board, UST/MSC pairs are maintained in software and are valid only during a transfer. Make calls to vlGetUSTMSCPair() and vlGetFrontierMSC() only during a transfer; these calls block until at least one buffer is successfully transferred. Note the following:

The code below is a high-level algorithm to maintain synchronization of two buffered media streams that send data from memory to hardware outputs; a corresponding one is necessary for the other direction:

create video buffer between me and the audio output;
create audio buffer between me and the video output; 
while (1)
{
   sleep until one of the buffers is getting empty;
   for (video buffer)
      {
         use UST/MSC to determine:
            “at what time (what UST) will the next video data I enqueue
            on the buffer actually go out the jack of the machine?”;
      }

   for (audio buffer)
      {
         (exact same thing as above, except for audio)
      }

   From the predicted video and audio USTs, determine
      “what is the synchronization error between the audio and video
      streams?”

   Enqueue more frames to fill up the audio and video buffer queues.
   If there is synchronization error, enqueue new frames to either skip
   frames on the stream that is behind or repeat frames on the stream
   that is ahead.
     }
}

The answers to the questions in the pseudocode above are obtained with three VL calls that manipulate UST and MSC and are explained in the next section.

UST/MSC calls allow you to associate a UST with a particular piece of data that just left a buffer or is about to enter a buffer. The VL calls for determining the MSC and UST—vlGetUSTMSCPair(3dm) , vlGetFrontierMSC(3dm) , and vlGetUSTPerMSC(3dm) —help synchronize input and output of different data streams in cases where the application is getting data from or putting data into each device via a buffer. The application is at the “frontier” end of this buffer and the devices are at the “device” end of the buffer.

These calls are used for extrapolating a UST/MSC pair as shown in vlGetFrontierMSC(). For other types of media streams, a similar mechanism extrapolates the UST/MSC pair; for example, for audio, use equivalent AL calls.

After you calculate the extrapolated UST/MSC pairs for both media streams, you can determine the synchronization error. The difference in the audio and video USTs for matching frame numbers is the amount they are out of sync. To resynchronize them, you must enqueue new frames to either skip frames on the stream that is behind or repeat frames on the stream that is ahead. The number of frames to be skipped or repeated is the difference in USTs divided by the frame rate.

To use UST/MSC, the application must have separate handles for each separate piece of data coming in or going out of some kind of buffer. The application can use these handles to specify, for example, a particular frame to output or pixels of a particular field to get.

---

Note: For complete details, including syntax, code examples, and caveats, see the man pages for these calls.

---