Chapter 4. Synchronizing Data Streams and Signals

You can use special signals recognized or generated by the OCTANE Personal Video board—UST (unadjusted system time), MSC (media stream count)—to synchronize data streams, and use the board-generated Internal Video Sync signal to synchronize video and audio signals. This chapter explains

Using UST, MSC, and Buffered Media Streams for Synchronization

Whenever a VL path is open in continuous mode, the OCTANE Personal Video board and certain other Silicon Graphics video devices continuously try to dequeue media stream samples from the path's buffer for input, or to enqueue media stream samples onto the path's buffer for output. If the buffer between the application and each device never underflows or overflows, then the application can measure and schedule the timing of input and output signals to 100% of the accuracy of the underlying device.

Occasionally, the application is held off and audio, video, or both come out late. Buffer underflow on output and overflow on input can result from the application not keeping the buffer adequately filled for the following reasons:

  • The application is busy with other tasks, allowing too much time between putting fields into the buffer.

  • Processes are subject to various interruptions (10 to 80 ms for some processes) under IRIX because

    • the process for filling the buffer is running at too low a priority

    • the process cannot get a resource from IRIX that it needs, such as memory pages

To get around this problem, a mechanism built into the VL helps keep track of data flow into and out of buffers by providing accurate timing information for each frame of video that enters or leaves the system. This mechanism, called UST/MSC, produces matched pairs of two numbers:

  • unadjusted system time (UST), a time value that is used to state timing measurements to applications

  • media stream count (MSC), a count value that identifies a particular media stream sample (a video field or frame)

The device keeps a counter called the device media stream count (device MSC), which increments by one every time the device attempts to enqueue or dequeue a media stream sample, whether or not the enqueue or dequeue attempt is successful. UST/MSC was designed to return timing information in a form that is valid whenever the buffer is not underflowing or overflowing.

The UST/MSC capability and the buffering that goes with it are appropriate for applications and devices such as movie players and digital video editing devices.

UST/MSC affords maximally accurate synchronization when scheduling cannot be guaranteed and some buffering is acceptable. Also, if scheduling becomes reliable at some later point, UST/MSC continues to function the same way with no code changes required; the buffers can be made smaller, and the result is a low-latency application with the same accurate synchronization.

Note that UST/MSC itself

  • does not add any latency to an application

    The buffer adds latency: it increases the time the application would take to respond to some output event by changing its input (and vice versa). This solution to the synchronization problem is useful for applications in which a small latency can be sacrificed for more accuracy.

  • does not require that an application trade off latency for accuracy

  • does not require that an application use any particular size buffer

  • delivers the full accuracy of the underlying hardware's timing support regardless of the scheduling characteristics of the application

  • could be useful for graphics and texture even for low-latency applications

The code below is a high-level algorithm to maintain synchronization of two buffered media streams that send data from memory to hardware outputs; a corresponding one is necessary for the other direction:

create video buffer between me and the audio output;
create audio buffer between me and the video output; 
while (1)
   sleep until one of the buffers is getting empty;
   for (video buffer)
         use UST/MSC to determine:
            “at what time (what UST) will the next video data I enqueue
            on the buffer actually go out the jack of the machine?”;

   for (audio buffer)
         (exact same thing as above, except for audio)

   From the predicted video and audio USTs, determine
      “what is the synchronization error between the audio and video

   Enqueue more frames to fill up the audio and video buffer queues.
   If there is synchronization error, enqueue new frames to either skip
   frames on the stream that is behind or repeat frames on the stream
   that is ahead.

The answers to the questions in the pseudocode above are obtained with three VL calls that manipulate UST and MSC and are explained in the next section.

Media Library Interfaces for UST and MSC

UST/MSC calls allow you to associate a UST with a particular piece of data that just left a buffer or is about to enter a buffer. The VL calls for determining the MSC and UST—vlGetUSTMSCPair(3dm), vlGetFrontierMSC(3dm), and vlGetUSTPerMSC(3dm)—help synchronize input and output of different data streams in cases where the application is getting data from or putting data into each device via a buffer. The application is at the “frontier” end of this buffer and the devices are at the “device” end of the buffer.

  • vlGetUSTMSCPair() gets the timing information for each frame or field as it enters or leaves the physical jack of a device.

    This call returns an atomic UST/MSC pair for the jack (specified with the VL_NODE) for a given path that contains a VL_MEM node. The returned MSC is not guaranteed to be the one currently at the jack, nor is it even guaranteed to be the number of any media stream sample currently in the application's buffer. To relate the returned MSC to a particular item in the application's buffer, you must use vlGetFrontierMSC().

  • vlGetFrontierMSC() gets the frontier MSC associated with a particular VL_MEM node.

    The frontier MSC, at the application end of the media stream, is the MSC of the next item that the application removes from or puts into the buffer.

  • vlGetUSTPerMSC() gets the time spacing of fields or frames in a path (the nominal average UST time elapsed between media stream samples in a given VLPath that includes a VL_MEM node).

These calls are used for extrapolating a UST/MSC pair as shown in vlGetFrontierMSC(). For other types of media streams, a similar mechanism extrapolates the UST/MSC pair; for example, for audio, use equivalent AL calls.

Once you have calculated the extrapolated UST/MSC pairs for both media streams, you can determine the synchronization error. The difference in the audio and video USTs for matching frame numbers is the amount they are out of sync. To resynchronize them, you must enqueue new frames to either skip frames on the stream that is behind or repeat frames on the stream that is ahead. The number of frames to be skipped or repeated is the difference in USTs divided by the frame rate.

To use UST/MSC, the application must have separate handles for each separate piece of data coming in or going out of some kind of buffer. The application can use these handles to specify, for example, a particular frame to output or pixels of a particular field to get.

Note: For complete details, including syntax, code examples, and caveats, see the references pages for these calls.

Using the Internal Video Sync Signal

Internal Video Sync refers to a synchronization signal produced or consumed by some audio and video devices. The purpose of the signal is to ensure that simultaneous audio and video signals are precisely synchronized.

This section explains

Internal Video Sync Producers and Consumers

While there may be multiple consumers of the Internal Video Sync signal, there can be only one Internal Video Sync producer (master of the Internal Video Sync line) in a system at any time. Table 4-1 lists Silicon Graphics options that produce or consume the Internal Video Sync signal.

Table 4-1. Internal Video Sync Signal Producers and Consumers



OCTANE Personal Video board

OCTANE Personal Video board

OCTANE Digital Video board

OCTANE Digital Video board

Digital Audio Option board

Digital Audio Option board

DIVO digital video option board for Origin2000™/Onyx2™

DIVO option board

OCTANE Compression board



InfiniteReality™ graphics


OCTANE and Onyx2 built-in audio

Setting the Internal Video Sync Signal Producer

Routines can use two Internal Video Sync calls, ksyncstat() and ksyncset(). ksyncstat() returns a list of Internal Video Sync-capable devices in the system. The devices are given as node names, not full pathnames; for example:

struct kstat_s ks_statbuf[64]
int i;

// Read system ksync configuration
ksyncstat( ks_statbuf, 64 );

// Find current Master
for( i=0; ks_statbuf[i].kName[0] != 0; i++ ) {
    if ( ks_statbuf[i].kFlags & KsyncIsProducer )
       // name of current master is in ks_statbuf[i].kName

// Search for potential producers..
for(i=0; ks_statbuf[i].kName[0] != 0; i++ ) {
    if( ks_statbuf[i].kFlags & KsyncProducerCapable ) {
            // found a producer, name is 
            // in ks_statbuf[i].kName
    else if ( ks_statbuf[i].kFlags & KsyncConsumerCapable ) {
            // found a consumer, name is 
            // in ks_statbuf[i].kName

The structure for ksyncstat() is as follows:

       ** ksync flag values

       #define KsyncIsProducer         0x1
       #define KsyncProducerCapable    0x2
       #define KsyncConsumerCapable    0x4
       #define KsyncActive             0x8

typedef struct {
           char        kName[ 64 ];
           int         kFlags;
       } kstat_t;

       int     ksyncstat( 
                       kstat_t         *buf, 
                       int             bufSz );        /* in bytes */

The buffer pointed to by buf is filled with as many kstat_t structures as there are Internal Video Sync devices on the system, or as many as the buffer holds. The element kName is the name of the device node on the hardware graph. Note that this name is the node name and not the full pathname.

ksyncset() causes a device to begin producing the Internal Video Sync signal. This call takes a string as an argument, for example:

ksyncset(“Personal Video”);

This example specifies a device. If another device is already producing the signal, that device immediately stops producing it and the device specified in the call begins producing it.


Specifying None has the effect of turning off the Internal Video Sync signal. Also, if a device is specified that is not active in the system, Internal Video Sync signal generation is turned off and an error message is produced.


If the string corresponds to a string returned by ksyncstat(), and that name corresponds to a potential producer, that device becomes the new Internal Video Sync master. If there are no such correspondences, all producers are shut off. Using the string None (or any string that does not correspond to a potential producer) also shuts off all producers.

The Internal Video Sync feature is also implemented as a panel. This feature is incorporated into vcp and apanel as well, accessible in the Utilities menu.