Chapter 4. Audio/Visual Paths

In ML, the logical connections between jacks and memory are called paths. For example, a video output path provides the means to transfer video information from buffers in memory through a video output jack.

This chapter discusses the following:

Opening a Logical Path

Before you send messages to a device, you must open a processing path that goes through it. This is done by calling mlOpen(3dm) as follows:

MLstatus mlOpen (MLint64 pathId, MLpv* options,
                     MLopenid* openid);


  • objectId is the 64-bit unique identifier for the path object to be opened. Obtain these identifiers by calling mlGetCapabilities.

  • options is a pointer to a list of optional parameters. This list may be NULL. Obtain these parameters by calling mlGetCapabilities.

  • openid is the resulting open device handle. Use this to refer to the open instance of the path object.

Think of a path as a logical device; a physical device (for example, a PCI card) may simultaneously support several such paths. A side effect of opening a path is that space is allocated for queues of messages from your application to the device and replies from the device back to your application. All of the messages sent to a queue share a common payload area and are required to observe a strictly ordered relationship. That is, if message A is sent before message B, then the reply to A must arrive before the reply to B.

Constructing a Message

Messages are arrays of parameters, where the last parameter is always ML_END. For example, set the image width to be 720 and the image height to be 480 as follows:

MLpv message[3];
message[0].param = ML_IMAGE_WIDTH_INT32;
message[0].value.int32 = 720;
message[1].param = ML_IMAGE_HEIGHT_INT32;
message[1].value.int32 = 480;
message[2].param = ML_END

Processing Out-of-Band Messages

In some cases, an application wishes to influence a device without first waiting for all previously enqueued messages to be processed. These cases are known as out-of-band messages. They are performed with the mlSetControls(3dm) or mlGetControls (3dm) calls.

Following is an example of how you can immediately get the width and height of an image:

MLpv message[3];
message[0].param = ML_IMAGE_WIDTH_INT32;
message[1].param = ML_IMAGE_HEIGHT_INT32;
message[2].param = ML_END
if( mlGetControls( somePath, message))
  printf("Image size is %d x %d\n",


  • somePath is pathID from a previously opened path

  • message is an array of parameter/value (MLpv) pairs

Out-of-band messages work well for simple sets and queries. They are blocking calls. If the call succeeds, the message has been successfully processed.

Sending In-Band Messages

Out-of-band messages are appropriate for simple control changes, but they provide no buffering between your application and the device. For most applications, processing real-time data will require using a queuing communication model. ML supports this with the following calls:


For example, to send a controls message to a device input queue, use the following:

MLstatus mlSendControls( MLopenid openId, MLpv* message);


  • openId is a previously-opened digital media object

  • message is an array of parameter/value (MLpv) pairs

Devices interpret messages in the order in which they are enqueued. Because of this, the time relationship is explicit between, for example, video buffers and changes in video modes.

Note: The mlSendControls and mlSendBuffers calls do not wait for a device to process the message. Rather, they copy it to the device input queue and then return.

When your application sends a message, it is copied into the send queue. The message is then split between a small fixed header on the input list and a larger, variable-sized space in the data area.

Sometimes, there is not enough space in the data area and/or send list for new messages. In that case, the return code indicates that the message was not enqueued. As a rule, a full input queue is not a problem -- it simply indicates that the application is generating messages faster than the device can process them.

For some devices, the system may use device-specific knowledge to best manage messaging transactions. For example, when you call mlSendBuffers, the system may copy the message exactly as described above, or it may send part or all of the message directly to the hardware. Regardless of what happens, the system always looks to your application as described here.

Each message you send is guaranteed to result in at least one reply message from the device, allowing you know when your message is interpreted and what is the result:

  • In the case of control parameters, you should check the return message to make sure your control executed correctly.

  • In the case of video buffers, you should allocate buffer space in your application and then send an indirect reference to that buffer in a message. Once your application receives a reply message, you can be certain the device has completed your request and finished with the memory, so you are free to reuse it.

Some devices can send messages to advise your application of important events (for example, some video devices can notify you of every vertical retrace). However, no notification messages will be generated unless you explicitly request them.

Processing In-Band Messages

The device processes messages as follows:

  1. Removes the message header from the send queue.

  2. Processes the message and writes any response into the payload area.

  3. Places a reply header on the receive queue.

In general, your application must allow space in the message for any reply you expect to be returned.

Note: The device performs no memory allocation, but rather uses the memory allocated when the application enqueued the input message. This guarantees that there will never be any need for the device to block because it did not have enough space for the reply.

Processing Exception Events

In some cases, an exception event occurs that requires the device to pass a message back to your application. Your application must explicitly ask for such events.

Possible exception events are:


The audio input sampling frequency changed.


An audio buffer was not available for an I/O transfer.


Device encountered an error and is unable to recover.


The device is not available for use.


A video buffer was not available for an I/O transfer.


Device detected a valid input video signal.


Device lost the video input signal.


Device detected a valid output genlock.


Device lost the output genlock sync signal.


A video vertical retrace occurred.

If you ask for events, your application must read its receive queue frequently enough to prevent the device from running out of space for messages that you have asked it to enqueue. If the queue starts to fill up, then the device will enqueue an event message advising that it is stopping notification of exception events.

Note: The device never needs to allocate space in the data area for reply messages. It will automatically stop sending notifications of events if the output list starts to fill up. Space is reserved in the receive queue for a reply to every message your application enqueues. If there is insufficient space, attempts to send new messages will fail.

Processing In-Band Reply Messages

To receive a reply message from a device, use mlReceiveMessage (3dm) as follows:

MLstatus mlReceiveMessage(MLopenid openId,
                          MLint32* messageType,
                          MLpv** reply);


  • openId is a previously-opened digital media object.

  • messageType indicates why this reply was generated. It could come from the following:

    • A call to mlSendControls, mlQueryControls , or mlSendBuffers

    • Generated spontaneously by the device as the result of an event

  • reply is a pointer that is guaranteed to remain valid until you attempt to receive a subsequent message. This allows a small optimization -- you can read the current message in place without first copying it off the queue. It is acceptable to overwrite a value in a reply message and then send that as a new message.

This call returns the earliest unread message sent from the device back to your application.

Beginning and Ending Transfers

Devices do not begin to process enqueued messages until explicitly instructed to by an application with the mlBeginTransfer(3dm) call:

MLstatus mlBeginTransfer( MLopenid openId);

This call frees the device to begin processing enqueued messages. It also commands the device to begin generating exception events. Typically, an application will open a device, enqueue several buffers (priming the input queue), and then call mlBeginTransfer. In this way, it avoids the underflow that could otherwise occur if the application were swapped out immediately after enqueueing the first buffer to the device.

To stop a transfer, call mlEndTransfer(3dm):

MLstatus mlEndTransfer( mlopenid openId);

This causes the device to do the following:

  • Stop processing messages containing buffers

  • Flush its input queue

  • Stop notification of exception events

Closing a Logical Path

When your application has finished using an open path, it may close it by using the following:

MLstatus mlClose( MLopenId openId);

This causes an implicit mlEndTransfer on any device with an active transfer. It then frees any resources used by the device. If you wish to have pending messages processed prior to closing a device, you must identify a message (perhaps by adding a piece of user data or by remembering its MSC number) and make sure it is the last thing you enqueue. When it appears on the output queue, you will know all messages have been processed. At that point, you can close the device.

For more information, see the mlClose (3dm) man page.