Chapter 4. Adding and Removing Clips

Clips are segments of audio and video. Managing clips involves adding and removing them from Video Server Toolkit (VST).

Managing clips is discussed in the following sections:

Overview of Adding Clips Procedure

This section provides a procedural overview of the tasks involved in adding clips to VST's cache.

The following sections explain the steps in the following procedure in greater detail.

To add clips to the cache, use the following procedure.

  1. Copy media to the VST machine.

    If you are not using a real-time filesystem, use FTP put or get commands.

    If you are using a real-time filesystem (recommended), you can either use the vtrutil tool included with VST, or a modified form of FTP, vtrftpd, installable from the VST image. You use the familiar FTP command, get, with vtrftpd to add files to VST.

    For more information, see “Adding Clips to VST”.

  2. Align the clips.

    If you used FTP or vtrftpd to add clips to VST, you must align them. For more information, see “Clip Alignment”.

    Clips added to VST using the vtrutil tool are already aligned.

  3. Register the clip with VST.

    Clips are registered automatically upon startup of VST.

    If you want notify VST more quickly, or if you have added clips to the system in a different way from those listed in step 1, use one of the following options:

    • Install the fsmon daemon from the VST image. It periodically updates the list of clips in the cache.

      At times, however, fsmon may not auto detect a clip because of an unrecognized format. In that case, you must use one of the following options to notify VST.

    • Use the mvcp command, CADD.

    • Use the vtrclip tool.

    For more information see, “Notifying VST”.

VST Media Formats and Types

VST can use different media types and formats

Media Types

VST supports two types of media:

  • Intraframe

    In intraframe media, the video data for each frame is self-contained and does not depend on the data from neighboring frames. Examples of intraframe media include JPEG, Rice, DVCPRO, and uncompressed video.

  • Interframe

    In interframe media, the compression techniques used for some video frames may require data from neighboring frames in order to decompress the video frame. An example of interframe media is MPEG1 and MPEG2.

Media Format

VST uses different formats for storing digital media in its clip cache filesystem(s):

  • Intraframe media

    • vframe format—for variable-sized frames, including Rice, JPEG, and uncompressed

    • DIF format—for fixed-sized frames, including DVCPRO and uncompressed

  • Interframe media

    • stream format - for MPEG2

vframe Format

The vframe (short for variable-size frame) format is specific to VST. The video and audio data for each frame is stored contiguously in the media file. A separate index file contains header information that describes the encoding parameters of the video and audio data, and a frame map that stores the offsets of the video and audio data for each frame within the media file.

The vframe format is used for recording and playback of intraframe media, including uncompressed video as well as Rice-coded and JPEG-compressed video.

The vtrvfutil command-line utility provides various analysis and update functions for the vframe format. See the vtrvfutil man page for further information.

DIF Format

DIF is the 4:1 video compression frame format used by Panasonic's DVCPRO.

Stream Format

The stream format consists of a media file, which is simply the native media bitstream, and an index file, which maps random access cueing points to the appropriate offset in the bitstream.

The stream format is used for playback of MPEG2 media.

The vtrmpegutil command line utility provides various analogies and update functions for MPEG2 stream format assets. See the vtrmpegutil man page for more information.

Adding Clips to VST

Adding clips to VST has been partially automated. For all clip formats:

  1. Transfer the media file into the filesystem, /usr/vtr/clips.

  2. For Vframe or stream format clips, transfer the index file for the clip into /usr/vtr/index.

Transferring Clips to VST

There are several ways to transfer clips into VST. Which process you use depends on the type of filesystem used. There are two types of files systems:

  • Non-real-time

  • Real-time

To guarantee real-time I/O rates, you must use a real-time filesystem.

Non-Real-Time Filesystems

Transferring files to a non-real-time filesystem is as easy as using ftp, cp, or rcp locally. You can also use ftpd from a remote site.

Real-Time Filesystems

To transfer clips to a real-time filesystem, use the customized version of ftpd (server) called vtrftpd. It is in subsystem vst_eoe.sw.ftpd. Only this version of FTP can write to real-time filesystems locally.

The feature enhancement in vtrftpd is used when you:

  • Use FTP (client) on a system connected to VST server.

  • Use FTP (client) proxy on the VST server.

Both of these methods set up a TCP/IP connection with the vtrftpd server running on the VST system.

To write to the filesystem, use of the following methods:

  • On the remote archive server, use FTP's put command to copy a clip to the clip cache of VST from a remote host.

  • On the machine running VST, use a proxy (secondary control) connection, vtrftpd, as follows to copy a clip from the remote archive system to the local clip cache of VST:

    ftp source-system
    ftp> cd source-directory
    ftp> proxy open localhost
    ftp> proxy get filename /usr/vtr/clips/filename
    ftp> cd index
    ftp> proxy get indexname /usr/vtr/index/indexname
    ftp> proxy bye
    ftp> bye

localhost is the VST machine.

vtrftpd is not used when you only use ftp (client) on the VST server because the ftpd server on another system would not be the enhanced version. To copy a clip from the archive server to VST on the machine running VST, login to the archive system, run ftp, and use the put command.

Clip Alignment

Certain clips might need to be aligned in the filesystem for enhanced performance. In single-disk systems, media files are aligned only with the filesystem's block size so that only one I/O operation is needed to access the data.

In multi-disk, RAID, striped systems, the media files must also be aligned with the stripes.

When VST is used to record media, it automatically places the media data in the correct, aligned locations on the disk.

When media data is transferred to VST, it often must be aligned with the disk's blocks and stripes. Whether or not media data must be aligned on your disks depends on the media type of the media file:

  • Vframe media data requires alignment.

  • DIF or Interframe media data does not require alignment.

The media data (video and audio for each frame) for a Vframe media clip must be aligned on natural filesystem and disk drive boundaries to enable efficient read and write access to the clip.

Degree of Alignment

Frame-oriented media data in an intraframe clip is aligned along two boundaries:

  • Minor alignment boundary

  • Major alignment boundary

A single element of a frame (a video field or audio chunk) never crosses a minor or major alignment boundary. VST will not do a read or write operation to the clip media file which crosses a major alignment boundary.

Minor Alignment

The minor alignment matches the greater of the filesystem block size, or the system memory page size. On O2 workstations, the minor alignment is usually 4 KB, unless a larger filesystem block size was used to construct an XFS filesystem. On Origin, the minor alignment is usually 16 KB, unless a larger filesystem block size was used.

Major Alignment

The major alignment matches the stripe size of the disk volume that holds the clip cache filesystem.

If the clip cache resides on a single disk, no major alignment is required.

If the clip cache resides on a single RAID subsystem, major alignment is required for efficient I/O access to the media data. Either make a real-time filesystem on the RAID and set the real-time extent size of the XFS filesystem to the desired I/O operation size, or add a configuration line to /usr/vtr/config/vtrfsinfo.conf. For example:

#/dev/root 2m

Note: vtrfsinfo.conf is not created by VST so you first must create the file.

If the clip cache resides on a striped XLV volume, the major alignment matches the stripe size of the XLV volume.

Realigning a vframe Clip

A vframe clip is automatically aligned if you use vtrvfutil to copy the clip, as follows:

% vtrvfutil OriginalClipName NewClipName

vtrufutil automatically stores the media files in /usr/vtr/clips/.

To copy a vframe clip from outside the clip cache, use the following syntax:

% vtrvfutil -i file=MediaFile,index=IndexFile - NewClipName

vtrvfutil can also be used to realign a clip in-place, as follows:

% vtrvfutil -c realign ClipName

See the vtrufutil man page for a complete list of the options.

Notifying VST

VST detects the clips when the clip is first loaded and every time VST starts. The clips are then listed in mcclips.

If you place a clip in the clip filesystem, the first time you attempt to access the clip through VST, it attempts to auto-detect the clip's format and add it to the VST clip list. However, until you attempt to access the clip, it will not be visible in a list of clips you obtain from VST. To see the clip immediately, use the following command:

# vtrclip add clipName

Note: There might be a delay because adding clips always has a lower priority then playout/record.

You can also use the following mvcp command to notify VST of an added clip.

CADD clipName

However, if you install the vst_eoe.sw_fsmon subsystem, the VST filesystem Monitor will monitor the clip filesystem for clips that are added or deleted and automatically update the internal VST clip list. Sometimes, FSmon cannot autodetect clips added to VST when the clips require the use of an index file, for example, for MPEG and Rice-encoded clips. In this case, you must use the vtrclip add command to add the index file and notify VST.

Transferring a Clip Segment

vtrftpd honors in and out points on DIF clips for transfers from the real-time filesystem on the server. You can transfer the segment of a clip between in and out points instead of having to transfer the whole clip, saving transfer time and disk space in many cases.

If a clip has no in or out point set, vtrftpd uses these defaults:

  • If the start point (beginning of the file) is invalid or missing, it is set to 00:00:00.00.

  • If the in point is invalid or missing, it is set to the start point.

  • If the out point is invalid or missing, it is set to the end of the file.

Overriding Clip Segment Transfer

If in and out points are set in a clip, but you want to transfer the entire clip, you can override this feature. To transfer an entire file that has in and out points set, enter the following at any point in the session:

ftp> site marks 

This command is a toggle. To turn the clip segment transfer feature on again, enter the command again.

Removing Clips

There are three ways to remove clips from VST.

  • VST Clip Manager (mcclips)

    For more information about using the VST Control panel to remove a clip, see the VST Developer's Guide.

  • Command line

    You can completely remove a clip from VST using the following command:

    # vtrclip rm clipName

    clipName is the name of the clip file.

  • MVCP command, CRM.

    For more information about the CRM command, see Appendix A, “Multiport Video Computer Protocol (MVCP) Command Summary.”

Exporting VST Clips

vtrvfutil makes the clip readable by Silicon Graphics digital media tools, such as dminfo, dmconvert, and mediaplayer, or third-party tools that use the Silicon Graphics Movie Library.

You can use the vtrvfutil command to add Quicktime formatting information to a clip:

# vtrvfutil -c makeqt clipName

clipName is the name of the clip file.

Note: Clip files are not necessarily usable by other tools.