Chapter 2. Configuring DMF

This chapter describes how to configure DMF, verify the configuration, and perform some periodic maintenance tasks.

Overview of the Configuration Steps

The steps outlined in the following procedure are required to configure DMF.

Procedure 2-1. Configuration Steps

  1. Install DMF, ensuring that FLEXlm licensing is set up correctly. Installation is described in the DMF Release and Installation Guide for IRIX Systems.


    Note: You must read “Installation Considerations” for a description of special configuration issues regarding installation.


  2. Ensure that your PATH and MANPATH environment variables are set to include DMF paths. See “Setting PATH Environment Variables”.

  3. Invoke dmmaint(8) so that you can create or modify your configuration file. Using dmmaint is described in the DMF Release and Installation Guide for IRIX Systems.

  4. Determine how you want to complete periodic maintenance tasks. See “Configuring Automated Maintenance Tasks”.

  5. Edit the configuration file to define the base object, daemon object, the objects for daemon maintenance tasks, and objects for automated space management. See “Configuring the Base Object”, through “DMF Policies”.

  6. Define the media-specific process (MSP) objects, the object for MSP maintenance tasks, set up the MSPs, and configure your mounting service. See “Setting up Tape MSPs”, through “Setting up Disk MSPs”.

  7. Verify the configuration with the dmcheck(8) script. See “Verifying the Configuration”.

  8. Start DMF. See “Initializing DMF”.

Installation Considerations

This section discusses installation considerations that will affect how your system is configured.

File System Mount Options

The Data Management API (DMAPI) is the mechanism within IRIX and the XFS file system for passing file management requests between the kernel and DMF. Ensure that you have installed DMAPI and the appropriate patches as listed in the files accessed by the News button on the DMF installation interface (dmmaint(8)).


Caution: In order for file systems to be managed by DMF, they must be mounted to enable the DMAPI interface. You can do this by using the mount -o dmi command or by declaring parameter 4 in the fstab entry to be dmi. (refer to the man pages for mount or fstab). Failure to enable dmi for DMF-managed file systems will result in a configuration error.


Inode Size Configuration

DMF state information is kept within a file system structure called an extended attribute. Extended attributes can be either inside the inode or in attribute blocks associated with the inode. DMF runs much faster when the extended attribute is inside the inode, because this minimizes the number of disk references that are required to determine DMF information. In certain circumstances, there can be a large performance difference between inode-resident extended attribute and non-resident extended attribute.

You should configure your file systems to ensure that the extended attribute is always inode-resident. This is done with the mkfs_xfs command. Declare the inode size to be 512 bytes using the -i size=512 option. File systems that already exist will have to be dumped, recreated, and restored. This change is not mandatory.

Configuring Daemon Database Record Length

A daemon database entry is composed of one or more fixed length records: a base record (dbrec) and zero or more path segment extension (pathseg) records. If the path value that is returned to the daemon by the MSP can fit into the path field of the daemon's dbrec record, DMF does not require pathseg records. If the MSP supplies a path value that is longer than the path field in the dbrec, DMF creates one or more pathseg records.

The default size of the path field of the dbrec is 34 characters. This size allows the default paths returned by the dmatmsp, dmdskmsp, and dmftpmsp to fit in the dbrec path field as long as the user name portion of the dmftpmsp or dmdskmsp path (username/bit_file_identifier) is 8 characters or fewer. In almost all cases, you should not need to reconfigure the daemon database record.

The default size of the path field in the pathseg record is 64. For MSP path values which are just slightly over the size of the dbrec path field, this will result in a large amount of wasted space for each record that overflows into the pathseg record. The ideal situation would be to have as few pathseg records as possible.

The advantage of having very few pathseg records lies in increased efficiency for retrieving daemon database records. There is no need to access the pathseg key and data files to retrieve a complete daemon database record.

The disadvantage of using the default path size arises mainly in the tape MSP application in which there is a small amount of wasted space in the daemon's dbrec data file. By extending the default path field size to 34 (8 bytes more than the tape MSP requires), there is a resulting 5% wasted space in the daemon's dbrec data file. For a 10 MB dbrec file, this is 500 Kbytes of wasted space.

For installations that run only the tape MSP and for which the 5% wasted disk space is an important consideration, the size of the path field in the daemon dbrec record can be configured at any time before or after installation. (The same holds true for any installation which might be using the dmftpmsp or dmdskmsp with a different path-generating algorithm or any other MSP which supplies a path longer than 34 characters to the daemon).

Procedure 2-2. Daemon Database Record Length Configuration

The steps to configure the database entry length are as follows:

  1. If the dmdaemon is running, use dmdstop(8) to halt processing.

  2. If a daemon database already exists, perform the following steps:

    1. cd HOME_DIR/daemon (HOME_DIR is the value of HOME_DIR returned by the dmconfig base command)

    2. dmdump -c . > textfile (textfile is the name of a file that will contain the text representation of the current database)

    3. cp dbrec* pathseg* dmd_db.dbd backup_dir (backup_dir is the name of the directory that will hold the old version of the database)

    4. rm dbrec* pathseg* dmd_db.dbd

  3. cd /etc/dmf/dmbase/lib/rdm

  4. Backup the dmd_db.dbd and dmd_db.ddl files that reside in /etc/dmf/dmbase/lib/rdm. This will aid in disaster recovery should something go wrong.

  5. Edit dmd_db.ddl to set the new path field lengths for the dbrec and/or pathseg records. For the most efficient use of disk space for the dmatmsp, set the dbrec path size to 26.

  6. Regenerate the new database definition:

    /etc/dmf/dmbase/etc/dmddlp -drsx dmd_db.ddl

  7. Backup the new versions of dmd_db.dbd and dmd_db.ddl for future reference or disaster recovery.

  8. If the daemon database was dumped to text in step 2, complete the following steps:

    1. cd HOME_DIR/daemon

    2. dmdadm -u -c "load textfile" (textfile was created in step 2)

  9. If the daemon was running in step 1, restart it by executing dmdaemon(8).

Interprocess Communication Parameters

Ensure that, in the operating system configuration file, the following IPC kernel configuration parameters are set equal to or greater than the default: MSGMAX, MSGMNI, MSGSEG, and MSGSSZ. The parameters are described in Appendix A of IRIX Admin: System Configuration and Operation, document number 007-2859.

Configuring Automated Maintenance Tasks

DMF lets you configure parameters for completing periodic maintenance tasks such as the following:

  • Making backups (full or partial) of user file systems to tape

  • Making backups of DMF databases to disk

  • Removing old log files and old journal files

  • Monitoring DMF logs for errors

  • Running hard deletes

  • Running dmaudit(8)

  • Monitoring the status of tapes in tape MSPs

  • Merging tapes that have become sparse (and stopping this process at a specified time)

Each of these tasks can be configured in the DMF configuration file through the use of TASK_GROUPS parameters for the DMF daemon and the tape MSP. The tasks are then defined as objects.

For each task you configure, a time expression defines when the task should be done and a script file is executed at that time. The tasks are provided for you in the etc/admin directory in the configured DMF directory (/etc/dmf/dmbase).

The automated tasks are described in “Configuring Daemon Maintenance Tasks”, for the daemon tasks and in “Configuring Tape MSP Maintenance Tasks”, for the tape MSP.

provides a summary of the automated maintenance tasks:

Table 2-1. Automated Maintenance Task Summary

Task

Purpose

Parameters

Related Object Type

run_audit

Audit databases

 

daemon

run_copy_databases

Backup DMF databases

DATABASE_COPIES

daemon

run_remove_journals

Remove old journal files

JOURNAL_RETENTION

daemon

run_remove_logs

Remove old log files

LOG_RETENTION

daemon

run_full_dump

Full backup of file system(s)

For restores, see dmxfsrestore(8)

DUMP_DEVICE

DUMP_INVENTORY_COPY

DUMP_FILE_SYSTEMS

DUMP_MIGRATE_FIRST

DUMP_RETENTION

DUMP_VSNS_USED

DUMP_TAPES

daemon

run_hard_deletes

Hard-delete files

Uses DUMP_RETENTION

daemon

run_merge_stop

Stop tape merges

msp

run_partial_dump

Partial backup of file system(s)

Uses parameters set for run_full_dump

daemon

run_scan_logs

Scan log files for errors

 

daemon

run_tape_merge

Merge sparse tapes

DATA_LIMIT

THRESHOLD

VOLUME_LIMIT

msp

run_tape_report

Create tape reports

 

msp


Setting PATH Environment Variables

To use DMF commands and DMF man pages, set your PATH and MANPATH environment variables. The DMF administrator commands and executables are installed in /etc/dmf/dmbase/etc; the user commands are installed in /etc/dmf/dmbase/bin. Man pages are installed in /etc/dmf/dmbase/man.


Note: If you are not familiar with setting the MANPATH environment variable, you should know that some paths are checked even though they are not listed by default. In other words, even though the command echo $MANPATH appears to indicate that no variable is defined (in ksh it returns no message or in csh it returns the message MANPATH - Undefined variable), certain paths are still searched for man pages. Setting the MANPATH environment variable as described below will supersede the fact that these paths are searched.

If MANPATH has not been set, you should read the man(1) man page to determined the paths that are checked and then include those paths in the commands below.

The following example uses sh syntax to set and display the DMF PATH environment variables:

# PATH=$PATH:/etc/dmf/dmbase/etc:/etc/dmf/dmbase/bin; export PATH
# MANPATH=$MANPATH:/etc/dmf/dmbase/man; export MANPATH
# env | grep PATH
MANPATH=/usr/man:/usr/share/catman:/usr/catman:/usr/local/man:/etc/dmf/dmbase/man
PATH=/usr/sbin:/usr/bsd:/sbin:/usr/bin:/bin:/etc:/usr/etc:/usr/bin/X11:/etc/dmf/dmbase/etc:
/etc/dmf/dmbase/bin

The following example uses csh syntax to set and display the DMF PATH environment variables:

% setenv PATH ${PATH}:/etc/dmf/dmbase/etc:/etc/dmf/dmbase/bin
% setenv MANPATH ${MANPATH}:/etc/dmf/dmbase/man
% env |grep PATH
MANPATH=/usr/man:/usr/share/catman:/usr/catman:/usr/local/man:/etc/dmf/dmbase/man
PATH=/usr/sbin:/usr/bsd:/sbin:/usr/bin:/bin:/etc:/usr/etc:/usr/bin/X11:/etc/dmf/dmbase/etc:
/etc/dmf/dmbase/bin

You can set the user command and man path names in the file /etc/profile for all ksh users and /etc/cshrc for all csh users, or provide a module for users.

Configuration Objects

The configuration file consists of configuration objects and parameters. The file uses seven types of configuration objects:

  • The base object, which defines path name and file size parameters necessary for DMF operation

  • The daemon object, which defines parameters necessary for dmdaemon(8) operation

  • The file system object, which define parameters necessary for migrating files in that file system

  • The policy objects, which specify parameters to determine MSP selection, automated space management policies, and/or file weight calculations in automatic space management

  • The MSP objects, which define parameters necessary for that MSP's operation

  • The device objects, which define parameters for DMF's use of tape devices

  • The taskgroup objects, which define parameters necessary for automatic completion of specific maintenance tasks

DMF configuration objects and parameters are also defined in the dmf_config(5) man page.

Configuring the Base Object

The base configuration parameters define path names and file sizes necessary for DMF operation. It is expected that you will modify the path names, although those provided will work without modification. All path names must be unique.

Parameter 

Description

TYPE 

base (type of object).

ADMIN_EMAIL 

Email address to which to send output from administrative tasks. The mail may include errors, warnings, and output from any configured tasks. You cannot specify a list of addresses, so if you want multiple recipients for this mail, you must set up an alias.

HOME_DIR 

Base path name used to construct directory names for DMF directories in which databases and related files reside. Generally referred to as HOME_DIR.

JOURNAL_DIR 

Base path name used to construct directory names for DMF directories in which the daemon and tape MSP database journal files will be written. To provide the best chance for database recovery, this directory should be a separate file system and a different physical device from HOME_DIR. Generally referred to as JOURNAL_DIR.

JOURNAL_SIZE 

Maximum size (in bytes) of the database journal file before DMF closes it and starts a new file.

LICENSE_FILE 

Full path name of the file containing the FLEXlm license used by DMF. The default is /usr/dmf/dmbase/flexlm/license.dat. You should have no need to edit this parameter.

OV_KEY_FILE 

File containing the OpenVault keys used by DMF. It is usually located in HOME_DIR and called ovkeys. There is no default. (Use this parameter only if you are using OpenVault as your tape mounting service.)

OV_SERVER 

Name returned by the hostname(1) command on the machine on which the OpenVault server is running. This parameter only applies when OpenVault is used as the mounting service. The default value is the host name of the machine on which you are running.

SPOOL_DIR 

Base path name used to construct the directory names for DMF directories in which DMF log files are kept. Generally referred to as SPOOL_DIR.

TMP_DIR 

Base path name used to construct the directory names for DMF directories in which DMF puts temporary files such as pipes. It is also used by scripts for temporary files and is the directory used by default by the tape MSP for caching files if the CACHE_DIR parameter is not defined. Generally referred to as TMP_DIR.


Warning: Do not change the directory names while DMF is running.

If you intend to run the OpenVault library management facility as the mounting service for DMF, you must configure the OV_KEY_FILE and OV_SERVER parameters. If you are running a different mounting service, you do not need these parameters. More configuration steps are necessary to configure DMF to use OpenVault; see “Using OpenVault for Tape MSPs”.

Procedure 2-3. Base Object Configuration

The following example defines a base object:

define  base
        TYPE            base
        ADMIN_EMAIL     [email protected]
        HOME_DIR        /dmf/home
        TMP_DIR         /tmp/dmf
        SPOOL_DIR       /dmf/spool/
        JOURNAL_DIR     /dmf/journals
        JOURNAL_SIZE    10m
        LICENSE_FILE    /var/flexlm/dmf_license.dat
        OV_KEY_FILE     /dmf/home/ovkeys
        OV_SERVER       localhost
enddef


Note: Do not use automated space management to manage the HOME_DIR, SPOOL_DIR, or JOURNAL_DIR directories because DMF daemon processes will deadlock if files that they are actively using within these directories are migrated. dmcheck(8) reports an error if any of the HOME_DIR, SPOOL_DIR, or JOURNAL_DIR parameters are also configured as DMF-managed file systems. Configure the daemon_tasks object to manage old log files and journal files in these directories (you can change the namedaemon_tasks to be anything you prefer). See “Configuring Daemon Maintenance Tasks”, for more information.


The following steps explain pertinent information for configuring the base object:

  1. Ensure that TYPE is set to base.

  2. Configure the email address specified by the ADMIN_EMAIL parameter to be the user to whom you want to send the output of the configured tasks described in “Configuring Automated Maintenance Tasks”.

  3. Configure the file system specified by the HOME_DIR configuration parameter (referred to as HOME_DIR) as a separate file system, and restrict its contents to DMF databases and relatively static files such as DMF scripts.

    DMF cannot run if HOME_DIR runs out of space, and such an event is more likely to happen if it is simply another directory in /usr.

  4. Set TMP_DIR to be any file system that can store temporary files. /tmp or a directory below /tmp is a common choice.

  5. Configure the log file directory (referred to as SPOOL_DIR) as a separate file system so that log file growth does not impact the rest of the system.

  6. Ensure that the journal file directory (referred to as JOURNAL_DIR) resides on a physical device completely separate from the one on which HOME_DIR resides. Backup copies of DMF databases should also be stored on the JOURNAL_DIR file system.

  7. Configure the JOURNAL_SIZE parameter to be the maximum size allowable for a journal file before DMF closes it.

  8. If you plan to run OpenVault, configure the OV_KEY_FILE parameter to be the name of the key file that holds security information for OpenVault. For more information, see Procedure 2-13.

  9. If you plan to run OpenVault, configure the OV_SERVER parameter to the name of the server that runs OpenVault. For more information, see Procedure 2-13.

Configuring the DMF Daemon

The daemon object defines configuration parameters necessary for dmdaemon operation. It is expected that you will modify the values for the path names and MSP names.

Parameter 

Description

TYPE 

dmdaemon (type of object)

MESSAGE_LEVEL 

Specifies the highest message level number that will be written to the daemon log. It must be an integer between 0 and 6; the higher the number, the more messages written to the log file. The default is 2. For more information on message levels, see “General Message Log File Format”.

MIGRATION_LEVEL 

Sets the highest level of migration service allowed on all DMF file systems (you can configure a lower service level for a specific file system). The value can be none (no migration), user (requests from dmput(1) or dmmigrate(8) only), or auto (automated space management). The default is auto.

MOVE_FS 

Names the scratch file system used by dmmove(8) to move files between MSPs. There is no default.

MSP_NAMES 

Names the MSPs used by this daemon. Necessary only if you have more than one MSP.

TASK_GROUPS 

Names the task groups that contain tasks the daemon should run. They are configured as objects of TYPE taskgroup. There is no default. For more information, see “Configuring Daemon Maintenance Tasks”.

SGI recommends that you use the task groups specified in the sample configuration file, changing the parameters as necessary for your site.

Procedure 2-4. Daemon Configuration

The following example defines a daemon object:

define  daemon
        TYPE                    dmdaemon
        MOVE_FS                 /move_fs
        MIGRATION_LEVEL         auto
        MSP_NAMES               cart1 cart2
        TASK_GROUPS             daemon_tasks dump_tasks
enddef

The following steps explain pertinent information for configuring the daemon object:

  1. Ensure that TYPE is set to dmdaemon. There is no default.

  2. If you have more than one MSP, ensure that the MOVE_FS parameter is set to a file system that can accept temporary files. This must be the root of a DMAPI file system. There is no default.

  3. The MIGRATION_LEVEL parameter determines the level of service for migration to offline media. Migration from offline media (either automatic or manual recall) is not affected by the value of MIGRATION_LEVEL.

    Configure MIGRATION_LEVEL to be none, user, or auto. This value is the highest level you want to allow anywhere in your DMF environment. You can configure a lower level for a specific file system. none means no migration will take place on any DMF file system. user means that users/administrators can perform dmput(1) or dmmigrate(8) commands and no other migration will take place. auto means that you want automated space management on at least one DMF file system. The default is auto. See “DMF Policies”, for information about configuring automated space management.

  4. Configure MSP_NAMES to be the names of the MSPs to be used by this daemon. You need to specify this parameter if you have more than one MSP. You will use these names when defining the MSP objects and in SELECT_MSP parameters within policies. See Procedure 2-10.

  5. Configure the TASK_GROUPS parameter to the name(s) of the object(s) used to define how periodic maintenance tasks are completed. In the example, daemon_tasks defines the tasks such as scanning and managing log files and journal files. The dump_tasks object defines tasks that back up DMF-managed file systems. You can change the object names themselves (dump_tasks and daemon_tasks) to be any name you like. There is no default value for the object. See “Configuring Daemon Maintenance Tasks” for more information.

Configuring Daemon Maintenance Tasks

You can configure daemon_tasks parameters to manage how the DMF daemon performs the following maintenance tasks:

  • Auditing databases (the run_audit.sh task)

  • Scanning recent log files for errors (the run_scan_logs.sh task)

  • Removing old log files (the run_remove_logs.sh task and the LOG_RETENTION parameter)

  • Removing old journal files (the run_remove_journals.sh task and the JOURNAL_RETENTION parameter)

  • Backing up DMF databases (the run_copy_databases.sh task and the DATABASE_COPIES parameter)

For each of these tasks, you can configure when the task should be run. For some of the tasks, you must provide more information such as destinations or retention times for output.

You can configure dump_tasks parameters to manage how the daemon completes the following tasks to back up the DMF-managed file systems:

  • Fully backing up DMF-managed file systems (the run_full_dump.sh task)

  • Partially backing up DMF-managed file systems (the run_partial_dump.sh task)

  • Hard-deleting files no longer on backup tape (the run_hard_deletes.sh task)

  • Managing the data from the file system dumps (the DUMP_TAPES, DUMP_RETENTION, DUMP_DEVICE, DUMP_MIGRATE_FIRST, DUMP_INVENTORY_COPY, DUMP_FILE_SYSTEMS, and DUMP_VSNS_USED parameters)

For each of these tasks, you can configure when the task is run. To manage the tapes, you must provide information such as tape and device names, retention times for output, whether to migrate files before dumping the file system, and locations for inventory files. Table 2-1, provides a summary of automated maintenance tasks.

Procedure 2-5. Configuring the daemon_tasks Object

The following steps explain how to define a daemon_tasks object. You can change the object name itself (daemon_tasks) to be any name you like.

Do not change the script names.

You may comment out the RUN_TASK parameters for any tasks you do not want to run.

The following example would configure a daemon_tasks object:

define  daemon_tasks
        TYPE                 taskgroup
        RUN_TASK             $ADMINDIR/run_audit.sh every day \
                                at 23:00
#
        RUN_TASK             $ADMINDIR/run_scan_logs.sh at 00:01
#
        RUN_TASK             $ADMINDIR/run_remove_logs.sh every \
                                day at 1:00
        LOG_RETENTION        4w
#
        RUN_TASK             $ADMINDIR/run_remove_journals.sh every \
                                day at 1:00
        JOURNAL_RETENTION    4w
#
        RUN_TASK             $ADMINDIR/run_copy_databases.sh \
                             every day at 3:00 12:00 21:00
        DATABASE_COPIES      /save/dmf_home /alt/dmf_home
enddef

  1. Define the object to have the same name that you provided for the TASK_GROUPS parameter of the daemon object. In the example it is daemon_tasks.

  2. Ensure that TYPE is set to taskgroup. There is no default.

  3. Configure the RUN_TASK parameters. DMF substitutes $ADMINDIR in the path with the actual etc/admin directory in the configured DMF directory (/etc/dmf/dmbase). When the task is run, it is given the name of the object that requested the task as the first parameter and the name of the task group (in this case daemon_tasks) as the second parameter. The task itself may use the dmconfig(8) command to obtain further parameters from either of these objects.

    All of the RUN_TASK parameters require that you provide a time_expression.

    The time_expression defines when a task should be done. It is a schedule expression that has the following form:

    [every n  period] [at hh:mm[:ss] ...] [on day ...]

    period is one of minute[s], hour[s], day[s], week[s], or month[s].

    n is an integer.

    day is a day of the month (1 through 31) or day of the week (sunday through saturday).

    The following are examples of valid time expressions:

    at 2:00
    every 5 minutes
    at 1:00 on tuesday

    Some of the tasks defined by the RUN_TASK parameters require more information. The following steps specify what you must provide.

    1. The run_audit.sh task runs dmaudit. For this task, provide a time_expression. If it detects any errors, the run_audit.sh task mails the errors to the email address defined by the ADMIN_EMAIL parameter of the base object (described in “Configuring the Base Object”).

    2. The run_scan_logs.sh task scans the DMF log files for errors. For this task, provide a time_expression. If the task finds any errors, it sends email to the email address defined by the ADMIN_EMAIL parameter of the base object.

    3. The run_remove_logs.sh task removes logs that are older than the value you provide by specifying the LOG_RETENTION parameter. You also provide a time_expression to specify when you want the run_remove_logs.sh to run. In the example, log files more than 4 weeks old are deleted each day at 1:00 A.M. Valid values for LOG_RETENTION are a number followed by m[inutes], h[ours], d[ays], or w[eeks].

      The run_remove_journals.sh task removes journals that are older than the value you provide by specifying the JOURNAL_RETENTION parameter. You also provide a time_expression to specify when you want the run_remove_journal.sh to run. In the example, journal files more than 4 weeks old are deleted each day at 1:00 A.M. Valid values for JOURNAL_RETENTION are a number followed by m[inutes], h[ours], d[ays], or w[eeks].


      Note: The run_remove_journals.sh and run_remove_logs.sh tasks are not limited to the daemon logs and journals; they also clear the logs and journals for MSP(s).


    4. The run_copy_databases.sh task makes a copy of the DMF databases. For this task, in addition to a value for time_expression, provide a value for the DATABASE_COPIES parameter that specifies one or more directories. If you specify multiple directories, breaking the directories among multiple disk devices minimizes the chance of losing all the copies of the database.

      The task copies a snapshot of the current DMF databases to the directory with the oldest copy. Integrity checks are done on the databases before the copy is saved. If the checks fail, the copy is not saved, and the task sends email to the email address defined by the ADMIN_EMAIL parameter of the base object.

Procedure 2-6. Configuring the dump_tasks Object

The following steps explain how to define a dump_tasks object. You can change the object name itself (dump_tasks) to be any name you like.

Do not change the script names.

You may comment out the RUN_TASK parameters for any tasks you do not want to run.

The following example would configure a dump_tasks object:

define  dump_tasks
        TYPE                  taskgroup
        RUN_TASK              $ADMINDIR/run_full_dump.sh on \
                                sunday at 00:01
        RUN_TASK              $ADMINDIR/run_partial_dump.sh on \
                                monday tuesday wednesday thursday \
                                friday saturday at 00:01
        RUN_TASK              $ADMINDIR/run_hard_deletes.sh
                                at 23:00
#
        DUMP_TAPES            HOME_DIR/tapes
        DUMP_RETENTION        4w
        DUMP_DEVICE           SILO_2
        DUMP_MIGRATE_FIRST    yes
        DUMP_INVENTORY_COPY   /save/dump_inventory
enddef

  1. Define the object to have the same name that you provided for the TASK_GROUPS parameter of the daemon object. In the example it is dump_tasks.

  2. Ensure that TYPE is set to taskgroup. There is no default.

  3. Configure the RUN_TASK parameters. See step 3 in Procedure 2-5, for information about $ADMINDIR and time_expression.

    The following steps specify the information you must provide for the tasks to run correctly.

    1. The run_full_dump.sh task runs a full backup of DMF-managed file systems at intervals specified by the time_expression. In the example, the full backup is run each week on Sunday morning one minute after midnight.

    2. The run_partial_dump.sh task backs up only those files in DMF-managed file systems that have changed since the time a full backup was completed. The backups are run at intervals specified by the time_expression. In the example, it is run each day of the week except Sunday, at one minute after midnight.

    3. The run_hard_deletes.sh task removes from the database any files that have been deleted but can no longer be restored because the backup tapes have been recycled (that is, it hard-deletes the files). The backup tapes are recycled at the time interval set by the DUMP_RETENTION parameter described in the next step. For more information on hard-deleting files, see “Soft- and Hard-deletes” in Chapter 7.

    4. Manage the data from the file system dumps by configuring the following parameters:

      DUMP_TAPES
      DUMP_RETENTION
      DUMP_DEVICE
      DUMP_MIGRATE_FIRST
      DUMP_INVENTORY_COPY
      DUMP_FILE_SYSTEMS
      DUMP_VSNS_USED

      The DUMP_TAPES parameter specifies the path of a file that contains tape volume serial numbers (one per line) for the dump tasks to use.

      The DUMP_RETENTION parameter specifies how long the backups of the file system will be kept before the tapes are reused. This is also the value used by the run_hard_deletes.sh task to determine how old soft-deleted database entries must be before removing them from the database. Valid values for DUMP_RETENTION are a number followed by m[inutes], h[ours], d[ays], or w[eeks].

      The DUMP_DEVICE parameter specifies the name of the device object in the configuration file that defines how to mount the tapes that the dump tasks will use. See “Device Objects”, for information about device objects.

      If you set DUMP_MIGRATE_FIRST to YES, the dmmigrate command is run before the dumps are done to ensure that all migratable files are migrated, thus reducing the tapes needed for the dump. The default is NO.

      The DUMP_INVENTORY_COPY parameter specifies the path name of a directory into which are copied the xfsdump(1M) inventory files for the backed-up file systems.

      The DUMP_FILE_SYSTEMS parameter specifies one or more file systems to dump. If not specified, the task dumps all the file systems configured in the configuration file. Use this parameter only if your site needs different dump policies (such as different dump times) for different file systems. It is safest not to specify a value for this parameter and therefore dump all file systems configured.

      The DUMP_VSNS_USED parameter is optional. It specifies the name of a file to which the tasks that dump the file systems will append the VSN, one per line, of each volume used by xfsdump. If you don't specify this parameter, the task uses /dev/null as the file name.

The dump_tasks object employs scripts that call the xfsdump(1m) command in conjunction with the dmtape DMF support program. This mechanism gives you flexible and efficient use of a predetermined set of backup volumes that are automatically allocated to the xfsdump program as needed during the backup. In order to allow you an equally flexible and efficient method for restoring files backed up by the dump_tasks object, the dmxfsrestore(8) command should be used any time a restore is required for a dump_tasks-managed file system. Please see the dmxfsrestore(8) man page for more information on running the command.

Configuring File Systems

You must have a filesystem object for each file system that can migrate files.

The filesystem object parameters are as follows:

Parameter 

Value

TYPE 

filesystem (type of object).

MESSAGE_LEVEL 

Specifies the highest message level number that will be written to the automated space management log (autolog). It must be an integer between 0 and 6; the higher the number, the more messages written to the log file. The default is 2. For more information on message levels, see “General Message Log File Format”.

MIGRATION_LEVEL 

Sets the level of migration service for the file system. Valid values are none (no migration), user (only user-initiated migration), or auto (automated space management). The migration level actually used for the file system is the lesser of the MIGRATION_LEVEL of the daemon object and this value. The default is auto.

POLICIES 

Specifies the names of the configuration objects defining policies for this file system.

TASK_GROUPS 

Names the task groups that contain tasks the daemon should run. They are configured as objects of TYPE taskgroup. There is no default. Currently there are no defined tasks for file systems.

Procedure 2-7. Configuring filesystem Objects

The following example defines a filesystem object:

define  /c
        TYPE                    filesystem
        MIGRATION_LEVEL         user
        POLICIES                fs_msp
enddef

The following steps explain pertinent information for configuring the above filesystem object:

  1. Ensure that define has a value that is the mount point of the file system you want DMF to manage. Do not use the name of a symbolic link. There is no default.

  2. Ensure that TYPE is set to filesystem. There is no default.

  3. The MIGRATION_LEVEL parameter determines the level of service for migration to offline media. Migration from offline media (either automatic or manual recall) is not affected by the value of MIGRATION_LEVEL.

    Configure MIGRATION_LEVEL to be one of none , user, or auto. none means no migration will take place on this file system. user means that users/administrators can perform dmput(1) or dmmigrate(8) commands but no other migration will take place. auto means that you want automated space management on this file system.

    The default is auto, which means that you do not need to include this line unless you want to specify user or none. See “DMF Policies” and Procedure 2-8, for information about configuring automated space management policies.


    Note: user is the highest migration level that can be associated with a real-time partition.


  4. The POLICIES parameter is used to declare one or more migration policies that will be associated with this file system. Policies are defined with policy objects (see “DMF Policies”). The POLICIES parameter is required; there is no default value. A policy can be unique to each DMF-managed file system, or it can be reused numerous times.

DMF Policies

A policy object is used to specify a migration policy. Three types of migration policies can be defined: automated space management, file weighting, and MSP selection.

The following rules govern the use of policy objects with the POLICIES parameter of the filesystem object:

  • The POLICIES parameter for a file system must specify one and only one MSP selection policy.

  • If the MIGRATION_LEVEL for a file system is auto, the POLICIES parameter for that file system must specify one and only one space management policy.

  • You do not need to specify a weighting policy if the default values are acceptable.

  • You can configure one policy that defines all three groups of policy parameters (space management, file weight, and MSP selection) and share that policy among all the file systems. Alternatively, you might create an MSP selection policy for all file systems and a space management policy (including weighting parameters) for all file systems.

The policy object parameters described below are grouped by function.

Automated Space Management Parameters

DMF lets you automatically monitor file systems and migrate data as needed to prevent file systems from filling. This capability is implemented in DMF with a daemon called dmfsmon(8). After the dmfsmon daemon has been initiated, it will begin to monitor the DMF-managed file system to maintain the level of free space configured (in the configuration file).

Chapter 3, “Automated Space Management”, describes automated space management in more detail.

The following are parameters that control automated space management on a file system:

Parameter

Description

TYPE

policy (type of object).

FREE_SPACE_MINIMUM

Minimum percentage of free file system space that dmfsmon maintains. dmfsmon will begin to migrate files when the available free space for the file system falls below this percentage value. This parameter is required; there is no default.

FREE_SPACE_TARGET

Percentage of free file system space that dmfsmon will try to achieve if free space reaches or falls below FREE_SPACE_MINIMUM. This parameter is required; there is no default.

MIGRATION_TARGET

Percentage of file system capacity that DMF maintains as a reserve of dual-state files whose online space can be freed if free space reaches or falls below FREE_SPACE_MINIMUM. dmfsmon tries to make sure that this percentage of the file system is migrated, migrating, or free after it runs to make space available. This parameter is required; there is no default.

FREE_SPACE_DECREMENT

Percentage of file system space by which dmfsmon will decrement FREE_SPACE_MINIMUM if it cannot find enough files to migrate so that the value is reached. The decrement is applied until a value is found that dmfsmon can achieve. If space later frees up, the FREE_SPACE_MINIMUM is reset to its original value. Valid values are between 1 and the value of FREE_SPACE_TARGET. The default is 2.

FREE_DUALSTATE_FIRST

When set to on, dmfsmon will free dual-state files before freeing files it will have to migrate first. The default is off.


Note: Ideal values for these parameters are highly site-specific, based largely on file system sizes and typical file sizes.



Note: The dump_tasks object employs scripts that call the xfsdump(1m) command in conjunction with the dmtape DMF support program. This mechanism gives you flexible and efficient use of a predetermined set of backup volumes that are automatically allocated to the xfsdump program as needed during the backup. In order to allow you an equally flexible and efficient method for restoring files backed up by the dump_tasks object, the dmxfsrestore(8) command should be used any time a restore is required for a dump_tasks-managed file system. Please see the dmxfsrestore(8) man page for more information on running the command.


File Weighting and MSP Selection Parameters

An important part of automatic space management is selecting files to migrate and determining where to migrate them. When DMF is conducting automated space management, it derives an ordered list of files, called a candidate list, and migrates or frees files starting at the top of the list. The ordering of the candidate list is determined by weighting factors that are defined by using weighting-factor parameters in the configuration file.

DMF can be configured to have many MSPs. Each MSP manages its own set of volumes. The MSP selection parameters allow you to direct DMF to migrate files with different characteristics to different MSPs.

The file weighting and MSP selection parameters can be used more than once to specify that different files should have different weighting or MSP selection values.

The policy parameters for file weighting are as follows:

Parameter 

Description

AGE_WEIGHT 

Specifies a floating point constant and floating point multiplier to use to calculate the weight given to a file's age. AGE_WEIGHT is calculated as constant + (multiplier * file_age_in_days). If DMF cannot locate values for this parameter, it uses a floating point constant of 1 and a floating point multiplier of 1.

SPACE_WEIGHT 

Specifies a floating point constant and floating point multiplier to use to calculate the weight given to a file's size. SPACE_WEIGHT is calculated as constant + (multiplier * file_disk_space_in_bytes). If DMF cannot locate values for this parameter, it uses a floating point constant of 0 and a floating point multiplier of 0.

The parameter for MSP selection follows:

Parameter 

Description

SELECT_MSP 

Specifies the MSP(s) to use for a file. You can list as many MSP names as you have MSP objects defined. A copy of the file will be migrated to each MSP listed. The special MSP name none means that the file will not be migrated. If you define more than one MSP, separate the names with white space. If no SELECT_MSP parameter applies to a file, it will not be migrated. The parameters are processed in the order they appear in the policy. There is no default.

The file weighting and MSP selection parameters accept an optional when clause to restrict the set of files to which that parameter applies.

This clause has the form when expression.

expression can include any of the following simple expressions:

Expression

Description

age

Days since last modification or last access of the file, whichever is more recent

space

Number of bytes the file occupies on disk (always a multiple of the blocksize, which may be larger or smaller than the length of the file)

gid

Group ID of one or more files

uid

User ID of one or more files

Combine expressions by using and, or, and ().

Use the operators =, >, <, =>, =<, and in to specify values.

The following are examples of valid expressions:

space < 10m            (space used is less than 10 million bytes)
uid <= 123             (file's user ID is less than or equal to 123)
gid = 55               (file's group ID is 55)
age >= 15              (file's age is greater than or equal to 15 days)
space > 1g             (space used is greater than 1 billion bytes)
uid in (10 82-110 200) (file's user ID is 10, between 82 and 110, or 200)
(gid = 55 or uid <= 123) and age < 5
                                          (file's age is greater than 5 days and its
                                          group ID is 55 or its user ID is higher than 123)

Configuring Policies

The following procedures explain how to create policies for automated space management (including file weighting) and MSP selection.

Procedure 2-8. Configuring Objects for Automated Space Management

The following example defines a policy object for automated space management:

define  fs_space
        TYPE                     policy
        MIGRATION_TARGET         50
        FREE_SPACE_TARGET        10
        FREE_SPACE_MINIMUM       5
        FREE_DUALSTATE_FIRST     off

        AGE_WEIGHT 0    0.00     when age < 10
        AGE_WEIGHT 1    0.01     when age < 30
        AGE_WEIGHT 10   0.05     when age < 120
        AGE_WEIGHT 50   0.1

        SPACE_WEIGHT  0  0
enddef

The following steps explain pertinent information for configuring the above policy object:

  1. Ensure that define has a value you set previously in the POLICIES parameter of a filesystem object. There is no default.

  2. Ensure that TYPE is set to policy. There is no default.

  3. Configure automated space management as follows:

    1. Configure MIGRATION_TARGET to an integer percentage of total file system space. DMF attempts to maintain this percentage as a reserve of space that is free or occupied by dual-state files that can be deleted if the file system free space reaches or falls below FREE_SPACE_MINIMUM. The default is 30.

    2. Configure FREE_SPACE_TARGET to an integer percentage of total file system space. DMF will try to achieve this level of free space when free space reaches or falls below FREE_SPACE_MINIMUM. The default is 20.

    3. Configure FREE_SPACE_MINIMUM to an integer percentage of the total file system space that DMF must maintain as free. DMF will begin to migrate files when the available free space for the configured file system reaches or falls below this percentage value. The default is 10.

    4. Configure FREE_DUALSTATE_FIRST to be on if you want DMF to free the space used by dual-state files before it migrates and frees regular files. The default is off.

  4. Configure the age and size weighting factors associated with a file when it is evaluated for migration as follows:

    1. The syntax of the AGE_WEIGHT parameter is a floating-point constant followed by a floating-point multiplier. The age weight is calculated as follows:

      constant + (multiplier x age_in_days)

      Add a when clause to select which files should use these values. DMF checks each AGE_WEIGHT parameter in turn, in the order they occur in the configuration file. If the when clause is present, DMF determines whether the file matches the criteria in the clause. If no clause is present, a match is assumed. If the file matches the criteria, the file weight is calculated from the parameter values. If they do not match, the next instance of that parameter is examined.

      An AGE_WEIGHT of 1 1.0 is used if no AGE_WEIGHT applies for a file.

      In the example policy, files that have been accessed or modified within the last 10 days have a weight of zero. File migration likelihood increases with the length of time since last access because the file will have a greater weight. The final line specifies that files which have not been accessed or modified in 120 days or more have a far greater weight than all other files.

    2. The syntax of SPACE_WEIGHT parameters is a floating-point constant followed by a floating-point multiplier. The space weight is calculated as follows:

      constant + (multiplier x file_disk_space_in_bytes)

      In the example policy, the size of the file does not affect migration because all files have SPACE_WEIGHT of zero.

      A SPACE_WEIGHT of 0 0.0 is used if no SPACE_WEIGHT applies for a file.

    3. Configure negative values to ensure that files are never automatically migrated. For example, you might want to set a minimum age for migration. The following parameter specifies that files that have been accessed or modified within 1 day are never automatically migrated:

      AGE_WEIGHT -1    0.0    when age <= 1
        

      The following parameter specifies that small files are never automatically migrated:
      SPACE_WEIGHT -1    0    when space <= 4k   


    Note: DMF calculates the size weight and age weight separately. If either value is less than zero, the file is not automatically migrated or freed. Otherwise, the two values are summed to form the file's weight.


Procedure 2-9. Configuring Objects for MSP Selection

The following example defines a policy object for MSP selection:

define  fs_msp
        TYPE                    policy
        SELECT_MSP none         when space < 65536
        SELECT_MSP cart1 cart2  when gid = 22
        SELECT_MSP cart1        when space >= 50m
        SELECT_MSP cart2
enddef

The following steps explain pertinent information for configuring the above policy object:


Note: The space expression references the number of bytes the file occupies on disk, which may be larger or smaller than the length of the file. For example, you might use the following line in a policy:
SELECT_MSP none when space < 4096



Your intent would be to restrict files smaller than 4 Kbytes from migrating.

However, this line may actually allow files as small as 1 byte to be migrated, because while the amount of data in the file is 1 byte, it will take 1 block to hold that 1 byte. If your file system uses 4-Kbyte blocks, the space used by the file is 4096, and it does not match the policy line.

To ensure that files smaller than 4 Kbytes do not migrate, use the following line:
SELECT_MSP none when space <= 4096



  1. Ensure that define has a value that you set previously in the POLICIES parameter of the filesystem object. There is no default.

  2. Ensure that TYPE is set to policy. There is no default.

  3. Ensure that the MSP name (or names) you specify as the first value of the SELECT_MSP parameter is a name you set previously in the MSP_NAMES parameter of the daemon object. There is no default.

  4. Configure MSP selection criteria as follows:

    1. If you want to select an MSP based on file size, use parameters such as the following, which send large files to cart1 and small files to cart2:

      SELECT_MSP cart1        when space >= 50m
      SELECT_MSP cart2        when space >= 65536

    2. If you want certain files to be copied to more than one MSP, use syntax such as the following, which migrates all files that have a group ID of 22 to both of the configured MSPs:

      SELECT_MSP cart1 cart2  when gid = 22

      Separate multiple MSP names with a blank space.

    3. If you want to ensure that some files are never migrated, you can designate the MSP selection as none. The following line from the sample file ensures that files smaller than 65,536 bytes are not migrated:

      SELECT_MSP none         when space < 65536

Setting up Tape MSPs

Each MSP you create must have an object defined in the configuration file.

The tape MSP entry has the following options, listed in the order in which they appear in the sample file:

Option 

Description

TYPE 

msp (type of object).

COMMAND 

Binary file to execute in order to initiate this MSP. For the tape MSP, this value must be dmatmsp.

CACHE_DIR 

Directory in which the MSP stores chunks while merging them from sparse tapes. If you do not specify this parameter, DMF uses the value of TMP_DIR from the base object.

CACHE_SPACE 

Amount of disk space (in bytes) that dmatmsp can use when merging chunks from sparse tapes. During merging, small chunks from sparse tapes are cached on disk before being written to a tape. The default is 0, which causes all files to be merged via sockets. For more information, see Step 5.

CHILD_MAXIMUM 

Maximum number of child processes the MSP is allowed to fork. The maximum value is 100; the default is 4.

DISK_IO_SIZE 

Transfer size (in bytes) used when reading from or writing to files within a DMF file system. The value must be between 4096 and 16 million (16m). The default is 65536.

HFREE_TIME 

Minimum number of seconds that a tape no longer containing valid data must remain unused before the MSP overwrites it. The default value is 172,800 seconds (2 days), and the minimum allowed value is 0.

When an MSP removes all data from a tape, it sets the hfree (hold free tape) flag bit in the tape's volume (VOL) database entry to prevent that tape from being immediately reused. The next time the MSP scans the database for volumes after HFREE_TIME seconds have passed, the MSP clears the hfree flag, allowing the tape to be rewritten. If HFREE_TIME is set to 0, the MSP will never clear hfree, so an unused tape will not be reused until you clear its hfree flag manually. See the dmvoladm man page for a description of how to set and clear the hfree flag manually.

MAX_CACHE_FILE 

Largest chunk (in bytes) that will be merged using the merge disk cache. Larger files are transferred directly via a socket from the read child to the write child. The default is 25% of the CACHE_SPACE value. Valid values are 0 through the value of CACHE_SPACE.

MAX_CHUNK_SIZE 

Specifies that the MSP should break up large files into chunks no larger than this value (specified in bytes) as it writes data to tape. If a file is larger than this size, it is broken up into pieces of the specified size, and, depending on other activity, more than one write child may be used to write the data to tape. If MAX_CHUNK_SIZE is 0 (the default) the MSP only breaks a file into chunks when an end of volume is reached.

MAX_PUT_CHILDREN 

Maximum number of write child processes the MSP will schedule. The default and the maximum are the value of CHILD_MAXIMUM; the minimum is 1.

MERGE_CUTOFF 

Limit at which the MSP will stop scheduling tapes for merging. This number refers to the sum of the active and queued children generated from gets, puts, and merges. The default is CHILD_MAXIMUM, which means that, if sparse tapes are available, children will be created until there are CHILD_MAXIMUM children, thus using tape efficiently. However, if any recall requests arrive, they will be started before new merges.

Setting this number below CHILD_MAXIMUM reserves some tape units for recalls as the expense of merge efficiency. Setting this number above CHILD_MAXIMUM increases the priority of merges relative to recalls.

MESSAGE_LEVEL 

Highest message level number that will be written to the MSP log. It must be an integer between 0 and 6; the higher the number, the more messages written to the log file. The default is 2. For more information on message levels, see “General Message Log File Format”.

MIN_TAPES 

Minimum number of unused tapes that can exist in the MSP VOL database before operator notification. If the number of unused tapes falls below MIN_TAPES, the operator will be asked to add new tapes. The default is 10; the minimum is 0.

TAPE_TYPE 

Specifies the name of a device object that describes how the tapes are accessed and used. There is no default. The device object is described in “Device Objects”.

TASK_GROUPS 

Names the task groups that contain tasks the MSP should run. They are configured as objects of TYPE taskgroup. There is no default. See “Configuring Tape MSP Maintenance Tasks”, for more information.

TIMEOUT_FLUSH 

Minutes after which the MSP will flush files to tape. The default is 120 minutes.

Procedure 2-10. Configuring Tape MSPs

The following procedure does not use all the possible options for configuring a tape MSP; it defines two tape MSPs named cart1 and cart2.

define  cart1
        TYPE                    msp
        COMMAND                 dmatmsp
        TAPE_TYPE               SILO_1
        CACHE_SPACE             110m
        CHILD_MAXIMUM           3
        MESSAGE_LEVEL           2
        TASK_GROUP              msp_tasks
enddef
#
define  cart2
        TYPE                    msp
        COMMAND                 dmatmsp
        TAPE_TYPE               SILO_2
        CACHE_SPACE             50m
        CACHE_DIR               /cache
        MAX_CACHE_FILE          50m
        CHILD_MAXIMUM           10
        TASK_GROUP              msp_tasks
enddef

The following steps explain pertinent information for configuring the msp objects:

  1. Ensure that define has a value that you set previously in the MSP_NAMES parameter of the daemon object. There is no default.

  2. Ensure that TYPE is set to msp. There is no default.

  3. Ensure that COMMAND is set to dmatmsp. There is no default.

  4. Define a TAPE_TYPE parameter that names the device type object for the MSP. There is no default. Use the value you set here in defining device objects. See “Device Objects”.

  5. Configure the CACHE_SPACE parameter to be at least twice the configured tape zone size. If you do not set this parameter, DMF will merge tapes via sockets, which means that the read and write children have to synchronize. Using CACHE_SPACE is far more efficient, especially for small files.

    The MSP is able to merge tapes more efficiently if it can stage most of the files to disk. Setting the CACHE_SPACE parameter tells the MSP how much disk space it can use. The MAX_CACHE_FILE parameter specifies the largest file it will place in the CACHE_SPACE. The default for CACHE_SPACE is 0, which causes all data to be transferred by sockets.

    See “Media Concepts” in Chapter 6, for more information on tape zone sizes.

  6. Configure the CHILD_MAXIMUM to be the number of tape drives this MSP can use. The default is 4, and the maximum is 24.

  7. Configure the MESSAGE_LEVEL of an MSP to be higher than 2 (the default) for debugging purposes only. Valid values are 0 to 6.

  8. Configure the MAX_CACHE_FILE to be the size (in bytes) of the largest chunk that will be merged using the merge cache space (defined by CACHE_SPACE). Large files are transferred directly via socket. The largest value you can use is the value of CACHE_SPACE, and the default is 25% of CACHE_SPACE.

  9. Configure the TASK_GROUPS parameter to the name(s) of the object(s) used to define how periodic maintenance tasks are completed. There is no default. See “Configuring Tape MSP Maintenance Tasks”, for more information.

Configuring Tape MSP Maintenance Tasks

You can configure parameters for how the tape MSP daemon performs the following maintenance tasks:

  • Creating tape reports (the run_tape_report.sh task)

  • Merging sparse tapes (the run_tape_merge.sh task and the THRESHOLD, VOLUME_LIMIT, and DATA_LIMIT parameters)

  • Stopping tape merges at a specified time (the run_merge_stop.sh task)

For each of these tasks, you can configure when the task is run. For the second task, you must provide more information such as what determines that a tape is sparse and how many tapes can be merged at one time.


Note: The run_remove_journals.sh and run_remove_logs.sh tasks are configured as part of the daemon_tasks object, but these tasks also clear the MSP logs and journals. These tasks are described in “Configuring Daemon Maintenance Tasks”.

Table 2-1, provides a summary of automated maintenance tasks.


Procedure 2-11. Configuring the msp_tasks Object

The following steps explain how to define the msp_tasks object. You can change the object name itself (msp_tasks) to be any name you like.

Do not change the path names or task names.

You may comment out the RUN_TASK parameters for any tasks you do not want to run.

define	msp_tasks
	TYPE			taskgroup
	RUN_TASK	$ADMINDIR/run_tape_report.sh at 00:10
#
	RUN_TASK	$ADMINDIR/run_tape_merge.sh on \
                   monday wednesday friday at 2:00
THRESHOLD          50
#VOLUME_LIMIT      20
#DATA_LIMIT        5g
#
RUN_TASK $ADMINDIR/run_merge_stop.sh at 5:00

  1. Define the object to have the same name that you provided for the TASK_GROUPS parameter of the tape msp object. In the example it is msp_tasks.

  2. Ensure that TYPE is set to taskgroup. There is no default.

  3. Configure the RUN_TASK parameters. DMF substitutes $ADMINDIR in the path with the actual etc/admin directory in the configured DMF directory (/etc/dmf/dmbase). When the task is run, it is given the name of the object that requested the task as the first parameter and the name of the task group (in this case msp_tasks) as the second parameter. The task itself may use the dmconfig(8) command to obtain further parameters from either of these objects.

    The RUN_TASK parameters require that you provide a time_expression.

    The time_expression defines when a task should be done. It is a schedule expression that has the following form:

    [every n  period] [at hh:mm[:ss] ...] [on day ...]

    period is one of minute[s], hour[s], day[s], week[s], or month[s].

    n is an integer.

    day is a day of the month (1 through 31) or day of the week (sunday through saturday).

    The following are examples of valid time expressions:

    at 2:00
    every 5 minutes
    at 1:00 on tuesday

    The following steps specify the information you must provide for the tasks to run correctly.

    1. The run_tape_report.sh generates a report on the tapes in the MSP tape pool and on MSP activity. In the example, it runs every day at 10 minutes after midnight.

    2. The run_tape_merge.sh task merges sparse tapes. You can specify the criteria that DMF uses to determine that a tape is sparse, as follows:

      • Use the THRESHOLD parameter to set an integer percentage of active data on a tape. DMF will consider a tape to be sparse when it has less than this percentage of data that is still active.

      • Use the VOLUME_LIMIT parameter to set the maximum number of tape volumes that can be selected for merging at one time.

      • Use the DATA_LIMIT parameter to set the maximum amount of data (in bytes) that should be selected for merging at one time.

    3. The run_merge_stop.sh task shuts down volume merging (tape merging) at a time you specify by using a time_expression. This task is an alternative to using the VOLUME_LIMIT and DATA_LIMIT parameters to stop merging at specified points. In the example, the limit parameters are commented out because run_merge_stop.sh is used to control volume merging.

Device Objects

Each tape device type name you use in the MSP or in the dump_tasks object should be defined as a device object in the configuration file. The parameters you define are based on which mounting service you intend to use.

The following parameters are common to all device objects:

Parameter 

Description

TYPE 

device (type of object).

BLOCK_SIZE 

Block size used when writing tapes from the beginning. The default depends upon the device, with DMF setting defaults as follows:

AMPEX DIS/DST       1199840
DLT                 131072
STK 9840            126796
Other devices       65536

LABEL_TYPE 

Label type used when writing tapes from the beginning. Possible values are nl (no label), sl (standard label, for IBM tapes), and al (ANSI label). The default is al.

MOUNT_SERVICE 

Specifies the mounting service to use. Supported values are openvault and tmf. This parameter is required; there is no default.

POSITIONING 

Specifies how the tape should be positioned to a zone; either skip or direct. skip specifies the use of tape mark skipping. direct specifies the use of block ID seek capability if the block ID is known. The default is direct.

POSITION_RETRY 

Level of retry in the event of a failure during zone positioning; one of none, lazy, or aggressive. lazy specifies that the MSP will retry if a reasonably fast alternative method of positioning is available. aggressive specifies that the MSP may try more costly and time-consuming alternatives. If the MSP is unable to position to a zone, the MSP aborts all recalls for files with data in that zone (however, DMF does not abort them if a copy exists in another MSP). The default is lazy.

VERIFY_POSITION 

Specifies whether the tape MSP write child should (prior to writing) verify that the tape is correctly positioned and that the tape was properly terminated by the last use. The default is to verify. Specifying no or off turns verification off; anything else ensures verification.

WRITE_CHECKSUM 

Specifies that tape block should be checksummed before writing. If a tape block has a checksum, it is verified when read. The default is on.

ZONE_SIZE 

Specifies approximately how much data the write child should put in a zone. The write child adds files and chunks to a zone until the data written equals or exceeds this value, at which time it writes a tape mark and updates the database. Smaller values allow faster recalls and better recoverability but poorer write performance. The MSP also uses zone size to determine when to start write children. The default is 50 MB.

Device Objects for OpenVault As Mounting Service

The device object may have the following parameters when it is configured for OpenVault:

Parameter 

Description

OV_ACCESS_MODES 

Specifies a list of access mode names that control how data is written to tape. The default value is readwrite when migrating and readonly when recalling. This parameter is optional.

OV_INTERCHANGE_MODES 

Specifies a list of interchange mode names that control how data is written to tape. This can be used to control whether the device compresses data as it is written. This optional parameter is applied when a tape is mounted or rewritten.

Examples of the use of these parameters are provided in Procedure 2-13.

OpenVault requires several configuration steps in addition to configuring the device object. They are described in “Using OpenVault for Tape MSPs”.

Device Objects for TMF as Mounting Service

Tape mounting can be accomplished by using the Tape Management Facility (TMF). To use TMF as a mounting service, there are no required parameters that you must specify, but the TMF_TMMNT_OPTIONS parameter allows you to specify some tmmnt options:

Parameter 

Description

TMF_TMMNT_OPTIONS 

Specifies command options that should be added to the tmmnt command when mounting a tape.

DMF uses the -Z option to tmmnt, so options controlling block size and label parameters are ignored. Use the BLOCK_SIZE and LABEL_TYPE device parameters instead.

Use -g if the group name is different from the device object's name. Use -i to request compression.

Procedure 2-12. Configuring Devices for TMF

The following example defines a device object for use with TMF:

define  SILO_3
        TYPE                    device
        MOUNT_SERVICE           tmf
        BLOCK_SIZE              131072
        LABEL_TYPE              sl
        TMF_TMMNT_OPTIONS       -g DLT
enddef

The following steps explain pertinent information for configuring the device object for TMF:


Note: DMF uses the -Z option to tmmnt, so options controlling block size and label parameters would be ignored if you were to specify them for the TMF_TMMNT_OPTIONS parameter. Use the BLOCK_SIZE and LABEL_TYPE device parameters instead.


  1. Ensure that define has a value that you set previously in the TAPE_TYPE parameter of the msp object. There is no default.

  2. Ensure that TYPE is set to device. There is no default.

  3. Configure the MOUNT_SERVICE to be tmf.

  4. Configure the BLOCK_SIZE parameter to be the block size used when writing tapes from the beginning. In the example, 131072 is used because DLTs write more efficiently with this blocksize.

  5. Configure the LABEL_TYPE parameter to be the label type used when writing tapes from the beginning. In the example, sl is used to specify standard label for IBM tapes.

  6. Configure the TMF_TMMNT_OPTIONS parameter to specify command options that should be added to the tmmnt command when mounting a tape. In the example, the -g option specifies that the TMF tape group is DLT. If this option on this parameter had not been specified, DMF would have used the name of this device object (in the example, SILO_3).

Using OpenVault for Tape MSPs

This section describes the steps you must take to configure OpenVault for a tape MSP. You must execute OpenVault commands, create security key files, and edit the DMF configuration file.

Procedure 2-13. Configuring DMF to Use OpenVault

The following procedure describes how to make OpenVault and DMF work together. The OpenVault setup script can be used to enable the DMF application. See the OpenVault Operator's and Administrator's Guide for a description of this script.


Note: The example that follows assumes that before you complete the steps described, the OpenVault server is configured and all drives and libraries are configured and OpenVault is running.


  1. On the OpenVault server, add DMF as both a privileged and unprivileged OpenVault application for this host. To do this, use the setup script, menu item 1, submenu 5.

  2. Add the DMF application as a valid user to appropriate drive groups. It is preferable that you use the OpenVault setup script, menu item 2, submenu 7. If for some reason you cannot use the setup script, you can enter the command manually, as follows:

    ov_drivegroup -a drive_group -A dmf

  3. Add DMF as a valid application to appropriate tape groups. It is preferable that you use the OpenVault setup script, menu item 2, submenu 8. You can enter the command manually, as follows:

    ov_cartgroup -a tape_group -A dmf

  4. Configure the base object for use with OpenVault:

    define  base
            TYPE            base
            HOME_DIR        /dmf/home
    .
    .
    .
            OV_KEY_FILE     /dmf/home/ovkeys
            OV_SERVER       hostname
    enddef

    1. Configure the OV_KEY_FILE parameter name of the key file that holds security information for OpenVault. It is usually located in HOME_DIR and called ovkeys.

    2. Configure the OV_SERVER parameter to the value returned by the hostname(1) command on the machine on which the OpenVault server is running. This parameter only applies when OpenVault is used as the mounting service. The default value is the host name of the machine on which you are running.

  5. Use the dmov_keyfile(8) command to create the file defined by the OV_KEY_FILE parameter. This command will prompt you for the privileged and unprivileged keys that you defined in step 1.

  6. Configure the device object for use with OpenVault:

    define  timber
            TYPE                    device
            MOUNT_SERVICE           openvault
            OV_ACCESS_MODES         readwrite
            OV_INTERCHANGE_MODES    compression
            ZONE_SIZE               200m
    enddef

    1. Ensure that define has a value that you set previously in the TAPE_TYPE parameter of the msp object. There is no default.

    2. Configure TYPE to be device. There is no default.

    3. Configure the MOUNT_SERVICE parameter to be openvault.

    4. Configure the OV_ACCESS_MODES parameter to be a list of access mode names that control how the tape is used. The parameter is optional. The default value is readwrite when migrating and readonly when recalling. Use this parameter to force readwrite.

      The other possible values that OpenVault can use are not configurable in DMF: for rewind/norewind, DMF uses rewind; for variable/fixed, DMF uses variable.

    5. Configure the OV_INTERCHANGE_MODES parameter to be a list of interchange mode names that control how data is written to tape. This can be used to control whether the device compresses data as it is written. This parameter is optional.

      To specify that you want data compressed, use OV_INTERCHANGE_MODES compression

      To force all tapes to be written as DLT4000, use OV_INTERCHANGE_MODES DLT4000

      This parameter is applied when a tape is first used or rewritten.

    6. Configure other parameters relevant to your site. The example sets the ZONE_SIZE parameter to 200 MB. The target zone size is a major factor in determining how much data is written before writing a tape mark and updating the MSP database. Here, the tapes used by the SILO_2 MSP will, in general, have more data written in a zone than DMF uses as a default. Smaller values allow faster recalls and better recovery, but they cause poorer write performance than larger values. The default is 50 MB. See “Media Concepts” in Chapter 6, for more information on how tape zone sizes are determined.

  7. Make the appropriate cartridges accessible to the MSPs by assigning the cartridges to the DMF application in OpenVault. To do this, you must know the following:

    • Cartridge type name. To determine the cartridge types allowed by a given drive, enter the following:

      ov_stat -c -D  drive | grep base

      The fourth column shown in the output is the cartridge type.

    • Cartridge group. To determine the possible cartridge groups, enter the following:

      ov_cartgroup -l -A dmf

    1. If you already have tapes defined in your MSP database, tell OpenVault about these tapes by entering the following:

      dmov_makecarts -g  cartgroup -t  carttype  msp

    2. If there are unmanaged cartridges in an OpenVault managed library, you can import the unmanaged cartridges, assign them to DMF, and add them to an MSP database by entering the following:

      dmov_loadtapes -l library -g  cartgroup -t carttype  msp

      This command will invoke a vi(1) session. In the vi(1) session, delete any cartridges that you do not want added to the MSP.

    3. If neither of the above cases are appropriate, you can manually configure the cartridges. The following commands can be useful in this effort:

      • To list cartridges in a library, enter the following:

        ov_stat -s -L library

      • To list information on cartridges known to OpenVault, enter the following:

        ov_lscarts -f '.*'

      • To import cartridges into OpenVault and optionally assign them to DMF use the ov_import command.

      • To assign a cartridge known to OpenVault to an application, use the ov_vol command with the -n option.

MSP Database Records

After you have added the tape MSP information to the configuration file, use the dmvoladm(8) command with the -m option to create any missing directories with the proper labels and to create the volume (VOL) and catalog (CAT) records in the MSP database.

You can follow the steps in Procedure 2-14 for all the tape MSPs you have defined.


Caution: Each tape MSP must have a unique set of volume serial numbers.


Procedure 2-14. Creating MSP Database Records

The following procedure is shown as an example that assumes you have an MSP called cart1.

  1. If you have not yet done so, set your PATH environment variable to include /etc/dmf/dmbase/etc. ( See “Setting PATH Environment Variables”.)

  2. Enter the following command and it will respond as shown:

    % dmvoladm -m cart1
    dmvoladm: at rdm_open - created database atmsp_db
    adm: 1>
    The response is an informational message indicating that dmvoladm could not open an existing MSP databases, so it is creating a new and empty one. You should get this message the first time you use dmvoladm for an MSP, but never again. The next line is the prompt for dmvoladm directives.

  3. Assume that you will use 200 tapes of type CART with standard labels PA0001 through PA0200.

    After the prompt, enter the following directive:

    adm:1> create PA0001-PA0200

  4. After entering this directive, you will receive 200 messages, one for each entry created, beginning with the following:

    VSN PA0001 created.
    VSN PA0002 created.

  5. Use the following dmvoladm directive to list all of the tape VSNs in the newly created library:

    adm:2> list all

    Note:: The dmvoladm tapesize field is purely for site documentation and is not used by the MSP. The blocksize field documents the value used when the tape is first written or rewritten. It should not be changed in the database; however, if you want another value, change the BLOCK_SIZE nnn configuration parameter of the device object.


  6. Issue the dmvoladm quit directive to complete setting up the MSP.

    adm:3> quit

Setting up FTP MSPs

To enable a file transfer protocol (FTP) MSP, include a name for it on the MSP_NAMES parameter in the daemon object and define an msp object for it in the DMF configuration file.

DMF has the capability to use an FTP MSP to convert a non-DMF file server to DMF with a minimal amount of down time for the switch over, and at site-determined pace. Contact your customer service representative for information about technical assistance with file server conversion.

An FTP MSP object has the following options (defaults are provided here or in Procedure 2-16):

Parameter 

Description

TYPE 

msp (type of object).

COMMAND 

Binary file to execute in order to initiate this MSP. For the FTP MSP, this value must be dmftpmsp.

CHILD_MAXIMUM 

Maximum number of child processes the MSP is allowed to fork. The default is 4; the maximum is 100.

DISK_IO_SIZE 

Transfer size (in bytes) used when reading from or writing to files within a DMF file system. The value must be between 4096 and 16 million (16m). The default is 65536.

FTP_ACCOUNT 

Account ID to use when migrating files to the remote system.

FTP_COMMAND 

Additional commands to send to the remote system. There may be more than one instance of this parameter.

FTP_DIRECTORY 

Directory to use on the remote system.

FTP_HOST 

Internet host name of the remote machine on which files are to be stored.

FTP_PASSWORD 

File containing the password to use when migrating files to the remote system. This file must be owned by root and be only accessible by root.

FTP_PORT 

Port number of the FTP server on the remote system. The default value is the value configured for ftp in the services file.

FTP_USER 

User name to use when migrating files to the remote system.

GUARANTEED_DELETES 

Number of child processes that are guaranteed to be available for processing delete requests. If CHILD_MAXIMUM is nonzero, its value must be greater than the sum of GUARANTEED_DELETES and GUARANTEED_GETS. The default is 1.

GUARANTEED_GETS 

Number of child processes that are guaranteed to be available for processing dmget(1) requests. If CHILD_MAXIMUM is nonzero, its value must be greater than the sum of GUARANTEED_DELETES and GUARANTEED_GETS. The default is 1.

IMPORT_ONLY 

Specifies that the MSP is used for importing only. Set this parameter ON when the data is stored as a bit-for-bit copy of the file and needs to be available to DMF as part of a conversion. The MSP will not accept dmput(1) requests when this parameter is enabled. The MSP will, by default, ignore hard-delete requests when this parameter is enabled.

When the DMF daemon recalls a file from an IMPORT_ONLY MSP, it makes the file a regular file rather than a dual-state file, and it soft-deletes the MSP's copy of the file.

IMPORT_DELETE  

Specifies if the MSP should honor hard-delete requests from the DMF daemon. This parameter applies only if IMPORT_ONLY is set to on. Set IMPORT_DELETE to on if you wish files to be deleted on the destination system when hard deletes are processed.

NAME_FORMAT 

Remote file name template that creates names for files stored on remote machines. The default is username/bfid (the bfid is the full bfid in hexadecimal).

MESSAGE_LEVEL 

Specifies the highest message level number that will be written to the MSP log. It must be an integer between 0 and 6; the higher the number, the more messages written to the log file. The default is 2. For more information on message levels, see “General Message Log File Format”.

MVS_UNIT 

Defines the storage device type on an MVS system. This must be specified when the destination is an MVS system. Valid values are 3330, 3350, 3380, and 3390.

TASK_GROUPS 

Names the task groups that contain tasks the MSP should run. They are configured as objects of TYPE taskgroup. There is no default. Currently there are tasks defined only for the tape MSP.

The MSP checks the DMF configuration file just before it starts child processes. If the DMF configuration file changed, it is reread.

If CHILD_MAXIMUM is non-zero, its value must be greater than the sum of GUARANTEED_DELETES and GUARANTEED_GETS.

The parameters COMMAND, FTP_HOST, FTP_USER, FTP_PASSWORD, and FTP_DIRECTORY must be present.

The MVS_UNIT parameter affects only IBM machines; they are further described in the dmf_config(5) man page.


Note:: The MSP will not operate if the FTP_PASSWORD file is readable by anyone other than root.

The default value for NAME_FORMAT creates a unique file name and a subdirectory on the remote machine. The subdirectory is named after the file's owner at the time of migration. This default works well if the remote machine runs an operating system based on UNIX. The default may not work at all if the remote machine runs an operating system that is not based on UNIX. The unique file name is the encoded bit-file identifier (bfid) of the file.

Possible substitutes you may specify to create the NAME_FORMAT file name are as follows:

%1 substitutes for the first 32 bits of the bfid in hexadecimal
%2 substitutes for the second 32 bits of the bfid in hexadecimal
%3 substitutes for the third 32 bits of the bfid in hexadecimal
%4 substitutes for the fourth 32 bits of the bfid in hexadecimal
%b substitutes for the full bfid in hexadecimal
%u substitutes for the user name of the file owner
%U substitutes for the user ID of the file owner
%g substitutes for the group name of the file
%G substitutes for the group ID of the file
%% substitutes for the literal % character

The %1, %2, %3, %4, and %b substitutions generate uppercase hexadecimal numbers. The NAME_FORMAT must include either %b or %2, %3, %4 in some combination.

Procedure 2-15. Configuring the ftp Object

The following example defines an FTP MSP:

define  ftp
        TYPE                    msp
        COMMAND                 dmftpmsp
        FTP_HOST                fileserver
        FTP_USER                dmf
        FTP_ACCOUNT             dmf.disk
        FTP_PASSWORD            /dmf/ftp/password
        FTP_DIRECTORY           ftpmsp
        FTP_COMMAND             umask 022
enddef

The following steps explain pertinent information for configuring the ftp object:

  1. Ensure that define has a value that you set previously in the MSP_NAMES parameter of the daemon object. There is no default.

  2. Ensure that TYPE is set to msp. There is no default.

  3. Ensure that COMMAND is set to dmftpmsp. There is no default.

  4. Set the FTP_USER parameter to the user name to use on the remote FTP server during session initialization. There is no default.

  5. Set the FTP_ACCOUNT parameter (if necessary) to the account to use on the remote FTP server during session initialization. Most FTP servers do not need account information. When account information is required, its nature and format will be dictated by the remote machine and will vary from operating system to operating system. There is no default.

  6. Set the FTP_PASSWORD parameter to the name of the file containing the password to be used on the remote FTP server during session initialization. This file must be owned by root and only be accessible by root. In the example, the password for the user dmf on fileserver is stored in the file /dmf/ftp/password. There is no default.

  7. Set the FTP_DIRECTORY parameter to the directory into which files will be placed on the remote FTP server. There is no default.

  8. If necessary, specify commands to the remote machine's FTP daemon. In the example, the umask for files created is set to 022 (removes write permission for group and other). There is no default.

Setting up Disk MSPs

To enable a disk MSP, include a name for it on the MSP_NAMES parameter in the daemon object and define an msp object for it in the DMF configuration file.

As with the FTP MSP, you can use a disk MSP to convert a non-DMF file server to DMF with a minimal amount of down time for the switch over, and at a site-determined pace. Contact your customer service representative for information about technical assistance with file server conversion.

A disk MSP object has the following options:

Parameter 

Description

TYPE 

msp (type of object).

COMMAND 

Binary file to execute in order to initiate this MSP. For the disk MSP, this value must be dmdskmsp.

CHILD_MAXIMUM 

Maximum number of child processes the MSP is allowed to fork. The default is 4; the maximum is 100.

DISK_IO_SIZE 

Transfer size (in bytes) used when reading from or writing to files within a DMF file system. The value must be between 4096 and 16 million (16m). The default is 65536.

GUARANTEED_DELETES 

Number of child processes that are guaranteed to be available for processing delete requests. The default is 1.

GUARANTEED_GETS 

Number of child processes that are guaranteed to be available for processing dmget(1) requests. The default is 1.

IMPORT_DELETE  

Applies only if IMPORT_ONLY is set to on. Set IMPORT_DELETE to on if you wish files to be deleted in STORE_DIRECTORY when hard deletes are processed.

IMPORT_ONLY 

MSP is used for importing only. Set this parameter on when the data is stored as a bit-for-bit copy of the file and needs to be available to DMF as part of a conversion. The MSP will not accept dmput(1) requests when this parameter is enabled. The MSP will, by default, ignore hard delete requests when this parameter is enabled.

MESSAGE_LEVEL 

Specifies the highest message level number that will be written to the MSP log. It must be an integer between 0 and 6; the higher the number, the more messages written to the log file. The default is 2. For more information on message levels, see “General Message Log File Format”.

NAME_FORMAT 

Template that creates names for files in STORE_DIRECTORY. The default is username/bfid (the bfid is the full bfid in hexadecimal).

STORE_DIRECTORY 

Specifies the directory used to store files for this MSP.

TASK_GROUPS 

Names the task groups that contain tasks the MSP should run. They are configured as objects of TYPE taskgroup. There is no default. Currently there are tasks defined only for the tape MSP.

The default value for NAME_FORMAT creates a unique file name and a subdirectory in the STORE_DIRECTORY. The subdirectory is named after the file's owner at the time of migration. The unique file name is the encoded bit-file identifier of the file.

Procedure 2-16. Configuring the dsk Object

The following example describes setting up a disk MSP:

define  dsk
        TYPE                    msp
        COMMAND                 dmdskmsp
        CHILD_MAXIMUM           8
        GUARANTEED_DELETES      3
        GUARANTEED_GETS         3
        STORE_DIRECTORY         /remote/dir
enddef

The following steps explain pertinent information for configuring the dsk object:

  1. Ensure that define has a value that you set previously in the MSP_NAMES parameter of the daemon object. There is no default.

  2. Ensure that TYPE is set to msp. There is no default.

  3. Ensure that COMMAND is set to dmdskmsp. There is no default.

  4. Set the CHILD_MAXIMUM parameter to the maximum number of child processes you want this MSP to be able to fork. The default is 4. The example allows 8.

  5. Set the GUARANTEED_DELETES parameter to the number of child processes that are guaranteed to be available for processing delete requests. The default is 1. The example allows 3.

  6. Set the GUARANTEED_GETS parameter to the number of child processes that are guaranteed to be available for processing dmget requests. The default is 1. The example allows 3.

  7. Set the STORE_DIRECTORY to the directory where files will be stored. This parameter is required; there is no default.

Verifying the Configuration

To verify the DMF configuration, run the dmcheck(8) script. This command checks the configuration file object and parameters, and reports on inconsistencies.

Initializing DMF

The DMF daemon database is created in HOME_DIR/daemon_name as dbrec.dat, dbrec.keys, pathseg.dat, and pathseg.keys. The database definition file (in the same directory) that describes these files and their record structure is named dmd_db.dbd. The database journal file is named dmd_db.yyyymmdd.[hhmmss]. It is created in the directory JOURNAL_DIR/daemon_name (JOURNAL_DIR is specified by the JOURNAL_DIR configuration parameter).

The dmmaint(8) utility sets up system startup and shutdown scripts to start and stop DMF. You can start the DMF daemon manually by executing the dmdaemon command and stop it by executing the dmdstop(8) command.

After dmdaemon is activated, the dmget(1) and dmput(1) user commands can be used to manage file system space manually.

General Message Log File Format

The dmdaemon, dmlockmgr, dmfsmon, and MSP processes all create message files that are used to track various DMF events. These DMF message log files use the same general naming convention and message format. The message log file names are created using the extension .yyyymmdd, which represents the year, month, and day of log file creation.

Each line in a message log file begins with the time the message was issued, an optional message level, the process ID number, and the name of the program that issued the message.

The optional message level is described below. The remainder of the line contains informative or diagnostic information. The following sections provide details about each of these log files:

Messages in the dmdlog, dmlocklog, and msplog files contain a 2-character field immediately following the time field in each message that is issued. This feature helps to categorize the messages and can be used to extract error messages automatically from these logs. Because the only indication of DMF operational failure may be messages written to the DMF logs, recurring problems can go undetected if you do not check the logs daily.

Possible message types for autolog, dmdlog, msplog, and dmlocklog are defined as follows; the corresponding message level in the configuration file is also provided:

Table 2-2. DMF Log File Message Types

Field

Message type

Message level

E

Error

0

O

Ordinary

0

I

Informative

1

V

Verbose

2

1

Debug level 1

3

2

Debug level 2

4

3

Debug level 3

5

4

Debug level 4

6