DMF transparently moves file data from high-performance but expensive disk to levels of decreased-performance but inexpensive media known as secondary storage. This lets you cost-effectively maintain a seemingly infinite amount of data without sacrificing accessibility for users.

As the filesystems fill, DMF frees the data blocks of the least-recently accessed files on MIS disk, thereby always keeping space free for new files and recalled files. During the time that file data resides on both the MIS disk and the secondary storage, it will be returned to the user immediately from MIS disk; after its disk data blocks are freed, it will be recalled from secondary storage after a delay (depending upon the secondary-storage characteristics). Regardless of its actual location, all of the data is always available to users via normal access methods.

Figure 1-1 describes the concept of the DMF migration cycle between the DMF-managed filesystem on MIS disk and the secondary storage.

In a full system, there can be up to eight user filesystems that will be monitored by DMF, one per MIS LUN. (A given filesystem can also consist of multiple entire LUNs.) You will determine this configuration when you create the filesystem; see “Create and Mount the User Filesystems” in Chapter 2.

All of the filesystems beneath the /dmf directory are required for administration of DMF and (if installed) LiveArc AE. You should not manually modify any files in this directory and you cannot unmount these filesystems nor export them via NFS or CIFS. These filesystems are allotted a total of 8 TB on the MIS disk.

In general, only the most timely data resides on the higher-performance MIS disk; DMF automatically migrates less timely data to secondary storage. However, all of the data is always available to users and applications using normal access methods, regardless of the data's actual location.

Although DMF moves file data, it leaves file metadata in place on the MIS disk so that users can access files without knowing the actual location of the data. Metadata consists of items such as index nodes (inodes) and directory structure. Migrated files appear as normal files to users and are always easily accessible via high-performance network connections.

Because migrated files remain cataloged in their original directories, users and applications never need to know where the data actually resides; they can access any migrated file using normal processes. In fact, when drilling into directories or listing their contents using standard POSIX-compliant commands, a user cannot determine the location of file data within the storage tier; determining the data's actual residence requires special commands or command options.

A file whose data blocks have been freed is considered from the DMF perspective to be offline and its data blocks are therefore available for new active data, either new files or recalled files. However, from the user perspective , the file always appears to be online because the inodes and directories remain in the DMF-managed filesystem, allowing users to access the file by normal means.

The only difference users might notice when accessing a file whose data blocks have been freed is a delay in response time, because the data must be retrieved from secondary storage. From the user's perspective, all data always appears to be available online, regardless of its actual location.

DMF continuously monitors the DMF-managed filesystems on MIS disk so that it can maintain a certain amount of free space in those filesystems. This free space permits the creation of new files and the recall of previously migrated files. DMF maintains free space by freeing data blocks on the MIS disk for files that have already been migrated. DMF chooses to release the data blocks of the least-recently accessed files until the filesystem is well below the free-space minimum threshold. From a user's perspective, all content is accessible all of the time.

Figure 1-2 shows the concept of the free-space minimum threshold, where DMF will free the data blocks of less-recently accessed files (such as represented by the letter A) to empty the MIS disk well below the threshold as new files are added or as previously migrated files (such as represented by the letters B and E) are recalled.

The SGI InfiniteStorage Gateway migration policy does the following:

The following figures show concepts of DMF data migration:

Figure 1-5 shows an example of the path that file data might take from an NFS client to a mount point on the MIS disk (for example, the /mnt0 mount point). The data is stored in the DMF-managed filesystem on the MIS disk and then migrated to the secondary storage (in this case, COPAN MAID shelves 0 and 1).

A user retrieves a file simply by accessing it normally through NFS or CIFS; DMF automatically recalls the file's data from the secondary storage, caching it on the MIS disk as shown in Figure 1-4. After the data is restored to the MIS disk, the file becomes dual-state; if the user changes it, it once again becomes a regular file.

If you recall more files than the DMF-managed filesystem can currently contain, DMF migrates files and will free the data blocks of the least-recently accessed files until the filesystem is once again well below the free-space minimum threshold.

Each day, the SGI InfiniteStorage Gateway will migrate all data to the secondary storage and back up the inodes and directories in the DMF-managed filesystems. For more details about backups, see the SGI InfiniteStorage Gateway release notes; to access the release notes, see “Getting Help” in Chapter 2.