Chapter 9. Data Backup: VOBs and Views

The most important maintenance task for a system administrator is ensuring frequent, reliable backups of essential disk storage. Because of this task's preeminent importance, we devote this chapter to it. The next chapter discusses other regular maintenance tasks.

Backup Tools

ClearCase does not include any data backup tools. All ClearCase data is stored in standard files, within standard directory trees; thus, you can use any backup tools at your disposal. The standard UNIX utilities tar(1) and cpio(1) are well-suited to backing up ClearCase data structures.

Backing Up a VOB

By default, a VOB storage directory is wholly contained in a single directory tree, which resides in a single disk partition. If you use a file-oriented backup tool, you need only specify the VOB storage directory to ensure a complete backup. If you use a disk-partition-oriented backup tool, you need only specify the partition name.


Note: The commands listed in the following sections don't require a ClearCase view context. The commands “don't care” whether or not a VOB is activated (mounted) on client hosts.


Determining a VOB's Location

To determine the location of a VOB's storage directory, use lsvob:

% cleartool lsvob -long /vobs/flex
VOB on host: ccsvr01
VOB server access path: /vobstore/flex.vbs
    (use this information if your backup program works locally)
 .
 .
  Tag: /vobs/flex
    Global path: /net/ccsvr01/vobstore/flex.vbs
     .
     .
    Region: uno
        (use this information if your backup program runs over the network)

Specify the appropriate pathname to your backup program. With a program that backs up entire disk partitions, you'll need to determine what partition the VOB storage directory resides in.

Ensuring a Consistent Backup

A backup will represent a self-consistent snapshot of a VOB storage directory's contents only if the VOB remains unmodified while the backup program is working. You need not unmount a VOB to ensure this—just lock it before backing it up, and unlock it afterward.

% cleartool lock -vob /net/ccsvr01/vobstore/flex.vbs
Locked versioned object base "/net/ccsvr01/vobstore/flex.vbs".

  <perform backup>

% cleartool unlock -vob /net/ccsvr01/vobstore/flex.vbs
Unlocked versioned object base "/net/ccsvr01/vobstore/flex.vbs".

Partial Backups

If you use a file-oriented backup program, you may wish to back up only some subdirectories within the VOB storage directory, in order to save time. Use the guidelines in Table 9-1 (carefully!) to determine the relative importance of the various components of a VOB:

Table 9-1. VOB Components for Partial Backups

VOB Component

Importance for Backup

top-level VOB storage directory

absolutely essential

VOB database subdirectory

absolutely essential

source storage pools

absolutely essential

derived object storage pools

important, but not essential

cleartext storage pools

optional

Backing up derived object storage pools is not crucial because, by definition, DOs can be rebuilt from sources. The importance of backing up these pools may change over time:

  • In the early stages of a project, when the source base in changing rapidly, the useful life of most derived objects is very short. Omitting DO storage pools from a backup regimen probably won't be much noticed by developers.

  • When a project is relatively stable, a VOB's DO storage pools contain many often-reused objects. At this stage, a complete build of a software system might wink-in virtually all DOs, rather than building them. Loss of a DO storage pool might increase the time requirement for such a complete system build by an order of magnitude.


Caution: A shared derived object has two parts: an object in the VOB database and a data container in a DO storage pool. Loss of DO data containers (for example, through failure to back them up) throws the VOB's database “out of sync” with its DO storage pools. To resynchronize, you must delete all the “dataless” DOs from the VOB database, using rmdo.

Backing up cleartext storage pools is not important at all, because they are merely caches that enhance performance. ClearCase type managers recreate cleartext data containers automatically, as necessary.

Example of Partial VOB Backup

Suppose you decide to back up all but the cleartext storage pools of the VOB located at /vobstore/flex.vbs. This would entail:

  • Backing up the files, but not the subdirectories of /vobstore/flex.vbs.

  • Backing up the entire contents of directory /vobstore/flex.vbs/.identity, which contains the VOB's ownership and group membership information.

  • Backing up the entire directory tree at /vobstore/flex.vbs/db, which contains the VOB database.

  • Backing up the entire directory tree at /vobstore/flex.vbs/s, which contains the VOB's source storage pools.

  • Backing up the entire directory tree at /vobstore/flex.vbs/d, which contains the VOB's derived object storage pools.

Backing Up a VOB with Remote Storage Pools

If a VOB has remote storage pools, then its on-disk storage is probably not wholly contained within a single disk partition. It is quite likely that the storage is distributed among two or more hosts. In this situation, the overall procedure is:

  1. Lock the VOB.

  2. Back up the VOB storage directory.

  3. Back up some or all of the VOB's storage pools.

  4. Unlock the VOB.

For Step #3, use the lspool command in any view to determine which storage pools are remote, and their actual locations:

% cleartool lspool -vob /vobs/flex
13-Jan.16:58   vobadm     pool "cdft"
  "Predefined pool used to store cleartext versions."
26-Jan.22:02   vobadm     pool "cltxt01"
  "remote cleartext storage pool for 'flex' VOB"
        (only remote storage pool)

13-Jan.16:58   vobadm     pool "ddft"
  "Predefined pool used to store derived objects."
13-Jan.16:58   vobadm     pool "sdft"
  "Predefined pool used to store versions."
% c lspool -long -vob /vobs/flex cltxt01 
pool "cltxt01"
     .
     .
  pool storage global pathname
   "/vobstore/flex.vbs/c/cltxt01"
     .
     .

If you are not sure which storage pools are remote, enter a lspool –long command to list the “pool storage global pathname” of every pool, and examine these pathnames to determine which ones specify remote locations.

Restoring a VOB from Backup

The following procedure restores a VOB backup without disrupting ongoing work. We assume that the VOB is the same one discussed in “Backing Up a VOB”.


Note: You can restore a VOB to a new location, on the same host or on another host. In this case, you must reregister the VOB at its new location (Step #8).


  1. Unmount the VOB—The VOB must be unmounted on each client host. For example:

    % cleartool umount /vobs/flex
    

  2. Go to the VOB host—Log in, as the root user, to the host where the VOB storage directory resides:

    % rlogin ccsvr01 -l root
    Password: <enter root password>
    #
    

  3. Check disk space availability—Make sure there is enough free space in the VOB's disk partition to load the backup copy:

    # cleartool space /vobstore/flex.vbs
      Use(Mb) %Use  Directory
      27.0     2%   VOB database /net/ccsvr01/vobstore/flex.vbs
      33.0     3%   cleartext pool /net/ccsvr01/vobstore/ 
                    flex.vbs/c/cdft
       .
       .
    --------  ----  ------------------------------------------
      312.9    28%  Subtotal 
      828.4    74%  Filesystem /net/ccsvr01/vobstore
                    (capacity 1115.1 Mb)
    

    If the available space is insufficient, delete the VOB storage directory, or use other means to make enough space available.

  4. Shut down ClearCase on this host—This ensures that ClearCase processes associated with the VOB are terminated. For example:

    # /etc/rc.atria stop
    


    Note: The name of the ClearCase shutdown script varies from system to system. See the init_ccase manual page.


  5. Move the original VOB aside—If it still exists, rename the VOB storage directory:

    # mv /vobstore/flex.vbs /vobstore/flex.OLD
    

  6. Restart ClearCase on this host—This reenables use of any other VOBs located there. For example:

    # /etc/rc.atria start
    

  7. Load the backup—Restore the VOB storage directory from the backup medium. For example:

    # mkdir /vobstore/flex.vbs
    # cd /vobstore/flex.vbs
    # tar -xvp
    


    Note: Each VOB storage area includes a directory named .identity, which stores files with special permissions: the setUID bit is set on file uid; the setGID bit is set on file gid. You must preserve these special permissions when you restore a VOB backup:


    • If you used tar(1) to back up the VOB, use tar's -p option when restoring the VOB. In addition, make sure to enter the tar command as the VOB owner or as the root user.

    • If you used cpio(1) to back up the VOB, no special options are required in the cpio command that restores the backup data.

    If the VOB has remote storage pools, restore the backups of these pools at this point. 

  8. Reregister the VOB—This is necessary only if you restored the VOB to a new location. For example, if you restored the VOB to new location /vobst_aux/flex.vbs:

    # cleartool unregister -vob /vobst_aux/flex.vbs
    # cleartool register -vob /vobst_aux/flex.vbs
        (Yes, unregister followed by register)
    # cleartool mktag -vob -replace -tag/vobs/flex/ \
      vobst_aux /flex.vbs
    

  9. Mount the restored VOB on one host—If the VOB host is also a client host, activate it there. Otherwise, use some other ClearCase host:

    # cleartool mount /vobs/flex
    

  10. Unlock the restored VOB—The VOB should have been locked before it was backed up. (See “Ensuring a Consistent Backup”.)

    # cleartool unlock -vob /vobstore/flex.vbs
    Unlocked versioned object base "/vobstore/flex.vbs".
    

  11. Mount the restored VOB on all hosts—The restored copy of the VOB is now ready to use. Have users remount the VOB on their workstations, using cleartool mount.

Reestablishing Consistency of a View's “Derived Object State”


Note: This section applies in any situation where a VOB's database has been “rolled back” to a previous state.

After you restore a VOB from backup, its VOB database may be out-of-date with respect to certain derived objects. The “old” database won't know about any DOs that were created in subsequent ClearCase builds. This will cause errors during hierarchical builds in which those “late-arriving” DOs are reused to construct higher-level targets:

======== Rebuilding "libbld.a"========
building libbld.a
        rm -f libbld.a
        rm -f /vobs/atria/sun5/pvtlib/libbld.a
        /opt/SUNWspro/bin/cc -c -o ...
        ar cq libbld.a  bld.o bld_pp.o ...

Will store derived object "/vobs/proj/sun5/libbld_V.o"
Will store derived object "/vobs/proj/sun5/libbld.a"
clearmake: Error: INTERNAL ERROR detected and logged in
"/var/adm/atria/error_log".

The error_log file shows:

Sunday 12/19/93 15:55:02. host "scandium", pid 440, user "chase"
Internal Error detected in "../bldr_vob.c"line 114
clearmake/cm/bldr_vob:
Error: VOB "scandium:/vobs/proj"
missing config record for derived object (OID) "0b5759d0.fb1811cc.a0af.08:00:69:02:2e:aa"

To reestablish the view's consistency with the VOB:

  1. Determine which DOs are causing the inconsistency—The cleartool ls command annotates them with [no config record]:

    % cleartool ls
    bldr_comm.ugh@@09-Dec.18:26.287028
    bldr_cr.msg.o [no config record]
    bldr_cr.o [no config record]
    bldr_cr.ugh [no config record]
    bldr_cr_cache.msg.c@@24-May.20:51.42929
      .
      .
    

  2. Remove the offending DOs—Use the standard rm(1) command:

    % rm bldr_cr.msg.o bldr_cr.o bldr_cr.ugh
    

Backing Up a View

Backing up views is similar to backing up VOBs, but is often simpler:

  • Users may create views under their home directories. In this case, whatever backup regimen you have in place to back up users' home directories will automatically pick up their view storage directories, as well.

  • Views do not have multiple storage pools, some of which may be local and others remote. A view has a single private storage area, its .s subdirectory, which can be either local or remote.

  • It never makes sense to attempt a partial backup of a view storage directory—all the data is important, because it's the only copy of one or more users' current work.

Use the following procedure to back up a view.

  1. Determine the location of the view storage directory—Use the lsview command:

    % cleartool lsview akp_vu
    View on host: neptune
    View server access path: /home/akp/views/akp.vws
            (use this information if your backup program works locally)
     .
     .
      Tag: akp_vu
        Global path: /net/neptune/home/akp/views/akp.vws
     .
     .
        Region: dvt
            (use this information if your backup program runs over the network)
    

  2. Ensure self-consistency of the backup—A view cannot be locked with the lock command. To guarantee that a view is inactive when it is backed up:

    • Warn ClearCase users that the view is about to become inactive.

    • Use kill(1) to terminate the associated view_server process on the host where the view storage directory resides.

    • A user's subsequent setview or startview command will invoke a new view_server process. If you are not confident that user's will refrain from such activity, rename the view storage directory. (In the following steps, we assume that you have not renamed it.)

  3. Determine whether the view has remote storage—You can use a standard ls(1) command:

    % cd /home/akp/views/akp.vws
    % ls -ld .s
       ...  .s -> /net/ccsvr04/viewstore/akp.stg
        (symbolic link indicates remote private storage area)
    

  4. Enter the backup command(s)—If the view's private storage area is local, then a single command can back up the entire view storage directory tree. Use the local address or the network-wide address listed by lsview in Step #1:

    % cd /home/akp/views
    % tar -cv akp.vws
    

    If the view's private storage area is remote, you need to perform a second backup (typically, using a different backup tape):

    % tar -cv /net/ccsvr04/viewstore/akp.stg
    

  5. Reactivate the view—If you renamed the view storage directory in Step #2, restore it to its original name. Inform users that they may resume using the view, with setview and startview commands.

Restoring a View from Backup

Use the following procedure to restore a backup view storage directory. We assume that the view is the same one discussed in “Backing Up a View”.


Note: You can restore a view to a new location, on the same host or on another host. In this case, you must reregister the view at its new location (Step #5).


  1. Go to the view host—Log in to the host where the view storage directory resides:

    % rlogin neptune
    

  2. Check disk space availability—Make sure there is enough free space in the view's disk partition to load the backup copy. If necessary, delete the view storage directory, or use other means to make enough space available.

  3. Move the original view aside—If it still exists, rename or delete the view storage directory:

    # mv /home/akp/views/akp.vws /home/akp/views/akp.vws.OLD
    

  4. Load the backup—Restore the view storage directory from the backup medium. For example:

    % cd /home/akp/views
    % tar -xv
    

    If the view's private storage area is remote, restore its backup, too.

  5. Reregister the view—This is necessary only if you restored the view to a new location. For example, if you restored the view to new location /usr2/akp.vws:

    # cleartool unregister -view /usr2/akp.vws
    # cleartool register -view /usr2/akp.vws
        (Yes, unregister followed by register)
    # cleartool mktag -view -replace -tag akp_vu /usr2/akp.vws
    

  6. Reactivate the view—The restored copy of the view is now ready to use. Use a setview or startview command to start a view_server process. Inform the user(s) of the view that it is available again.