Chapter 8. NetWorker Operations

This chapter describes the everyday tasks for operating NetWorker. Generally, five daily tasks are required:

This chapter concludes with instructions for a management task that might need to be performed occasionally: recovering files that have been removed from the index.

Checking the NetWorker Server Status

The NetWorker Administrator window displays the status of the NetWorker server in five displays. If you do not already have the Administrator window on your screen, enter the nwadmin(1M) command at the prompt in a shell window to start the NetWorker Administrator program:

# /usr/etc/nwadmin&

Figure 8-1 shows the window. For more information on window features, read Chapter 3, “Using NetWorker Windows and Menus.”

Figure 8-1. NetWorker Administrator Window


To check the status of the NetWorker server, follow these steps:

  1. In the Messages display, scroll through the available messages. Error messages in this display indicate problems that should be investigated. Pull down the window to display more messages at a time, if necessary.

  2. Look at the Pending display. If there are messages in this display, there may or may not be a problem:

    • If there are active save sets running (denoted by changing messages in the Messages display), NetWorker is operating normally.

    • If there are no active save sets running, there may be a problem and it should be investigated.

  3. Open the Group Control window by selecting “Group Control” from the Server pulldown menu. For each group, select the group, then click the Details button. Messages in the Failed Save Sets display indicate problems. For more information, see “Viewing Details of a Scheduled Backup” later in this chapter.

  4. Check the contents of /var/adm/SYSLOG for the last 24 hours. Look for

    • messages output by NetWorker as it works: check for error messages

    • error messages output by the kernel that might affect the operation of NetWorker; for example, messages about SCSI hardware errors

Monitoring and Controlling Backups

NetWorker provides various windows for monitoring and controlling your backups: the NetWorker Administrator window, the Groups window, and the Group Control window. Use the Group Control window to

  • monitor a scheduled backup

  • view the details of a scheduled backup

  • preview a backup

  • immediately start a scheduled backup

  • stop a scheduled backup

  • restart a scheduled backup

Open the Group Control window by selecting “Group Control” from the Server pulldown menu or by clicking the Group Control speedbar button, as shown in Figure 8-2.

Figure 8-2. Opening the Group Control Window


Figure 8-3 shows the Group Control window.

Figure 8-3. Group Control Window


The Groups scrolling list displays information about all of the backup groups known to the NetWorker server:

Name  

Displays the name of the backup group.

Last Run  

Lists the date and time the group was last backed up.

Next Run 

Shows the date and time for the next enabled scheduled backup.

Status  

Displays the progress of the backup. NetWorker displays these four progress messages:

  • Never Run: the group has not been backed up since the last reboot or restart of NetWorker service

  • Finished: the backup has finished

  • Not Finished: the backup is still running or the backup has exited prematurely without finishing

  • Preview Run: the preview of a backup is running

Start button  

Starts a backup immediately.

Restart button  

Restarts a backup that had been stopped.

Stop button  

Stops a scheduled backup.

Details button 

Displays details of completed or currently running backups.

Preview button  

Previews an upcoming backup.


Note: Use the Groups window to create new groups and to enable and set the backup time for a group.


Viewing Details of a Scheduled Backup

To view more detailed information about a group's completed or currently running backup, click the Details button in the Group Control window. NetWorker displays the Group Control Details window, as shown in Figure 8-4.

Figure 8-4. Group Control Details Window


The Group Control Details window displays save sets in the process of being backed up and save sets that have completed their backups. Use the window to determine which client's save sets backed up successfully and which save sets may have failed.

The fields in the Group Controls Details window have the following functions:

Pending Save Sets 


Displays the filesystems that have not yet been backed up.

Complete Save Sets  


Displays the filesystems that NetWorker has successfully backed up.

Failed Save Sets  


Displays the filesystems that NetWorker was unable to back up and error messages concerning failures.

If NetWorker was unable to back up save sets for a client in the group, determine the cause for the failed backup. Typical reasons for failed save sets include the following:

  • the server crashed during the backup

  • the client crashed during the backup

  • the network connection failed during the backup


Tip: If you previously set the Client retries to a number greater than zero in the Groups window and NetWorker fails to back up a client, filesystems might be temporarily listed in the Failed Save Sets and Pending Save Sets scrolling lists while NetWorker tries again to back up the client.


Previewing a Group Backup

You can preview a group backup by clicking the Preview button in the Group Control window. When you click the Preview button, the Group Control Details window appears, but instead of displaying past information about completed group backups, you see future information about how a group will perform during its next scheduled backup.

NetWorker simulates a backup for the group you select in the Groups scrolling list in the Group Control window. By using the preview feature, you can determine if there will be any problems with an upcoming group backup, and take the necessary steps to correct them.

You can also preview a backup group by checking if NetWorker can access the client in order to back up its files every time you add a new client to NetWorker. Use the savegroup  –p command at the system prompt to see a preview of a group backup without actually backing up any files. For example:

# /usr/etc/savegroup -p group_name

This command shows you a backup preview of the clients assigned to the backup group group_name. If NetWorker cannot access a client in the backup group, an error message appears.

If you see an error message, make sure nsrexecd(1M) is running and that it lists the server`s hostname in the command line. To make sure that nsrexecd is running, you can use the command ps(1) on the client. See the section “Installing NetWorker Software on Clients” in Chapter 2 for more information on nsrexecd.

Starting a Scheduled Backup Immediately

To start a backup immediately from the Group Control window, select the group you wish to back up from the Groups scrolling list and click the Start button.

NetWorker immediately backs up the clients in the group, overriding the backup schedule, and displays “running” in the Status field of the Group Control window.


Tip: You can also immediately start a backup in the Groups window with the Start now button.


Stopping a Backup Group

If you need to stop a scheduled backup in progress, use the Stop button in the Group Control window. Figure 8-5 shows this window.

Figure 8-5. Group Control Window


To stop a backup, select the group you wish to stop from the Groups scrolling list and click the Stop button.

As soon as the current save set completes backing up, NetWorker halts the scheduled backup and displays “not finished” in the Status field in the Group Control window.

The next network-wide backup starts as scheduled in the Start time field of the Groups window, or you can restart the backup by clicking the Restart button in the Group Control window.

Restarting a Backup Group

If you wish to restart a scheduled backup that you have stopped, use the Restart button in the Group Control window. To restart a backup, select the name of the halted backup group from the Groups scrolling list in the Group Control window and click the Restart button.

NetWorker resumes the scheduled backup for the group, and displays “running” in the Status field.

Handling Open Files

If a client's open files change during a scheduled backup, NetWorker backs up the file and “notices” that it is changing. In the Group Control Details window, the name of the file that changed during the backup is displayed in a warning message like this:

warning: file name changed during save

You can restart the backup group, back up the client manually, or allow NetWorker to back up the client during the next scheduled backup.


Note: You cannot recover files from a backup that was terminated when the NetWorker daemons were killed, because the media index was not updated before the daemons died. Consequently, NetWorker does not know on which volume the requested file is located.


Monitoring Backup Devices

To monitor NetWorker backup devices, use the Media menu, as shown in Figure 8-6.

Figure 8-6. Media Menu


To find out how much space is available in the jukebox (or other autochanger), use either the Jukebox Mounting window or the nsrjb command. The Jukebox Mounting window displays all the media in the jukebox and the percentage used of each tape, as shown in Figure 8-7.

Figure 8-7. Jukebox Mounting Window


If you prefer, use the nsrjb command in an IRIX shell:

# nsrjb -jjukeboxname

NetWorker displays information about the backup volumes in the jukebox that looks similar to this:

Jukebox arc-db:
slot	volume	used	pool	mode
1:	moon.010			Default 
2:	moon.011		full	Default 
3:	moon.012			Default 
4:	moon.013	full		Default 

4 volumes, 2 less than 80% full.
2305 MB total capacity, 2200 MB remaining (5% full)

drive 1 (/dev/rmt/tps4d6nrnsv) slot 3: moon.012 

Notice the information about the registered volumes, total capacity, and remaining capacity. This information tells you how much space is still available in the jukebox.

Monitoring and Managing Index Disk Space Usage

Every time a backup completes, NetWorker creates entries for the files saved for each client in online indexes. These indexes require disk space and must be monitored to be sure they are not getting too large. Entries can be removed from the indexes when they become too large. You can also manage your indexes automatically by selecting different index policies in the Policies window. See “Determining Browse and Retention Policies (Index Policies)” in Chapter 5 for more information.

This section explains

  • viewing index information

  • viewing save sets to determine resource usage

  • removing the oldest cycle

  • reclaiming index space

Viewing Index Information

To view the disk space usage of the indexes, click the Indexes speedbar button or select “Indexes” from the Clients pulldown menu, as shown in Figure 8-8.

Figure 8-8. Opening the Indexes Window


The Indexes window appears, as shown in Figure 8-9.

Use features of this window as follows:

Clients scrolling list  


View index space allotted to various clients. The Used column indicates the percentage of the index file being used. If the percent listed is 100%, the index has completely filled the allocated disk space and there is very little, if any, disk space to reclaim. The smaller the percentage, the more disk space there is to reclaim, because the index is not using all of the currently allocated disk space. The allocated disk space grows automatically as the index size increases.

Reclaim Space button  


Click to decrease the size of the client file index. When you use the Reclaim Space button, NetWorker compresses the space in the index left by removed entries, thus freeing up disk space. A confirmation box appears once a reclaim space operation has begun.

Save Sets scrolling list  


View a list of save sets contained in the file index for the selected client. The save sets are grouped according to their unique save set names. For example, all backups of /usr in a client file index are grouped under the save set name /usr.

  • Name: shows client save set names

  • Size: displays an estimate of the amount of index space used by the save set group in the client file index

  • Cycles: displays the number of cycles for the save set group contained in the client file index. One cycle starts with a full backup and ends with the next full backup. A cycle includes the incremental and level 1-9 backups, if any exist.

Remove oldest cycle button  


Click to remove the save set's oldest cycle, if the index requires further reduction. Select the save set and then click the Remove oldest cycle button. After removing a save set, click Reclaim space to remove the holes left by removing the oldest cycle. A confirmation box appears once the removal of an oldest cycle has begun.

For instructions on reclaiming disk space in the indexes, see “Reclaiming Index Space” later in this chapter.

Viewing Save Sets to Determine Resource Usage

To plan the amount of disk space needed for the online indexes, you can determine the size of various save sets. Save sets are groups of files, usually contained in a single filesystem, that have been backed up by NetWorker. Save sets are created each time a backup is started. Generating a save set creates one or more entries in both the file and media indexes.

You can also generate a report detailing NetWorker client backups to fulfill administrative requirements.

To view the Instances window, follow these steps:

Use NetWorker's Instances window to view detailed information about save sets.

  1. Select a client from the Clients scrolling list in the Indexes window.

  2. Select a client save set from the Save Sets scrolling list.

  3. Click the Instances button.

    The Instances window appears, as shown in Figure 8-10.

    Figure 8-9. Instances Window


The features of this window have the following functions:

title bar 

Displays the name of the currently selected save set.

Instances scrolling list 


Contains the following information:

  • Id: internal NetWorker identification number for the save set

  • Files: number of files in the save set

  • Size: size of the save set

  • Date: date the save set was backed up

  • Level: level of backup: full, incr, or level

You can use the Print button to print the contents of the Instances window. To save the contents of the Instances window to a file, click the Save button. The information is saved to a file named networker.rpt, which NetWorker automatically creates, unless you choose a new filename in the dialog box that appears.

Removing the Oldest Cycle

To remove the oldest full-to-full cycle of a group of save sets from the file index for a client, follow these steps:

  1. Open the Indexes window by selecting “Indexes” from the Clients pulldown menu.

  2. Select the client from the Clients scrolling list.

  3. Select the save set group from the Save Sets scrolling list.

  4. Click the Remove oldest cycle button to remove the save set group's oldest cycle. A confirmation box appears, prompting you to confirm that you want to remove the oldest cycle of the selected save set.

  5. Click Ok in the confirmation box to proceed, or Cancel to discontinue the operation.


Note: If you do not highlight any save sets in the Save sets scrolling list, NetWorker removes all the oldest cycles of all the save sets displayed in the scrolling list for the client highlighted in the Clients scrolling list.

Figure 8-11 diagrams what happens when you remove a cycle from the index. Removing the oldest cycle opens up space so that other entries can fit into the empty space without increasing the size of the index.

Figure 8-10. Removing a Cycle From an Index


Note that removing index entries does not decrease the size of the index file, so it still takes up as much space as it did before. To decrease the amount of space the index uses, remove the unneeded index entries and use the Reclaim space button.

After the “Remove oldest cycle” operation has finished, the statistics in the Indexes window are updated to reflect the current state of the file index.


Note: The last full cycle cannot be removed using the Remove oldest cycle button because it might be needed for recovery.


Reclaiming Index Space

Use these instructions to reclaim empty space in the file index created when entries are automatically or manually removed:

  1. Open the Indexes window by selecting “Indexes” from the Clients pulldown menu.

  2. Click the Reclaim Space button. You do not need to select individual clients or save set groups from the scrolling lists.

    The Reclaim Space confirmation box appears, as shown in Figure 8-12.

    Figure 8-11. Reclaim Space Confirmation Box


  3. Click the Ok button to start the reclaim space operation. The Progress bar moves to show the progress of the index operation. When an index is busy, the Ok button is disabled.

    The Reclaim Space confirmation box changes, as shown in Figure 8-13.

    Figure 8-12. Reclaiming Space


  4. Click the Cancel button to dismiss the dialog box. This action closes the window; however, the reclaim space operation continues in the background. You can check the progress of the operation at any time by clicking the Reclaim space button to redisplay the confirmation box.

The index is rewritten without the deleted entries; consequently, it shrinks in size and opens up space for new entries. The space is returned to the filesystem.

Figure 8-14 diagrams what happens when you use the Reclaim Space button.

Figure 8-13. Reclaiming Index Space



Note: You cannot simultaneously reclaim space and remove an oldest cycle for one client. The index is busy during both operations.

After the reclaim space operation has finished, the statistics in the Indexes window are updated to reflect the current state of the file index.

Checking NetWorker Notices

Checking NetWorker notices is an important daily task for ensuring the smooth operation of NetWorker. NetWorker sends notices about significant NetWorker events. These notices include

  • messages requesting backup volumes to be mounted

  • nightly backup completion notices

  • messages concerning the size of the online indexes

  • jukebox request messages

  • registration status

NetWorker is shipped with preconfigured notification instructions to send electronic mail to root about most events. You can check NetWorker notices three ways:

  • Log in as root, and read your electronic mail.

  • Look at the system console messages.

  • Use the NetWorker Notifications window.

Using the Notifications Window

To open the Notifications window, select “Notifications” from the Customize pulldown menu, as shown in Figure 8-15.

Figure 8-14. Opening the Notifications Window


The Notifications window appears, as shown in Figure 8-16.

The Notifications scrolling list displays the names of all the notifications known to the NetWorker server, including NetWorker preconfigured notifications. Information about the currently selected notification is displayed in the lower half of the window.

Name field  

Displays the name of the currently selected event notification.

Action field  

Displays the type of action NetWorker takes with the notice. For example, NetWorker can use electronic mail to notify individuals that a backup volume needs to be mounted in the server. You can change the Action field to customize how the notifications are posted.

Using NetWorker Preconfigured Notifications

NetWorker includes seven preconfigured types of notification: Registration, Log default, Index size, Savegroup completion, Tape mount request 1, Tape mount request 2, and Tape mount request 3. If you are using a jukebox with NetWorker, you also receive a Jukebox request notification.

Most of these notices alert you about important NetWorker events. For example, if a group of clients did not complete a nightly backup, a savegroup completion notice appears in your electronic mail.

Registration 

The registration notification sends a message to root that your NetWorker products are not properly registered. The registration notification appears once a day or each time you start NetWorker. The notification message includes related information about each of the NetWorker products that are not licensed correctly.

Log Default 

The log default notification uses an IRIX facility called syslog to log and distribute notification about all NetWorker events. These events include requests for backup volume mounts, index size notices, and savegroup completion notices. How this information is distributed depends on how you have configured syslog. When NetWorker was installed, it created entries in the syslog configuration file for logging and contacting operators; you can customize these entries. See the syslogd(1M) reference page for information on configuring the distribution of log information.

Index Size 

NetWorker checks the size of its online indexes and sends a notification if it looks as if the indexes may run out of disk space. NetWorker automatically sends the electronic mail message to root. If you want the message to be mailed to someone other than root, edit Action in the Notification window, substituting a different user login name or a mailing list for root. To reduce the size of the indexes, see “Removing the Oldest Cycle” earlier in this chapter.

Savegroup Completion 


When NetWorker finishes backing up a group of clients, it sends a completion message via electronic mail to root.

Backup Media Request Notices 


When NetWorker needs backup media mounted for a backup, or a specific backup volume mounted to fill a recovery request, it displays a media request message in the NetWorker Administrator window. If no one fills the request, NetWorker sends another request after 15 minutes. NetWorker sends a third request after another 37 minutes, if no one fills the request.

The first mount request has a blank Action field, so the request appears only in the Pending display of the NetWorker Administrator window. The second mount request sends an alert to syslog, and the third request sends electronic mail messages to root.

Jukebox Request 


NetWorker sends a jukebox request when a backup volume is required.

Recovering Files Removed From the Index

You can recover files whose entries have been removed from the online index because they have passed the Browse policy period as long as the files are still stored on a backup volume. The recover process is not nearly as convenient as when the entries are still in the online index, however.

If you do not want to rebuild the index, and the save sets you need are still in the media index, and you know which save set contains the file you want, you can use the save set recover feature to recover the entire save set or selected directories and files. The save set recover feature is most useful for recovering from full backups, and is limited to root and users belonging to the group operator.

To rebuild the file index so that you can browse for the file you lost, follow this procedure:

  1. Use the Volume Management window to find out the name of the backup volume that contains the save set.

  2. Use the mminfo command to determine the save set ID. Use this syntax:

    mminfo -v -s server -c client -N saveset volume_name
    

  3. Rebuild the file index entries for the save set using the scanner -i -s save_set_id# command at the system prompt. Enter the save set ID number determined above for save set id#. Rebuilding the file index using the scanner command takes some time.

  4. Use the NetWorker Recover window to identify the needed file(s) and initiate the recovery.


Note: Recovery is considerably easier if the file information is still in NetWorker's online index. Set a browse policy that is long enough to cover almost all recovery requests.