Appendix A. Troubleshooting

This appendix contains troubleshooting information that answers common questions about operating and configuring NetWorker:

Maximum Filename Length

NetWorker supports a maximum filename size of 1024 characters. This limitation is the same as the IRIX svid limitation.

Checking the NetWorker Daemons

If you have trouble starting NetWorker or see error messages in /var/adm/SYSLOG from NetWorker daemons, the daemons may not be running properly. To check the daemons, enter this command:

# ps -ef | grep nsrd

If you discover that you need to start the NetWorker daemons, enter the commands below:

# cd /
# /etc/init.d/networker start

/nsr Not Configured

If you have not created the directory /nsr during installation and enabling of the NetWorker server software (see the section “Enabling Server Software” in Chapter 2), this message appears on the system console and in /var/adm/SYSLOG when the NetWorker daemons are started:

nsrd: error: /nsr must be a directory or a symlink to a directory
nsrd: info: Please create a symbolic link called /nsr that points
nsrd: info: to a directory in a filesystem with sufficient space
nsrd: info: to store NetWorker's index files

Follow step 6 or 7 in the section “Installing Server Software” in Chapter 2 to solve this problem.

Solving Licensing Problems

The following messages, and others, indicate that NetLS licenses for NetWorker products have not been properly enabled:

NetWorker Registration: (notice) server disabled; No valid Base or Advanced licenses found,
nsrd: either install new licenses or recover a valid resource file. (15017)
NetWorker registration: (info) Server is disabled (Install base enabler).
nsrcap: RPC error, Program not registered (24015)
nsrcap: nsrd did not respond. nsrd must be started before running nsrcap. If nsrd is running, you may have exceeded a resource limit.

To correct licensing problems, follow the procedures in the section “Enabling Server Software” in Chapter 2 to enable your NetLS licenses. To verify that licenses have been enabled, display them in the Registration window, as described in the section “Displaying NetWorker Licenses” in Chapter 4. Licenses that don't appear in this window aren't enabled.

Renaming a Client

NetWorker maintains an index for every client it backs up. If you change the name of the client, the index for that client is no longer associated with the client, and NetWorker cannot recover any files it backed up under the client's old name.

To change the name of a NetWorker client, you must first delete the old client name, then add the new client name, and rename the directory which contains the corresponding index.

Follow these steps:

  1. Create and configure the new client.

  2. Open the Clients window, and delete the old client name.

  3. In an IRIX shell, become root on the NetWorker server.

  4. Change directories to /nsr/index.

  5. Use the mv command to rename the client's index. For example:

    # mv old_client_name new_client_name

  6. Restart the NetWorker daemons, using the nsrd command. The daemon nsrmmdbd renames all the instances of the save set under the old client name to the new client name.

These steps cause the database daemon nsrmmdbd(1M) to rename the client inside its database:

  1. Make the old client name an alias of the new name.

  2. Kill the NetWorker daemons. To kill the daemons, use this command:

    # /etc/init.d/networker stop

  3. Restart the NetWorker daemons. To restart the daemons, use this command:

    # /etc/init.d/networker start 

As soon as possible, complete a full backup of the renamed client's files.

Using Recover Access

System administrators control client recover access by configuring the client. The Recover access list in the Clients window lists the systems that can recover the client's files.

These users can recover any files on any client:

  • root

  • operator

  • member of the operator group

Other users can only recover files for which they have read permission, relative to the file mode and ownership at the time the file was backed up. Files recovered by a user other than root, operator, or the operator group, are owned by that user.

Using Savegroup Completion Messages

In the Notifications window, you configured NetWorker to mail the event notification about your savegroups. The Notifications window is preconfigured to mail the savegroup completion messages to [email protected]_name. This section contains descriptions of error messages that can appear in the savegroup completion mail. Possible solutions are suggested.

Binding to Server Errors

NetWorker is designed to follow the client/server model. In a client/server model, servers provide services to the client through the Remote Procedure Call (RPC). These services live inside of long-lived IRIX processes, known as daemons.

For clients to find these services, the services must be registered with a registration service. When daemons start up, they register themselves with the registration service. In IRIX, the portmapper program provides the registration service.

NetWorker servers provide a backup and recover service: they receive data from clients, store the data on backup media, and retrieve it on demand. If the NetWorker daemons are not running and a service is requested (for example, save, recover, mminfo, and so on), these messages may appear in your savegroup completion mail:

Server not available
RPC error, remote program is not registered (24015)

These messages indicate that the NetWorker daemons nsrd, nsrindexd, nsrmmd, and nsrmmdbd are not running.

To restart the nsr daemons, enter nsrd at the system prompt:

mars# /usr/etc/nsrd

Saving Remote Filesystems

The following error message might appear in your savegroup completion mail when you are backing up a remote filesystem:

All: host hostname cannot request command execution

You may also see this message:

All: sh: permission denied

This means the nsrexecd on the client was not configured to allow the server hostname to back up its files. See the nsrexecd(1M) reference page for information on how to reconfigure nsrexecd.

When saving remote filesystems, files, or remote clients, NetWorker first tries to use nsrexecd, and if that fails, invokes an rsh(1M) to that client as the rcmd(3N) user (the default rcmd(3N) user is root). If the client is supposed to run nsrexecd, restart it. If not, make sure the NetWorker server can rsh as the rcmd user to the remote clients by adding this entry in the clients' /.rhosts file(s):

NetWorker_server rcmd_user

Two areas where you may encounter problems with NetWorker and rsh are as follows:

  • remote shell

    The remote shell is determined by the rcmd user's login shell (the login shell is determined by /etc/passwd file). One of the actions of rshd(1M) is to read commands from a file. This file is usually named .<type_of_shell>rc. For example, in the C shell this file is called .cshrc.

    One of the commands rshd may find is the PATH variable. The PATH has to have the directory in which the NetWorker executables are located. If the PATH command is not in the remote shell's rc file or does not have the correct directory where the NetWorker executables are located, then NetWorker is unable to find client executables.

    The following error messages may be sent if the PATH is not correct:

    clientname probe unexpectedly exited
    ...: savefs not found

    If the remote shell's rc file does not have the PATH variable set, then the hidden attribute executable path in the NSR client resource has to have the full pathname of the NetWorker executables as its value. If the PATH does not have the directory of the executables, then add the directory and give a command to read the .<remote_shell>rc file.

    Also, the remote shell has to allow an interactive session.

  • rcmd user and .rhosts

    The rcmd user is the user login name the NetWorker server uses to remotely log in and run commands on the client. The default rcmd user is root. If you are using root as the rcmd user then [email protected]_server_name should be in the /.rhosts file. Otherwise, NetWorker_server rcmd_user has to be in /etc/hosts.equiv or .rhosts file in the home directory of the local user on whom access is being attempted.

    The hidden attribute rcmd user is displayed in the Clients window when NetWorker is invoked with the –x option. If you are using an rcmd user other than root, enter a valid user_name in the rcmd user hidden attribute field.

    If the above conditions are not met, then these error messages may appear:

    .... : client rcmd(3) problem for command....
    .... : Permission denied

    Make sure that rcmd user is either operator or in operator's group.

File Changed During Backup

NetWorker backs up the image that is in the filesystem at the time it comes across the file. NetWorker notifies you that the file was changed during the backup in the Backup Status window and the savegroup completion mail. You can back up the file manually after it is through being used, or wait until the next incremental backup.

Cannot Print Bootstrap Information

If your bootstraps are not being printed, you may need to enter the printer name as a hidden attribute using these steps:

  1. Open the Groups window and select “Details” from the View menu.

  2. In the Printer field, enter the name of the printer you are using to print the bootstrap.

  3. Click Apply to save your changes.

Using Savegroup Completion Warning Messages

Occasionally the savegroup completion message includes one or more additional messages to help the administrator understand why NetWorker performs certain tasks. For example:

quattro:/usr no cycles found in media db; doing full save 

In this example, the filesystem /usr on the client quattro has no full saves listed in the media database. Therefore, despite the backup level preselected for that client's schedule, NetWorker performs a full backup. This feature is important because it allows you to perform disaster recoveries for that client.

This message can also appear if the server and client's clocks are not synchronized. To avoid this situation, make sure the NetWorker server and client:

  • are in the same time zone

  • have their clocks synchronized

Also make sure that a non-full save both begins and ends after midnight.

This savegroup message can also appear:

NetWorker_server:index Saving server index because server is not in an active group

If your server belongs to a group that is not enabled, NetWorker saves the server's bootstrap information along with this group to avoid a long recovery process. As soon as possible you should enable the group your NetWorker server belongs to, or add the server to another group that is enabled.

Errors in Executing External ASMs

This error message may appear during a backup:

client:/ save: external ASM `/xxxxxxasm' not found or not executable

All external ASMs (see uasm(1M) for a complete list of external ASMs) are separate commands, and invoked as needed. For security reasons, external ASM names must end in “asm” and be located in the same directory as the originally invoked command (typically save or recover). Rename or move the ASM following these conventions to correct this error.

Solving X Error Messages

The following error message can appear when the nwadmin & command is executed:

Xlib: connection to “client:0.0” refused by server
Xlib: Client is not authorized to connect to Server
X error: Cannot open display on window server: client:0.0 (Server package)

This message indicates that the client is not authorized to display NetWorker.

To correct this situation, follow these steps:

  1. Enter this command at the client system:

    client% /usr/bin/X11/xhost Networker_server

  2. Remotely log in to the NetWorker server and enter this command at the server prompt:

    Networker_server% setenv DISPLAY client:0.0

    Note: For shells other than csh use the following commands instead:

    # DISPLAY=client:0.0
    # export DISPLAY

Moving Indexes

Because the index databases are holey files, the cp command creates a file that consumes more disk space than the original file. To move indexes, enter the following command in the /nsr/index directory:

mars# uasm -s -i <client index directory name>|(cd target_dir; uasm -r)

Note: On an EFS filesystem, do not use this command; EFS filesystems have no holey files.

Recovering Files From an Interrupted Backup

You cannot recover files from a backup that was terminated by killing the NetWorker daemons, because the media index was not updated before the daemons died. Consequently, NetWorker does not know on which volume the requested file is located.

Determining the NetWorker Server

If you start NetWorker from a remotely mounted directory, the following message might appear:

Using server server_name as server for client_name.

NetWorker looks for the system that is the fileserver of a remotely mounted directory and uses the NetWorker server assigned to that system as the backup server. To bypass this message, start NetWorker from a local filesystem.

Using nsrexecd

The nsrexecd daemon runs on NetWorker client systems. This daemon provides a secure and restrictive way for NetWorker to start automatic backups on clients. The nsrexecd daemon allows you to restrict access to a select set of NetWorker servers. When you install an IRIX client, nsrexecd is started, and statements are added to /etc/rc2.d/S95networker to restart nsrexecd each time the client reboots.

Security is increased by a challenge and response scheme, which ensures that only the NetWorker server is initiating connections, and not another command. Commands are restricted to those needed for saving data (the save and savefs commands are described in Appendix C, “Theory of Operations”).

If you must reconfigure nsrexecd, for example, to allow a different NetWorker server to back up the client, follow these steps:

  1. In the file /etc/rc2.d/S95networker on the client, make the changes to the nsrexecd startup command.

    Note: See the nsrexecd(1M) reference page for a description of the command line configuration options.

  2. Restart nsrexecd by entering these commands:

    # /etc/init.d/networker stop
    # /etc/init.d/networker start