Chapter 4. Testing the Resources

To ensure that the NFS and statd_unlimited resources have been correctly configured, you should test the set of action scripts located in the following directories:

/var/cluster/ha/resource_types/NFS
/var/cluster/ha/resource_types/statd_unlimited

After executing an action script, see the contents of the output file. The output file must contain the resource name and status of action for the resource. If resource status is zero, the action was successful for the resource; if the status is nonzero, there was an error that could mean the resource was not configured properly for FailSafe. For more information on error codes, see the IRIS FailSafe Version 2 Programmer's Guide.

For example, to test an NFS resource named /shared1, you would test each action script by starting with the following commands, using an input file named /tmp/ipfile and an outputfile named /tmp/opfile:

$ cd /var/cluster/ha/resource_types/NFS
$ echo /shared1 > /tmp/ipfile

You can then execute each action script with the following command:

$ ./actionscript /tmp/ipfile /tmp/opfile

where actionscript is one of the following action script names:

start
stop
monitor
exclusive
restart


Note: By default, the NFS and statd_unlimited resource types are not configured to be restarted. Although an action script file by the name of restart must exist, it is not necessary to test it unless you change the configuration to use the script.

For more information about action scripts, see the IRIS FailSafe Version 2 Administrator's Guide.

For example:

# cd /var/cluster/ha/resource_types/NFS
# echo /shared1 > /tmp/ipfile
# ./start /tmp/ipfile /tmp/opfile
# cat /tmp/opfile
/shared1 0
# rm /tmp/opfile
# ./stop /tmp/ipfile /tmp/opfile
# cat /tmp/opfile
/shared1 0
etc.

The output for each action script is logged to the following file:

/var/cluster/ha/log/script_hostname

You can increase the amount of data logged from scripts by changing the HA_CURRENT_LOGLEVEL variable in the following file on each node:

/var/cluster/ha/common_scripts/scriptlib

The default value for HA_CURRENT_LOGLEVEL is 2; to make logging more verbose, you can change it to 11. For more information about log levels, see the IRIS FailSafe Version 2 Administrator's Guide.

The rest of this chapter discusses the following:

Obtaining More Details

To view the individual action script executions, you must edit the action scripts and add set -x to the function.

In the following example, to see what the start_nfs() function does, edit the start script for the NFS resource and add set -x in the start function.

# Start the resource on the local machine.
# Return HA_SUCCESS if the resource has been successfully started on
# the local machine and HA_CMD_FAILED otherwise. The resource name is
# the nfs instance id.
#
start_nfs()
{
set -x
 # for all nfs resource configured
...

Following is an example showing output obtained when set -x is used:

# cd /var/cluster/ha/resource_types/NFS
# echo /shared1 > /tmp/ipfile
# ./start /tmp/ipfile /tmp/opfile
+ HA_SCRIPTNAME=/var/cluster/ha/resource_types/NFS/start
+ set_global_variables
+ set_local_variables
+ ha_check_args /tmp/ipfile /dev/null
+ [ 0 -ne 0 ]
+ ha_read_infile
+ start_nfs
+ NFSFILEDIR=/var/cluster/ha/tmp/shared1
+ HA_CMD=/sbin/mkdir -p /var/cluster/ha/tmp/shared1
+ ha_execute_cmd creating nfs status file directory
+ get_nfs_info /shared1
+ [ 0 -ne 0 ]
+ ha_get_field export-info rw,wsync,anon=root
filesystem /shared1 export-point
+ [ 0 -ne 0 ]
+ ha_get_field export-info rw,wsync,anon=root
filesystem /shared1 export-info
+ [ 0 -ne 0 ]
+ export_opts=rw,wsync,anon=root
+ retstat=0
+ /usr/etc/exportfs
+ grep /shared1$
+ 1> /dev/null 2>& 1
+ retstat=1
+ [ 1 -eq 1 ]
+ /usr/etc/exportfs
+ grep /shared1 
+ grep rw,wsync,anon=root$
+ 1> /dev/null 2>& 1
+ retstat=1
+ [ 1 -eq 1 ]
+ HA_CMD=/sbin/grep /shared1 /etc/mtab > /dev/null 2>&1
+ ha_execute_cmd check if the export-point exists
+ [ 0 -eq 0 ]
+ HA_CMD=/usr/etc/exportfs -i -o rw,wsync,anon=root /shared1
+ ha_execute_cmd export /shared1 directories to NFS clients
+ [ 0 -ne 0 ]
+ ha_write_status_for_resource /shared1 0
+ exit_script 0

Testing the start Script

Before testing the start script, you must ensure that the file system has not already been exported. You can use the exportfs(1M) command to check if the file system has been exported.

If it has already been exported, you must unexport the file system by using the exportfs -u command, as in the following example:

# exportfs
/dmf_home
/move_fs
/usr/cluster/bin
/shared2
shared1 -rw,wsync,anon=root
# exportfs -u /shared1

After running the test, you can use the exportfs(1M) command again to verify that the resource is exported.

Testing the stop Script

Before testing the stop script, you must use the exportfs(1M) command to ensure that the NFS file system is already exported. If it has not already been exported, you must export it by using the exportfs command before running the stop script.

After running the test, you can use the exportfs(1M) command again to verify that the resource has been stopped by the stop script and is therefore not exported.

Testing the monitor Script

Use the tail(1) command on the script log to verify that the monitor script works as expected. For example:

# tail /var/cluster/ha/log/script_node1
Mon Dec  9 17:34:12 <D0 /var/cluster/ha/resource_types/NFS/monitor script 182977:0>
/var/cluster/ha/resource_types/NFS/monitor called with  /var/cluster/ha/tmp/srmMCXa004xn,
/var/cluster/ha/tmp/srmNCXa004xn and /var/cluster/ha/tmp/srmOCXa004xn
Mon Dec  9 17:34:13 <D0 /var/cluster/ha/resource_types/NFS/monitor script 183688:0> Entry:
monitor_nfs()
Mon Dec  9 17:34:14 <D0 /var/cluster/ha/resource_types/NFS/monitor script 183688:0> check the
filesystem /shared1 is exported
Mon Dec  9 17:34:14 <D0 /var/cluster/ha/resource_types/NFS/monitor script 183688:0> awk '{print $1}'
/var/cluster/ha/tmp/exportfs.183688 | grep -x /shared1 exited with status 0
Mon Dec  9 17:34:14 <D0 /var/cluster/ha/resource_types/NFS/monitor script 183688:0> Entry:
exec_rpcinfo()
Mon Dec  9 17:34:14 <D0 /var/cluster/ha/resource_types/NFS/monitor script 183688:0> trying rpcinfo
on localhost for nfs ver 3
Mon Dec  9 17:34:15 <D0 /var/cluster/ha/resource_types/NFS/monitor script 183688:0>
/usr/cluster/bin/ha_exec2 5 2 /usr/etc/rpcinfo -n 2049 -u 127.0.0.1 nfs 3 exited with status 0
Mon Dec  9 17:34:15 <D0 /var/cluster/ha/resource_types/NFS/monitor script 183688:0> Return:
exec_rpcinfo() value 0
Mon Dec  9 17:34:15 <D0 /var/cluster/ha/resource_types/NFS/monitor script 183688:0> Exit:
exit_script()

Testing the exclusive Script

Use the tail(1) command on the script log to verify that the exclusive script works as expected. For example:

# tail /var/cluster/ha/log/script_node1
Mon Dec  9 17:37:19 <D0 /var/cluster/ha/resource_types/NFS/exclusive script 183918:0>
/var/cluster/ha/resource_types/NFS/exclusive called with /var/cluster/ha/tmp/srmYLXa004xn,
/var/cluster/ha/tmp/srmZLXa004xn and /var/cluster/ha/tmp/srmAMXa004xn
Mon Dec  9 17:37:19 <D0 /var/cluster/ha/resource_types/NFS/exclusive script 183918:0> Entry:
exclusive_nfs()
Mon Dec  9 17:37:20 <D0 /var/cluster/ha/resource_types/NFS/exclusive script 183918:0> checking for
/shared1 exported directory
Mon Dec  9 17:37:20 <N /var/cluster/ha/resource_types/NFS/exclusive script 183918:0> checking for
/shared1 exported directory failed
Mon Dec  9 17:37:20 <N /var/cluster/ha/resource_types/NFS/exclusive script 183918:0>
Mon Dec  9 17:37:20 <D0 /var/cluster/ha/resource_types/NFS/exclusive script 183918:0> /sbin/grep
/shared1 /var/cluster/ha/tmp/showmount.183918 >> /dev/null 2>&1 exited with status 1
Mon Dec  9 17:37:20 <N /var/cluster/ha/resource_types/NFS/exclusive script 183918:0> resource
/shared1 exclusive status: NOT RUNNING
Mon Dec  9 17:37:20 <D0 /var/cluster/ha/resource_types/NFS/exclusive script 183918:0> Exit:
exit_script()

Testing the restart Script

You should test the restart script if the resource is configured to restart; by default, the NFS and statd_unlimited resource types are not configured to restart.

After running the test, you can use the exportfs(1M) command to verify that the resource is exported.

Testing Resource Group Failovers

You can test the failover policy by using either the cmgr(1M) or the FailSafe GUI to move the resource group to another node in the cluster and then display the resource group states. The following example uses cmgr to test the failover policy:

cmgr> admin offline resource_group NFS in cluster nfscluster

cmgr> admin move resource_group NFS in cluster eagan to node node2

cmgr> admin online resource_group NFS in cluster nfscluster

For more information about states, see the IRIS FailSafe Version 2 Administrator's Guide.