Follow this procedure to start the IRIS FailSafe system:

Under unusual circumstances you may find that you need to bring up just one node in an IRIS FailSafe cluster and have it run in degraded state (providing all high-availability services). The procedure below can be used to bring up one node in degraded state, even if the other node in the cluster is physically disconnected from the node you are bringing up. However, if the other node is connected, it will be power cycled during the procedure.

To stop IRIS FailSafe software on one node or both nodes in the cluster, follow these steps:

The subsections below provide information that system administrators need to know to successfully administer IRIS FailSafe nodes.

local7.info;local7.debug        |/var/ha/ha.filt        /var/ha/ha.log

#!/sbin/sh
# This filter only accepts IRIS FailSafe messages
read line
echo $line | grep "FS -"
if [ $? -eq ]; then
    echo "$line\c"
    exit 0;
fi
exit 0;

# /etc/init.d/raid5 stop 

# /etc/init.d/raid5 start 

When IRIS FailSafe is running on the nodes in a cluster, users need to know these things:

Users should use a fixed IP address, such as the hostname, in these situations:

Client users should use a high-availability IP address for a node instead of its hostname in these situations that require hostnames:

The netstat -i command tells you which interfaces are ifconfig'd up and the IP addresses they own. For each interface, the first line of output from netstat -i for that interface lists the fixed IP address for the interface. The IP addresses on the second and later lines of output for an interface are high-availability IP addresses.

Because IP aliases are ifconfig'd up only when a node is in normal or degraded state, the output of netstat -i varies. The examples below from the configuration shown in Figure 2-4 show netstat -i output for both nodes for the three possible situations:

To display the current state of a node (referred to as the node controller state in the output), enter this command on that node:

# /usr/etc/ha_admin -i 

A possible return might be

ha_admin: Node controller state normal 

Table 1-1 explains the possible states returned from this command.

To get the state of the other node in the cluster, enter this command:

# /usr/etc/ha_admin -i hostname 

hostname is the hostname of the other node.

The ha_admin -a command provides information about the cluster including each node's state and IP aliases. Each volume, filesystem, NFS filesystem, and Web server is listed along with its primary node. For example:

# /usr/etc/ha_admin -a 
Node controller states
          Node:            xfs-ha1      State: normal
          Node:            xfs-ha2      State: normal

Interface-pairs
Interface-pair:                  one    Owner: xfs-ha1
IP aliases in interface-pair one: stocks
Interface-pair:                  two    Owner: xfs-ha2
IP aliases in interface-pair two: bonds

XLV Volumes
    XLV volume:          shared1_vol    Owner: xfs-ha1
    XLV volume:          shared2_vol    Owner: xfs-ha2

Filesystems
    Filesystem:           shared1_fs    Owner: xfs-ha1
    Filesystem:           shared2_fs    Owner: xfs-ha2

NFS
           NFS:           export1_fs    Owner: xfs-ha1
           NFS:           export2_fs    Owner: xfs-ha2

Webservers
     Webserver:               webha1    Owner: xfs-ha1

Failover status: on
Heartbeat network: private network

This output shows an NFS filesystem being exported from each node (the NFS portion shows two NFS filesystems with different owners) and one Netscape server (Webserver). The names in the middle column of the output (xfs-ha1, one, shared1_vol, and so on) are all labels for blocks (node, interface-pair, volume, and so on) in the configuration file.

The failover status line near the bottom shows whether or not failovers occur if local monitoring or heartbeat monitoring detects failures. If the failover status is on, failovers occur; if it is off, they do not occur. Failover status can be changed by using the -F option of the ha_admin command (see the section “Choosing Monitoring Frequency Values” in Chapter 4).

The heartbeat network line at the end of the ha_admin -a output shows which network, the private network or a public network, is being used for heartbeat monitoring and IRIS FailSafe control messages.

When a node is in standby state, it is not providing any high-availability services. The -rf option and the -G option of ha_admin are used to tell the node to begin providing high-availability services.

If both nodes are running, the ha_admin -rf command is used to move a node in standby state to a state that enables it to provide high-availability services. The state that it moves to depends upon the state of the other node:

On the node in standby state that you want to move to normal or degraded state, enter this command:

# /usr/etc/ha_admin -rf 

If the other node is physically disconnected (no Ethernet connection, no FDDI connection, no serial line connection, and no connection to shared disk storage), you must use the ha_admin -G command to move the node in standby state to degraded state. In degraded state it provides all high-availability services for which it is the primary or backup node. Follow this procedure:

The -s option of ha_admin is used to move a node from normal state to standby state (a state where it provides no high-availability services). When a node is in standby state, it is said to be removed from the cluster. All high-availability services provided by the node are failed over to the other node.

You can a node that is in normal state to standby state by entering a command from either node:

The -sf option of ha_admin is used to move a node from degraded state (providing all high-availability services for which it is the primary or backup node) to standby state (not providing any high-availability services). Moving the node to standby state removes it from the cluster. The other node remains in standby state.

You can remove a node that is in degraded state by entering a command from either node:

The -r option of the ha_admin command is used to move a node from controlled failback state (the node does not provide high-availability services and is actively monitoring the other node) to normal state (providing the high-availability services for which it is the primary node). The other node (in degraded state) also moves to normal state.

You can move a node in controlled failback state to normal state by entering a command from either node:

The -s option of the ha_admin command is used to move a node from controlled failback state to standby state (the node, which has not been providing high-availability services, doesn't begin to provide them, but it does stop monitoring the other node). All the high-availability services are provided by the other node (which remains in degraded state).

You can move a node from controlled failback state to standby state by entering a command from either node:

If there is a failure of the private network between two nodes in a cluster, the heartbeat messages that are normally transmitted on the private network are switched automatically to the public network if the parameter hb-public-ipname is set to a fixed IP address. When the problem with the private network has been resolved, you must manually switch the heartbeat messages from the public network back to the private network.

To switch the heartbeat messages from the public network back to the private network, follow this procedure on either node: