This chapter explains how to plan the configuration of high-availability services on your IRIS FailSafe cluster. The major sections of this chapter are as follows:
Configuration planning involves making decisions about how you plan to use the IRIS FailSafe cluster, and based on that, how the disks and interfaces must be set up to meet the needs of the high-availability services you want the cluster to provide. Questions you must answer during the planning process are:
What do you plan to use the two nodes for?
Your answers might include uses such as offering home directories for users, running particular applications, providing Netscape World Wide Web service, and providing file service.
Which of these uses will be provided as a high-availability service?
The IRIS FailSafe NFS option enables you to provide exported NFS filesystems as high-availability services. Similarly, the IRIS FailSafe Web option is used for the Netscape FastTrack and Enterprise Servers, the IRIS FailSafe INFORMIX option is used for Informix databases, the IRIS FailSafe Oracle option is used for Oracle databases, and the IRIS FailSafe Sybase option is used for Sybase databases. To offer other applications as high-availability services, you must develop a set of shell scripts that provide switch over and switch back functionality. Developing these scripts is described in the IRIS FailSafe Programmer's Guide .
Which node will be the primary node for each high-availability service?
The primary node is the node that provides the service (exports the filesystem, is a Netscape server, provides the database, and so on) when the node is in normal state.
For each high-availability service, how will the software and data be distributed on shared and non-shared disks?
Each application has requirements and choices for placing its software on disks that are failed over (shared) or not failed over (non-shared).
Are the shared disks going to be part of a CHALLENGE RAID storage system or are they going to be disks in CHALLENGE Vaults that have plexed XLV logical volumes on them?
Shared disks must be part of a CHALLENGE RAID storage system or in CHALLENGE Vaults with plexed XLV logical volumes on them.
Will the shared disks be used as raw XLV logical volumes or XLV logical volumes with XFS filesystems on them?
XLV logical volumes are required by IRIS FailSafe; filesystems must be XFS filesystems. The choice of volumes or filesystems depends on the application that is going to use the disk space.
Which interfaces on primary nodes will be used by clients of high-availability services?
Multiple interfaces may be required on each node because a node could be connected to more than one network or because there could be more than one interface to a single network.
How many high-availability IP addresses on each network interface will be available on primary nodes to clients of the high-availability services?
At least one high-availability IP address must be available for each interface on each primary node that is used by clients of high-availability services.
Which IP addresses on primary nodes are going to be available to clients of the high-availability services?
Each public interface that is used by clients of high-availability services must have at least two IP addresses, one fixed IP address and one high-availability address. The fixed IP address is not available to clients after a failover.
For each high-availability IP address that is available on a primary node to clients of high-availability services, which interface on the other node will be assigned that IP address after a failover?
Every high-availability IP address used by a high-availability service must be mapped to a pair of interfaces, one on the primary node and one on the backup node. The high-availability IP addresses are failed over from the interface in the primary node to the interface in the backup node.
As an example of the configuration planning process, say that you have an IRIS FailSafe cluster that is a departmental server. You want to make four XFS filesystems available for NFS mounting and have two Netscape FastTrack servers, each serving a different set of documents. These applications will be high-availability services.
You decide to distribute the services across the two nodes, so each node will be the primary node for two filesystems and one Netscape server. The filesystems and the document roots for the Netscape servers (on XFS filesystems) are each on their own plexed XLV logical volume. The logical volumes are created from disks in a CHALLENGE RAID storage system connected to both nodes.
Two public interfaces are available on each node, ec0 and ec2. Both are used; ec0 by clients of the NFS filesystems and ec2 by clients of the Netscape servers. Each interface has two IP addresses, one fixed and one high-availability. The high-availability IP address is publicized for use by clients. In the event of failover, the high-availability IP address on ec0 fails over to the ec0 interface on the other node, and the IP address on ec2 fails over to the other ec2 interface.
The following sections help you answer the questions above, make additional configuration decisions required by IRIS FailSafe, and collect the information you need to perform the configuration tasks described in Chapter 3, “Configuring Nodes for IRIS FailSafe,” and Chapter 4, “Creating the IRIS FailSafe Configuration File.”
The first subsection below describes NFS filesystem issues that must be considered when planning an IRIS FailSafe system. The second subsection gives an example of an NFS filesystem configuration on an IRIS FailSafe system. The third subsection explains the aspects of the configuration that must be specified as parameters in the IRIS FailSafe configuration file.
The IRIS FailSafe NFS option enables IRIS FailSafe to provide failover protection for
filesystems that have been configured to be exported by the exportfs command
NFS file-locking information, as supported by rpc.lockd/rpc.statd
In an IRIS FailSafe cluster, one or both nodes can export NFS filesystems. If a node that exports NFS filesystems fails, the other node provides backup service.
When the NFS server on one node fails, the surviving node must take over NFS file-locking information from the failed node. IRIS FailSafe does this by storing the NFS locking state for each exported filesystem. The information for each node is stored on that node in a single directory that is in one of the filesystems on the shared disk. This directory is called the statmon directory and its name must be statmon.
Do not place NFS filesystems on shared disks in the /etc/exports file. IRIS FailSafe exports these filesystems only after making sure that the other node does not have these filesystems exported.
 | Note: When clients are actively writing to a FailSafe NFS filesystem during filesystem failover, data corruption can occur unless filesystems are exported with the mode wsync. This mode requires that local mounts of the XFS filesystems use the wsync mount mode as well. Using wsync can affect performance considerably. See the section “Wsync Filesystem Options” in Chapter 4 for more information. |
For example, if the node named stocks exports the NFS filesystem /shared1 and the node named bonds exports the NFS filesystems /shared2 and /shared3, the two directories /shared1/statmon and /shared2/statmon should be created to hold NFS file-locking information. The directory /shared1/statmon stores NFS lock information that allows bonds to recover NFS locks for the /shared1 filesystem if stocks fails. The directory /shared2/statmon stores NFS lock information that allows stocks to recover NFS locks for /shared2 and /shared3 if bonds fails.
Table 2-1 lists the name of the parameter that specifies the statmon directory.
Table 2-1. NFS Configuration Parameters
Parameter | Value for stocks | Value for bonds | Comment |
|---|---|---|---|
statmon-dir | /shared1/statmon | /shared2/statmon | The value is a directory called statmon on any NFS filesystem on a shared disk. It is used to store NFS locks for each node. |
The procedure for configuring NFS filesystems for IRIS FailSafe is described in the section “Configuring NFS Filesystems” in Chapter 3.
Configuration of one or more high-availability Netscape servers on an IRIS FailSafe cluster requires the use of the IRIS FailSafe Web option. One subsection below discusses several choices you must make when configuring Netscape servers. The remaining subsections describe two example Netscape configurations and the configuration parameters for these examples.
In configuring one or more Netscape servers on an IRIS FailSafe cluster, you need to consider the locations of these components:
documents (HTML pages), which are stored in a directory called the document root
access control lists, which are stored in /usr/ns-home/httpacl
log files for Netscape usage accounting
Most Netscape sites run Netscape usage accounting. One common way of accounting for Netscape usage is to parse the access log files. These files are normally held within the httpd configuration directory tree (for example, /usr/ns-home/httpd-<server_id>/logs, where <server_id> is the server identifier), which is within the server root.
Note: The accesses made by the IRIS FailSafe monitoring scripts are recorded in the log file records. Therefore, you must subtract these accesses from the total number of hits to get an accurate count. An almost-accurate way to do this is to eliminate all accesses made from both nodes in the cluster. Doing so also eliminates accesses made by any users on the nodes in the cluster, but because these nodes are servers, removing these accesses should present no serious problems.
Failover of a Netscape server has these requirements:
The Netscape server software is available on each node acting as a Netscape server.
Each Netscape server's configuration information is available on each node.
Log files must be available to each Netscape server on each node.
A document root must be available to each Netscape server on each node.
The IP addresses associated with each Netscape server must fail over.
The primary and backup nodes of the Netscape server, of the server's high-availability IP address, and of the shared disks used by the server, must be the same.
Most Netscape servers on FailSafe clusters should have this configuration:
The Netscape server software and configuration information are duplicated on local disks on each node.
The log files, access control lists, and documents are on shared disks.
To have a Netscape server configuration with no shared disks (server software, configuration information, log files, access control lists, and documents are duplicated on local disks on each node), these requirements must be met:
Each active server must serve identical documents.
The document content must be read-only.
The first subsection below describes an active/backup Netscape server configuration—one node is used as a Netscape server and the other node serves as a backup in case the first node fails. The second subsection explains the aspects of the configuration that must be specified as parameters in the IRIS FailSafe configuration file.
This example is an active/backup configuration with two Netscape Enterprise servers on the primary node xfs-ha1. The backup node is called xfs-ha2. Two IP aliases for a network interface on xfs-ha1 are 190.0.2.3 and 190.0.2.5. One Netscape server uses port number 80 and has its server root in /usr/ns-home/httpd-xfs-ha1. The other Netscape server uses port number 90 and has its server root in /usr/ns-home/httpd-xfs-ha2. The script used to start and stop the Netscape servers is /etc/init.d/ns_enterprise (rather than the default /etc/init.d/ns_fasttrack). The configuration file /etc/config/ns_enterprise.options (instead of the default /etc/config/ns_fasttrack.options) is used to start the Netscape servers.
| Note: Although this example has two Netscape servers on a single node, listening on ports 80 and 90, a typical configuration is a single Netscape server listening to a single port. This example is included to illustrate how to specify two Netscape servers on a single node in the IRIS FailSafe configuration file. |
Table 2-2 shows the various labels, section names, and parameters used in the configuration file for the active/backup example in the previous section.
Table 2-2. Netscape Configuration Parameters (Active/Backup Configuration)
Label, Section, or Parameter | Value | Comments |
|---|---|---|
label | webxfs-ha1 | A unique name. |
server-node | xfs-ha1 | The primary node. |
backup-node | xfs-ha2 | The backup node. |
httpd-script | /etc/init.d/ns_enterprise | The startup and shutdown script for the server. |
httpd-options-file | ns_enterprise.options | The configuration file for the Netscape server. |
webserver-num | 2 | The number of Netscape servers on the primary node. |
section name for the first server | web-config1 | This value is web-config followed by a digit. |
ip-address (for the first server) | 190.0.2.3 | Any IP alias for the primary node in X.X.X.X form. |
port-num (for the first server) | 80 | The port number used by this server. |
httpd-dir (for the first server) | /usr/ns-home/httpd-xfs-ha1 | Server root for the first server. |
section name for the second server | web-config2 | This value is web-config followed by a digit that is different from the other server. |
ip-address (for the second server) | 190.0.2.3 | Any IP alias for the primary node in X.X.X.X form. |
port-num (for the second server) | 90 | The port number used by this server. |
httpd-dir (for the second server) | /usr/ns-home/httpd-xfs-ha2 | Server root for the second server. |
As described in the section “Planning Netscape Server Configuration” in this chapter, the location of the log files, access control lists, and documents depends somewhat on the document content. If it is read-only, the log files, access control lists, and documents can be on local disks on each node or on shared disks. If it is read-write, these files must be on one or more shared disks.
The section “Configuring a Netscape Server” in Chapter 3 describes the procedure for configuring Netscape servers.
The first subsection below describes a dual-active Netscape server configuration—each node serves as a Netscape server during normal operation. If one node fails, the other node takes over the server duties of the failed node. The second subsection explains the aspects of the configuration that must be specified as parameters in the IRIS FailSafe configuration file.
This example is a dual-active configuration with one Netscape server on each of the nodes xfs-ha1 and xfs-ha2. The default startup and shutdown script and configuration file are used. xfs-ha1 has the IP alias 190.0.2.3, and xfs-ha2 has the IP alias 190.0.2.4. Each server uses port 80. The server root on xfs-ha1 is /usr/ns-home/httpd-xfs-ha1. The server root on xfs-ha2 is /usr/ns-home/httpd-xfs-ha2.
Table 2-3 shows the various labels, section names, and parameters used in the configuration file for the dual-active Netscape server example in the previous section.
Table 2-3. Netscape Configuration Parameters (Dual-Active Configuration)
Label, Section, or Parameter | Value for xfs-ha1 | Value for xfs-ha2 | Comments |
|---|---|---|---|
label | webxfs-ha1 | webxfs-ha2 | A unique name. |
server-node | xfs-ha1 | xfs-ha2 | The primary nodes. |
backup-node | xfs-ha2 | xfs-ha1 | The backup nodes. |
webserver-num | 1 | 1 | The number of Netscape servers on the primary node. |
section name for the first server | web-config1 | web-config1 | This value is web-config followed by a digit. |
ip-address | 190.0.2.3 | 190.0.2.4 | An IP alias for the primary node. |
port-num | 80 | 80 | The port number. |
httpd-dir | /usr/ns-home/ httpd-xfs-ha1 | /usr/ns-home/ httpd-xfs-ha2 | The server root. |
As described in the section “Planning Netscape Server Configuration” in this chapter, the location of the log files, access control lists, and documents depends somewhat on the document content. If it is read-write or if it is different for the active servers on each node, the log files, access control lists, and documents must be on one or more shared disks.If the document content is identical for active servers on each node and is read-only, these files can be on local disks on each node or on shared disks.
If local disks are used, the backup node does not restart the Netscape server after failover because the node already has Netscape server processes running. When the failed node returns to normal state, the Netscape server is restarted on the failed node. For this special case of Netscape server configuration, the httpd-restart parameter must be set to true in the webserver block of each server in the IRIS FailSafe configuration file.
The section “Configuring a Netscape Server” in Chapter 3 describes the procedure for configuring Netscape servers.
The first subsection below describes the disk configuration issues that must be considered when planning an IRIS FailSafe system. It explains the basic configurations of shared and non-shared disks and how they are reconfigured by IRIS FailSafe after a failover. The second subsection explains how disk configurations are included in the IRIS FailSafe configuration file.
For each disk in an IRIS FailSafe cluster, you must choose whether to make it a shared disk, which enables it to be failed over, or a non-shared disk. Non-shared disks are not failed over.
Both nodes in an IRIS FailSafe cluster must follow these requirements:
The system disk must be a non-shared disk.
The IRIS FailSafe software, in particular the directory /var/ha, must be on a non-shared disk.
Choosing to make a disk shared or non-shared depends on the needs of the high-availability services that use the disk. Each high-availability service has requirements about the location of data associated with the service:
Some data must be placed on non-shared disks.
Some data must not be placed on shared disks.
Some data can be on shared or non-shared disks.
The figures in the remainder of this section show the basic disk configurations on IRIS FailSafe clusters before failover. Each figure also shows the configuration after failover. The basic disk configurations are these:
a non-shared disk on each node
a shared disk in an active/backup cluster
a shared disk in a dual-active cluster
In each of the before and after failover diagrams, just one or two disks are shown. In fact, many disks could be connected in the same way as each disk shown. Thus each disk shown can represent a set of disks.
An IRIS cluster can contain a combination of the basic disk configurations listed above.
Figure 2-1 shows two nodes in an IRIS FailSafe cluster, each of which has a non-shared disk. When non-shared disks are used by high-availability applications, the data required by those applications must be duplicated on non-shared disks on both nodes. When a failover occurs, IP aliases fail over. The data that was originally available on the failed node is still available from the surviving node by using the IP alias to access it.
Figure 2-2 shows a shared disk in an active/backup configuration. In this configuration, the disk has a primary node, which is the node that accesses the disk prior to a failover. It is shown by a solid line connection. The backup node, which accesses the disk after a failover, is shown by a dotted line. Thus, the disk is shared between the nodes. In an active/backup cluster, all shared disks have the same primary node. The backup node doesn't run any high-availability applications until a failover occurs.
Figure 2-3 shows two shared disks in a dual-active cluster. In this case, each node serves as a primary node for one shared disk. The solid line connections show the connection to the primary node prior to failover. The dotted lines show the connections to the backup nodes. After a failover, the surviving node accesses all shared disks.
Other sections in this chapter and similar sections in the IRIS FailSafe Oracle Administrator's Guide , IRIS FailSafe INFORMIX Administrator's Guide , and IRIS FailSafe Sybase Administrator's Guide provide more specific information about choosing between shared and non-shared disks for various types of data associated with each high-availability service.
The choice of an active/backup cluster or a dual-active cluster is based on your preference: in an active/backup cluster, one node serves only as a “hot spare” for high-availability applications on the other node. In a dual-active cluster, the high-availability applications, and therefore the load, are distributed between the nodes.
There are no configuration parameters associated with non-shared disks. They are not specified in the IRIS FailSafe configuration file. Only shared disks (actually, the XLV logical volumes on shared disks) are specified in the configuration file. See the section “Configuration Parameters for Logical Volumes” in this chapter for details.
The first subsection below describes logical volume issues that must be considered when planning an IRIS FailSafe system. The second subsection gives an example of an XLV logical volume configuration on an IRIS FailSafe system. The third subsection explains the aspects of the configuration that must be specified as parameters in the IRIS FailSafe configuration file.
All shared disks must have XLV logical volumes on them. You can work with XLV logical volumes on shared disks as you would work with other disks. However, for correct operation of the IRIS FailSafe configuration, you must follow these rules:
All data that is used by high-availability applications on shared disks must be stored in XLV logical volumes.
XLV allows multiple volumes to be created on the same physical disk. In an IRIS FailSafe environment, if you create more than one volume on a single disk, they must all be owned by the same node. For example, if a disk has two partitions that are part of two XLV volumes, both XLV volumes must be owned by the same node. (See the sections “Creating XLV Logical Volumes and XFS Filesystems” in Chapter 3 and “Trouble With Logical Volumes” in Appendix B for more information about XLV volume ownership.)
Each disk in a CHALLENGE Vault or RAID LUN must be owned by one of the IRIS FailSafe nodes. (A disk is owned by the node that is its primary node.) Therefore, you must divide the Vault disks and RAID LUNs into two sets, one for each node. If you create multiple volumes on a Vault disk or RAID LUN, all those volumes must be owned by one node.
Do not access a shared XLV volume from both nodes simultaneously. Doing so causes data corruption.
The IRIS FailSafe software relies on the XLV naming scheme to operate correctly. A fully qualified XLV volume name is pathname/volname or pathname/nodename.volname. The components are these:
pathname, which is /dev/dsk/xlv or /dev/rdsk/xlv for IRIX 6.2 and /dev/xlv or /dev/rxlv for IRIX 6.4
nodename, which by default is the same as the hostname of the node the volume was created on
volname, a name specified when the volume was created; this component is commonly used when a volume is to be operated on by any of the XLV tools
For example, if volume vol1 is created on node ha1 using disk partitions located on a shared disk, the raw character device name for the assembled volume is /dev/rdsk/xlv/vol1 on IRIX 6.2 and /dev/rxlv/vol1 on IRIX 6.4. On the peer ha2, however, the same raw character volume appears as /dev/rdsk/xlv/ha1.vol1 on IRIX 6.2 and /dev/rxlv/ha1.vol1 on IRIX 6.4, where ha1 is the nodename component, and vol1 is the volname component. As can be seen from this example, when the nodename component is the same as the local hostname, it does not appear as part of the device node name.
One nodename is stored in each disk or LUN volume header. This is why all volumes with volume elements on any single disk must have the same nodename component. If this rule is not followed, the IRIS FailSafe software does not operate correctly.
The IRIS FailSafe software modifies the nodename component of the volume header as volumes are transferred between nodes during failover and recovery operations. This is important because xlv_assemble assembles only those volumes whose nodename matches the local hostname. Some of the other XLV utilities allow you to see (and modify) all volumes, regardless of which node owns them.
If you use XLV logical volumes as raw volumes (no filesystem) for storing database data, the database system may require that the device names (in /dev/rdsk/xlv and /dev/dsk/xlv on IRIX 6.2 and in /dev/rxlv and /dev/xlv on IRIX 6.4) have specific owners, groups, and modes. See the documentation provided by the database vendor to determine if the XLV logical volume device names must have owners, groups, and modes that are different from the default values (the default owner, group, and mode for XLV logical volumes are root, sys, and 0600).
As an example of XLV logical volume configuration, say that you have these logical volumes on four disks on an IRIX 6.4 system that we will call disk 1 through disk 5:
A logical volume called /dev/xlv/volA (volume A) that contains disk 1 and a portion of disk 2.
A logical volume called /dev/xlv/volB (volume B) that contains the remainder of disk 2 and disk 3.
A logical volume called /dev/xlv/volC (volume C) that contains disks 4 and 5.
Volumes A and B must have the same primary node (stocks, for example), because they share a disk. Volume C could have either node as its primary node. For this example, say that bonds is the primary node, which makes this a dual-active configuration.
Configuration parameters for XLV logical volumes list
the primary node and backup node for each volume
the nodes that act as primary nodes (one node for active/backup clusters and two nodes for dual-active clusters)
the owner, group, and mode of the device filename for the volume
Table 2-4 lists a label and parameters for individual logical volumes.
Table 2-4. XLV Logical Volume Configuration Parameters
Label or Parameter | Value for Volume A | Value for Volume B | Value for Volume C | Comments |
|---|---|---|---|---|
volume label | volA | volB | volC | Each volume is given a label to identify it in the configuration file. |
server-node | stocks | stocks | bonds | The primary node. |
backup-node | bonds | bonds | stocks | The backup node. |
devname | volA | volB | volB | The volume name of the XLV logical volume. |
devname-owner | root | root | root | The owner of the device name. |
devname-group | sys | sys | root | The group of the device name. |
devname-mode | 0600 | 0600 | 0600 | The mode of the device name. |
The server-node parameter is used in another context as well. Each node specified as a server-node in Table 2-4 is also specified as a server-node in a block of the configuration file called “application-class volumes.” This block lists each node that is a primary node (one node for active/backup clusters and two nodes for dual-active clusters).
See the section “Creating XLV Logical Volumes and XFS Filesystems” in Chapter 3 for information about creating XLV logical volumes.
The first subsection below describes filesystem issues that must be considered when planning an IRIS FailSafe system. The second subsection gives an example of an XFS filesystem configuration on an IRIS FailSafe system. The third subsection explains the aspects of the configuration that must be specified as parameters in the IRIS FailSafe configuration file.
The IRIS FailSafe software supports the automatic failover of XFS filesystems on shared disks. Shared disks must be in CHALLENGE Vault or RAID storage systems that are shared between the two nodes in the IRIS FailSafe cluster.
The following are special issues that you need to be aware of when you are working with filesystems on shared disks in an IRIS FailSafe cluster:
All filesystems to be failed over must be XFS filesystems.
All filesystems to be failed over must be created on XLV logical volumes on shared disks.
For availability, filesystems to be failed over in an IRIS FailSafe cluster must be created on either mirrored disks (using the XLV plexing software) or on the CHALLENGE RAID storage system.
Make filesystems for which this node is primary only on those volumes owned by this node. Create the mount points for these filesystems on both nodes.
When you set up the various IRIS FailSafe filesystems on each node, make sure that each filesystem uses a different mount point. For example, do not use the mount point /shared on each node. In the degraded state, in which all filesystems are owned by the surviving node, that node must be able to concurrently service requests for all filesystems. If the mount point directories are different, such as /shared1 and /shared2, the surviving node can ensure that all NFS requests sent to the failed node are handled properly, even though they are being handled by the surviving node.
Do not simultaneously mount filesystems on shared disks on both nodes. Doing so causes data corruption. Normally, IRIS FailSafe performs all mounts of filesystems on shared disks. If you manually mount a filesystem on a shared disk, make sure that it is not being used by the other node.
Do not place filesystems on shared disks in the /etc/fstab file. IRIS FailSafe mounts these filesystems only after making sure that the other node does not have these filesystems mounted.
| Note: When clients are actively writing to a FailSafe NFS filesystem during failover of filesystems, data corruption can occur unless filesystems are exported with the mode wsync. This mode requires that local mounts of the XFS filesystems use the wsync mount mode as well. Using wsync affects performance considerably. See the section “Wsync Filesystem Options” in Chapter 4 for more information. |
Continuing with the example configuration from the section “Example Logical Volume Configuration” in this chapter, say that volumes A and B have XFS filesystems on them:
The filesystem on volume A is mounted at /sharedA with modes rw and noauto. Call it filesystem A.
The filesystem on volume B is mounted at /sharedB with modes rw and noauto. Call it filesystem B.
Table 2-5 lists a label and configuration parameters for each filesystem.
Table 2-5. Filesystem Configuration Parameters
Label or Parameter | Value for Filesystem A | Value for Filesystem B | Comments |
|---|---|---|---|
filesystem label | filesysA | filesysB | Each filesystem is given a label to identify it in the configuration file. |
mount-point | /sharedA | /sharedB | The filesystem mount point (on the primary and backup nodes). |
volume-name | volA | volB | The label of the logical volume on which the filesystem was created. |
mode | rw,noauto | rw,noauto,wsync | The modes of the filesystem (identical to the modes specified in /etc/fstab). |
See the section “Creating XLV Logical Volumes and XFS Filesystems” in Chapter 3 for information about creating XFS filesystems.
The first subsection below describes network interface and IP address issues that must be considered when planning an IRIS FailSafe system. The second subsection gives an example of the configuration of network interfaces and IP addresses on an IRIS FailSafe system. The third subsection explains the aspects of the configuration that must be specified as parameters in the IRIS FailSafe configuration file.
Follow these guidelines when planning the configuration of the interfaces to the private network between nodes in a cluster:
Each interface has one IP address.
The IP addresses used on each node for the interfaces to the private network are on a different subnet from the IP addresses used for public networks.
An IP name can be specified for each IP address in /etc/hosts.
Choosing a naming convention for these IP addresses that identifies them with the private network can be helpful. For example, precede the hostname with “priv-” (for private), as in priv-xfs-ha1 and priv-xfs-ha2.
Follow these guidelines when planning the configuration of the node interfaces in a cluster to one or more public networks:
If re-MACing is required, each interface to be failed over requires a dedicated backup interface on the other node (an interface that does not have a high-availability IP address and is not used while the cluster is in normal state). Thus, a cluster that is used in an active/backup configuration requires just one public interface on each node minimum. A cluster that is used in a dual-active configuration requires a minimum of two interfaces to the public network on each node: one to be a primary interface and the other to be a dedicated backup interface to the primary interface on the other node.
Each interface has one fixed IP address (fixed IP addresses don't fail over).
Each fixed IP address has an IP name, which can be specified in /etc/hosts.
On each node, one IP name for a fixed IP address must be the same as the hostname of the node. In other words, the hostname cannot be a high-availability IP address.
Each interface used by clients to access high-availability services must have at least one IP alias.
If re-MACing is required, all of the IP aliases configured to a single interface must have the same backup interface.
Making good choices for IP aliases is important; these are the “hostnames” that will be used by users of the high-availability services, not the true hostnames of the nodes.
Make a plan for publicizing the IP aliases to the user community, since users of high-availability services must use IP aliases (high-availability IP addresses) instead of the output of the hostname command. See the section “Educating the User Community About IRIS FailSafe” in Chapter 6 for more information.
Follow the procedure below to determine whether re-MACing is required (see the section “Network Interfaces and IP Addresses” in Chapter 1 for information about re-MACing). It requires the use of three nodes: node1, node2, and node3. node1 and node2 can be the nodes of an IRIS FailSafe cluster, but they need not be. They must be on the same subnet. node3 is a third node. If you need to verify that a router accepts gratuitous ARP packets (which means that re-MACing is not required), node3 must be on the other side of the router from node1 and node2.
Configure an IP address on one of the interfaces of node1:
# /usr/etc/ifconfig interface inet ip_address netmask netmask up
interface is the interface to be used access the node. ip_address is an IP address for node1. This IP address is used throughout this procedure. netmask is the netmask of the IP address.
From node3, ping the IP address used in step 1:
# ping -c 2 ip_address PING 190.0.2.1 (190.0.2.1): 56 data bytes 64 bytes from 190.0.2.1: icmp_seq=0 ttl=255 time=29 ms 64 bytes from 190.0.2.1: icmp_seq=1 ttl=255 time=1 ms ----190.0.2.1 PING Statistics---- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max = 1/1/1 ms
Enter this command on node1 to shut down the interface you configured in step 1:
# /usr/etc/ifconfig interface down
On node2, enter this command to move the IP address to node2:
# /usr/etc/ifconfig interface inet ip_address netmask netmask up
From node3, ping the IP address:
# ping -c 2 ip_address
If the ping command fails, gratuitous ARP packets are not being accepted and re-MACing is needed to fail over the IP address.
Figure 2-4 shows an example of an IRIS FailSafe cluster configured with one public interface on each node.
The components of this diagram are as follows:
| public network |
| |
| ec0, ec3 | An interface name, as reported by the Name column in the output of the command netstat -i. Examples of interface names are ec0, fxp0, and xpi0. | |
| xfs-ha1, xfs-ha2 |
| |
| stocks, bonds | These names are IP aliases. They are IP addresses in internet notation, X.X.X.X, or IP names. IP aliases are known as high-availability IP addresses. If their node fails, these IP aliases failover—they become IP addresses for the backup (secondary) interface associated with this interface. | |
| 190.0.2.1, 190.0.2.2 |
| |
| priv-xfs-ha1, priv-xfs-ha2 |
| |
| private Ethernet |
|
In configuring the interfaces on your IRIS FailSafe cluster, it is very helpful to draw a diagram similar to Figure 2-4 for your configuration.
The interface names, IP addresses, IP aliases, and so on in Figure 2-4 are used in the IRIS FailSafe configuration file that you develop as you configure an IRIS FailSafe system. Table 2-6 lists the per-node interface-related configuration file parameters and the values they would have for the example shown in Figure 2-4.
Table 2-6. Per-Node Interface Configuration Parameters
Parameter | Value or Label for Node xfs-ha1 | Value or Label for Node xfs-ha2 | Comments |
|---|---|---|---|
interface | xfs-ha1-ec0 | xfs-ha2-ec0 | A unique name for the public interface. A suggestion is a combination of the IP address (name form) and the interface name. |
name | ec0 | ec0 | The public interface name. |
ip-address | xfs-ha1 | xfs-ha2 | The fixed IP address of the public interface. |
hb-private-ipname | priv-xfs-ha1 | priv-xfs-ha2 | The IP address of the private interface. |
hb-public-ipname | xfs-ha1 | xfs-ha2 | The IP address of a public interface that will provide an alternative interface for heartbeat messages if the private network fails. |
Table 2-7 lists the information about the high-availability IP addresses in the cluster. High-availability IP address information is organized in terms of network interface pairs. Each pair of network interfaces consists of an interface called the primary interface and an interface called the secondary interface. These interfaces are on different nodes. The primary interface owns a set (one or more) of high-availability IP addresses in normal state. The high-availability IP addresses are moved to the secondary interface when there is a failover.
Table 2-7. Per-Interface Configuration Parameters
Parameter | Value or Label for Interface xfs-ha1-ec0 | Value or Label for Interface xfs-ha2-ec0 | Comments |
|---|---|---|---|
interface-pair label | one | two | Each interface-pair is given an arbitrary label. There is an interface-pair for each primary interface. |
primary-interface | xfs-ha1-ec0 | xfs-ha2-ec0 | These match the interface labels from the first line of Table 2-6. |
secondary-interface | xfs-ha2-ec0 | xfs-ha1-ec0 | These match interface labels from the first line of Table 2-6 and are the backup interface for the primary interface of this interface-pair. |
ip-aliases | stocks | bonds | The IP alias for the primary interface of this interface-pair. |
If MAC address impersonation is necessary, the parameter re-mac is used. Table 2-8 shows examples of the parameter for the MAC address for each node.
Table 2-8. Interface MAC Address Configuration Parameters
Parameter | Value for Node xfs-ha1 | Value for Node xfs-ha2 | Comment |
|---|---|---|---|
re-mac | false | false | Possible values are true and false. Set to true if MAC address impersonation is necessary. |
MAC-address | 8:0:69:9:31:f8 | 8:0:69:8:d:1 | Use this parameter only if re-mac is set to true. The value is the MAC address (physical address) exactly as it is output by the macconfig command (it may be missing a leading 0). |
The procedure for configuring interfaces and IP addresses for IRIS FailSafe is described in the section “Configuring Network Interfaces” in Chapter 3.
A serial cable on each node is connected to the remote power control unit or to the system controller port of the other node. This serial cable is used to reset the other node when necessary. The device name of its serial tty port must be specified in the configuration file. Its device name is /dev/ttydport, where port is the serial port number. For example, say that port 2 is being used for the reset serial cable on both nodes. The device name is /dev/ttyd2. More information about serial ports is available in the serial(7) reference page. Table 2-9 shows the configuration parameter that specifies the device name.
Table 2-9. Serial Cable Configuration Parameter
Parameter | Value for Node xfs-ha1 | Value for Node xfs-ha2 | Comment |
|---|---|---|---|
reset-tty | /dev/ttyd2 | /dev/ttyd2 |
|
The serial tty port used by the serial cable must be configured to have its getty process turned off (for more information about getty, see the getty(1M) reference page). For the procedure to configure the port, see the section “Configuring the Serial Ports” in Chapter 3.