On SGI Foundation Software nodes, CXFS supports either an administration node containing the cluster administration daemons (fs2d, crsd, cad , and cmond), the CXFS control daemon ( clconfd), and the cluster database or a client-only node containing the cxfs_client daemon. The software you install on a node determines the node type.
 | Note: SGI Foundation Software is an overlay product that adds or enhances features in the supported Linux base distributions. |
Nodes that you intend to run as metadata servers must be installed as administration nodes; all other nodes should be client-only nodes.
This chapter discusses the following:
After completing these steps, see Chapter 9, “Initial Setup of the Cluster”. For details about specific configuration tasks, see Chapter 10, “CXFS GUI”.
The following limitations and considerations apply to server-capable administration nodes. See also client-only node information in CXFS 5 Client-Only Guide for SGI InfiniteStorage.
By default, DMAPI is turned off on SLES 10 systems. When you install DMF on a server-capable administration node, it automatically enables DMAPI.
However, If you want to mount filesystems on a SLES 10 client-only node with the dmi mount option, you must set DMAPI_PROBE="yes" in the /etc/sysconfig/sysctl file on the node. Changes to the file will processed on the next reboot. After setting that system configuration file, you can immediately enable DMAPI by executing the following:
# sysctl -w fs.xfs.probe_dmapi=1
If you run a DMAPI application other than DMF, you must also change the parameter on the server-capable administration nodes.
CXFS does not support external log or real-time filesystems.
GPT partition tables, often created by operating system installers or the parted partitioning tool, store labels in two locations. If you reuse a disk that previously had a GPT label, you must be careful; using tools such as fdisk to repartition the drive will not eliminate the backup GPT label. When you reboot, EFI scans the disks before the operating system is started. It assumes any backup labels it finds are valid and restores them. This can corrupt or destroy filesystems. You can use the parted tool to detect this situation and fix it.
Note: The parted tool has a mkpartsect command that accepts start and end values for partitions being created in sectors rather than MB. For more information, see the XVM Volume Manager Administrator's Guide and http://support.sgi.com/content_request/838562/index.html on Supportfolio. CXFS filesystems with XFS version 1 directory format cannot be mounted on Linux nodes.
The implementation of file creates using O_EXCL is not complete. Multiple applications running on the same node using O_EXCL creates as a synchronization mechanism will see the expected behavior (only one of the creates will succeed). However, applications running between nodes may not get the O_EXCL behavior they requested (creates of the same file from two or more separate nodes may all succeed).
The CXFS metadata server is only capable of managing permissions for users with 32 or fewer group memberships. All accounts (including root) on CXFS clients must be limited to 32 or fewer groups.
 | Caution: CXFS is a complex product. To ensure that it is installed
and configured in an optimal manner, you must purchase
initial setup services from SGI. You should read through the following
chapters before attempting to install and configure a CXFS cluster:
Also see the CXFS 5 Client-Only Guide for SGI InfiniteStorage. |
Following is the order of installation and configuration steps:
Install the operating system (if not already done). See the CXFS release notes for supported levels.
Install and verify the RAID. See Chapter 3, “SGI RAID for CXFS Clusters”
Install and verify the switch. See Chapter 4, “Switches”.
Obtain and install the CXFS license keys and (if needed) XVM license keys. See Chapter 5, “CXFS License Keys”.
Prepare the node, including adding a private network.
Install the CXFS software. See the SGI InfiniteStorage Software Platform release note. The ISSP release note is located in the /docs/README.txt on the DVD and is available on the Supportfolio download page.
Configure the cluster to define the new node in the pool, add it to the cluster, start CXFS services, and mount filesystems. See “Guided Configuration Tasks” in Chapter 10.
Install the client-only nodes, as described in CXFS 5 Administration Guide for SGI InfiniteStorage.
To verify that the CXFS software has been installed properly, use the rpm -qa command to display all of the installed packages. You can filter the output by searching for particular package name.
For example, to verify that the sgi-sysadm_base-lib package has installed:
admin]# rpm -qa | grep sgi-sysadm_base-lib sgi-sysadm_base-lib-3.0-sgi06092521 |
| Note: The output above is an example. The version level may not match the installed software. |
To verify the SGI Foundation Software release, display the /etc/sgi-foundation-release file.
To verify licenses, see “Verifying the License Keys” in Chapter 5.
If you want to use the cluster diagnostics to test node connectivity, the root user on the node running the CXFS diagnostics must be able to access a remote shell using the rsh command (as root) on all other nodes in the cluster. There are several ways of accomplishing this, depending on the existing settings in the pluggable authentication modules (PAM) and other security configuration files.
Following is one possible method. Do the following on all administration nodes in the cluster:
Install the rsh-server RPM using YaST.
Enable rsh by changing disable yes to disable no in the /etc/xinetd.d/rsh file.
Restart xinetd:
server-capable# service xinetd restart
Add the hostname of the node from which you will be running the diagnostics into the /root/.rhosts file. Make sure that the mode of the .rhosts file is set to 600 (read and write access for the owner only).
After you have completed running the connectivity tests, you may wish to disable rsh on all cluster nodes.
For more information, see the operating system documentation and the hosts.equiv(5) man page.