This chapter discusses the following:
Nodes that you intend to run as metadata servers must be installed as server-capable administration nodes; all other nodes should be client-only nodes.
Server-capable administration nodes contain the cluster administration daemons (fs2d, crsd, cad , and cmond), the CXFS control daemon ( clconfd), and the cluster database.
The following limitations and considerations apply to server-capable administration nodes in addition to the information in “CXFS Restrictions” in Chapter 1:
See also client-only node information in CXFS 7 Client-Only Guide for SGI InfiniteStorage.
DMAPI is required to mount DMF-managed CXFS filesystems. For those nodes that are also DMF servers or DMF parallel data-mover nodes, DMAPI is automatically enabled when installing the dmf or dmf-mover packages.
However, if you have a CXFS server-capable administration node running RHEL that does not have DMF installed, you must edit the /etc/modprobe.d/sgi-cxfs-xvm.conf file to include the following:
install xfs /sbin/modprobe --ignore-install xfs; /sbin/modprobe --ignore-install xfs_dmapi install xfs_dmapi /bin/true |
You must also manually enable DMAPI for SLES 10 and SLES 11 client-only nodes that are not running DMF software. See the Linux chapter of the CXFS 7 Client-Only Guide for SGI InfiniteStorage .
GPT partition tables, often created by operating system installers or the parted partitioning tool, store labels in two locations. If you reuse a disk that previously had a GPT label, you must be careful; using tools such as fdisk to repartition the drive will not eliminate the backup GPT label. When you reboot, EFI scans the disks before the operating system is started. It assumes any backup labels it finds are valid and restores them. This can corrupt or destroy filesystems. You can use the parted tool to detect this situation and fix it.
The implementation of file creates using O_EXCL is not complete. Multiple applications running on the same node using O_EXCL creates as a synchronization mechanism will see the expected behavior (only one of the creates will succeed). However, applications running on multiple nodes may not get the O_EXCL behavior they requested (creates of the same file from two or more separate nodes may all succeed).
If you are installing the CXFS server package on a system that is currently running standalone XVM, you may see messages similar to the following:
WARNING: /lib/modules/NNN-smp/weak-updates/xvm/sgi-xvm-cell.ko needs unknown symbol xvm_trace_enter WARNING: /lib/modules/NNN-smp/weak-updates/xvm/sgi-xvm-cell.ko needs unknown symbol xvm_physlab_ver_to_cur |
You should ignore these messages and reboot the system as documented in the installation instructions.
 | Caution: CXFS is a complex product. To ensure that it is installed
and configured in an optimal manner, you must purchase
initial setup services from SGI. You should read through the following
chapters before attempting to install and configure a CXFS cluster: Also see the CXFS 7 Client-Only Guide for SGI InfiniteStorage. |
Following is the order of installation and configuration steps:
Install the operating system. See the CXFS release notes for supported levels.
Install and verify the RAID. See Chapter 3, “SGI RAID for CXFS Clusters”.
Install and verify the switch. See Chapter 4, “Switch Configuration”.
Obtain and install the CXFS license keys. See Chapter 5, “CXFS Licensing”.
Prepare the node, including adding a private network.
Install the CXFS software. See the SGI InfiniteStorage Software Platform release note. The ISSP release note is located in the /docs/README.txt on the DVD and is available on the download page. For a complete list of the RPM packages and their location on the media, see the /docs/RPMS.txt file.
Configure the cluster to define the new node in the pool, add it to the cluster, start CXFS services, and mount filesystems. See “Guided Configuration Tasks” in Chapter 10.
Install the client-only nodes, as described in CXFS 7 Client-Only Guide for SGI InfiniteStorage.
To verify that the CXFS software has been installed properly, use the rpm -qa command to display all of the installed packages. You can filter the output by searching for particular package name.
For example, to verify that the sgi-sysadm_base-lib package has installed:
# rpm -qa | grep sgi-sysadm_base-lib sgi-sysadm_base-lib-3.0-sgi200a6.sles11 |
 | Note: The output above is an example. The version level may not match the installed software. |
To verify licenses, see “License Key Verification” in Chapter 5.
If you want to use the cluster diagnostics to test node connectivity, the root user on the node running the CXFS diagnostics must be able to access a remote shell using the rsh command (as root) on all other nodes in the cluster. There are several ways of accomplishing this, depending on the existing settings in the pluggable authentication modules (PAM) and other security configuration files.
Following is one possible method. Do the following on all administration nodes in the cluster:
Install the rsh-server RPM.
Enable rsh by changing disable yes to disable no in the /etc/xinetd.d/rsh file.
Restart xinetd:
server-capable# service xinetd restart
Add the hostname of the node from which you will be running the diagnostics into the /root/.rhosts file. Make sure that the mode of the .rhosts file is set to 600 (read and write access for the owner only).
After you have completed running the connectivity tests, you may wish to disable rsh on all cluster nodes.
For more information, see the operating system documentation and the hosts.equiv(5) man page.