CXFS supports a client-only node running the AIX operating system. This chapter contains the following sections:
This section contains the following information about CXFS on AIX:
In addition to the items listed in “Requirements” in Chapter 1, using an AIX node to support CXFS requires the following:
IBM AIX 5L: Version 5.3 Maintenance Level 3 (64-bit mode) APAR number IY71011 or its successor
To verify the operating system level, use the following command:
oslevel -r
IBM FC5716, FC6228, or FC6239 2-Gbit Fibre Channel host bus adapters (HBAs)
One or more of the following IBM hardware platforms:
pSeries 570 pSeries 575 pSeries 595 pSeries 610 pSeries 620 pSeries 630 pSeries 640 pSeries 650 pSeries 660 pSeries 670 pSeries 680 pSeries 690
For the latest information, see the CXFS AIX release notes.
The following commands are shipped as part of the CXFS for AIX package:
| /usr/cxfs_cluster/bin/cxfs_client |
| /usr/cxfs_cluster/bin/cxfs_info |
| /usr/cxfs_cluster/bin/grioadmin |
| /usr/cxfs_cluster/bin/grioqos |
| /usr/cxfs_cluster/bin/xvm |
The cxfs_client and xvm commands are needed to include a client-only node in a CXFS cluster. The cxfs_info command reports the current status of this node in the CXFS cluster.
The lslpp output lists all of the software added; see “Installing CXFS Software on AIX”.
For more information on these commands, see the man pages. For information about the GRIO commands, also “Guaranteed-Rate I/O (GRIO) and CXFS” in Chapter 1 and “GRIO on AIX”.
The cxfs_client command creates a /var/tmp/cxfs_client log file. To rotate this log file, use the -z option in the following file:
/usr/cxfs_cluster/bin/cxfs_client.options |
See the cxfs_client man page for details.
Some daemons related to CXFS output a message in the console log. To see the contents of this log file, use the following command:
alog -o -t console |
The console log is rotated.
For information about the log files created on server-capable administration nodes, see the CXFS Administration Guide for SGI InfiniteStorage.
Also see the AIX /etc/syslog.conf file.
AIX supports the CXFS mount scripts. See “CXFS Mount Scripts” in Chapter 1 and the CXFS Administration Guide for SGI InfiniteStorage.
Although it is possible to mount a JFS or NFS filesystem on top of an AIX CXFS filesystem, this is not recommended.
There is no default access control list (ACL) in AIX. Therefore, the setup and display of the default ACL cannot be completed using the following commands:
aclget aclput acledit
If an IRIX ACL exists, the ACL becomes effective when the default ACL is set up by IRIX and a file and a directory are made under that directory in AIX.
There is no MASK entry in AIX, but the access permissions in AIX follow those established when an ACL set up by Linux contains a MASK entry. If the default ACL is set up for a given directory and the MASK entry exists, then that MASK entry is used when a file or a subdirectory is made by AIX. When the MASK entry does not exist, rwx is used.
ACL control of the following, which the AIX JFS filesystem has, cannot be applied to CXFS:
The access to a certain user or the group is rejected (deny)
When a user belongs to the specific group, access is permitted or rejected (specify)
If deny or specify is used, an error occurs (EINVAL) because these features are not in CXFS.
Socket files cannot be copied. The following error is returned:
AIX:The socket does not allow the requested operation.
You can use the fuser command to extract process information about the mounted filesystem, but you cannot extract process information about the file or the directory.
When a CXFS mount is performed on a mirror volume created by XVM, the AIX system goes into panic status. The mirror volume cannot be mounted on the AIX CXFS system.
The AIX node does not automatically detect the worldwide port number (WWPN). In order to use I/O fencing, you must list the WWPN in the /etc/fencing.conf file. See “License Keys” in Chapter 1.
If your users want to use a file size/offset maximum greater than 1 GB, you must change their user properties to allow files of unlimited size. To do this, use the smit command. For more information, see the smit man page.
By default, the maximum request size for direct I/O is 512 MB (524288 KB). A direct I/O request larger than 512 MB will revert to buffered I/O. However, you can change the maximum XVM direct memory access (DMA) size to improve direct I/O performance. To do this, use the chdev command to modify the xvm_maxdmasz attribute. The actual maximum limit will always be 4 KB less than any of the supplied or displayed values (for example, the default is actually 512 MB minus 4 KB).
Note: The XVM module must be loaded if any attribute changes are to be noticed and applied. To display the current setting, use the following command:
lsattr -E -l xvm -a xvm_maxdmasz
To change the current setting, use the following command:
chdev [-P|-T] -l xvm -a xvm_maxdmasz=NewValue
Legal values for NewValue are specified in KB units in the range 256 to 2097152 (that is, 256 KB to 2 GB).
By default, using chdev on a running system makes a permanent change for subsequently mounted filesystems. (Running filesystems will not be changed until they are remounted, either manually or after a reboot.)
If you use -P, the change is deferred until the next boot and after that it is permanent. If you use -T (temporary), the change is immediate for subsequently mounted filesystems, but lasts only until the next boot.
For example, to change the DMA size to 2 GB for subsequently mounted filesystems on the currently running device and in the database, enter the following:
aix# chdev -l xvm -a xvm_maxdmasz=2097152
For more information, see the lsattr and chdev man pages.
Due to FC controller limitations, large ( > 256K) direct I/O requests may be problematic.
If you use the frametest command on AIX, make sure that the posix_aio0 device is available. Do the following:
Change the setting of the posix_aio0 device to available:
aix# chdev -l posix_aio0 -a autoconfig=available
Add the device to the system:
aix# mkdev -l posix_aio0
Verify that the device is available. For example:
aix# lsdev|grep aio aio0 Defined Asynchronous I/O (Legacy) posix_aio0 Available Posix Asynchronous I/O
For more information about the AIX commands, see their man pages.
For an overview of frametest, see the section about generation of streaming workload for video streams in the CXFS Administration Guide for SGI InfiniteStorage. For details about frametest and its command-line options, see the frametest (1) man page.
See also Appendix B, “Filesystem and Logical Unit Specifications”.
By default, the maximum CXFS I/O request size for normal filesystem I/O is 1 MB (1024 KB). However, depending on filesystem size and internal layout, the actual request size can be smaller or larger:
Requests that are smaller than 1 MB are unaffected by the limit and proceed normally
Requests larger than 1 MB are automatically split into multiple smaller requests in order to accommodate the limit
The cxfs_maxiosz attribute determines the CXFS maximum I/O size request. To display the current setting, use the lsattr command. For example:
aix# lsattr -E -l xvm -a cxfs_maxiosz |
To change the CXFS maximum I/O request size, use the chdev command to modify the cxfs_maxiosz attribute. For example:
aix# chdev [-P|-T] -l xvm -a cxfs_maxiosz=NewValue |
| Note: For attribute changes to be noticed and applied, the XVM module must be loaded. |
Legal values for NewValue are specified in KB units in the range 64 through 2048 (that is, 64 KB to 2 MB).
By default, using chdev on a running system makes a permanent change for subsequently mounted filesystems. (Running filesystems will not be changed until they are remounted, either manually or after a reboot.)
If you use -P, the change is deferred until the next boot and after that it is permanent. If you use -T (temporary), the change is immediate for subsequently mounted filesystems, but lasts only until the next boot.
For example, to change the CXFS maximum I/O request size to 512 KB for subsequently mounted filesystems on the currently running device xvm and in the database, enter the following:
aix# chdev -l xvm -a cxfs_maxiosz=512 |
For more information, see the lsattr and chdev man pages.
There is a possibility that CXFS I/O limits may conflict with AIX's internal disk driver limits. In such cases, you will see console error messages from CXFS that specify an illegal request size error. You can use one of the following ways to correct this problem:
You can decrease CXFS maximum I/O size to match the limit imposed by the AIX disk driver using a procedure similar to the above. This AIX limit is per physical disk drive and is described by the AIX attribute max_transfer. You can display this limit with the lsattr command if you know the name of the physical disk that corresponds to your XVM volume. For example, where hdiskXX is the subsystem name that AIX chooses for each physical disk driver it finds at boot time (the XX number will vary depending upon controller configuration and number of drives):
aix# lsattr -E -l hdiskXX -a max_transfer max_transfer 0x40000 Maximum TRANSFER Size True
The hexadecimal value 0x40000 is 256 KB. From the CXFS error messages on the console, you can find the transfer size that CXFS tried to use; it will likely be hexadecimal 0x80000 (512 KB), which is too large. You can decrease the CXFS maximum I/O size to 256 KB to match AIX's max_transfer limit. This decrease may slightly decrease overall filesystem performance.
You can increase AIX's per-physical-disk max_transfer attribute to 512 KB to match the CXFS maximum I/O request size. You must perform the following command for each physical disk that is part of the cluster configuration:
aix# chdev -l hdiskXX -a max_transfer=0x80000
You can verify the change by using lsattr command as described above.
After modifying AIX's disk driver limits, you must reboot the machine to allow the changes to take effect.
All CXFS files have UNIX mode bits (read, write, and execute) and optionally an ACL. For more information, see the AIX chmod, acledit, aclget, and aclput man pages.
If you want to use an AIX node to restore a CXFS file with an ACL, you should use the backup and restore commands. If you use the tar, cpio, or pax command, the ACL will not be used because these tools behave "intelligently" by not calling acl subroutines to set an ACL. These tools will only set the file mode.
When using the ls command to display access permissions for a file with an ACL, the mode reported for a CXFS file follows Linux semantics instead of AIX JFS semantics.
The CXFS model calls for reporting the ACL MASK for the group permission in the mode. Therefore, if the GROUP entry is r-x and the MASK entry is rw-, the group permission will be reported as rw-. Although it appears that the group has write permission, it does not and an attempt to write to the file will be rejected. You can obtain the real (that is, effective) group permission by using the AIX aclget command.
| Note: Normally, AIX filesystem ACLs can have up to one memory page (4096 bytes) for a file and a directory. However, CXFS filesystems on AIX nodes in a multiOS cluster must maintain compatibility with the metadata server. The CXFS filesystems on an AIX node are limited to a maximum of 25 ACL entries converted to Linux ACL type for a file and a directory. |
IBM hosts running the AIX 5L operating system set the QERR mode page bit to 1 for support storage (other than IBM storage), which does not work well with SGI ProPack for Linux server-capable administration nodes: Linux will leave CTQ enabled but suffer from timeouts.
There is an administrative work-around for this problem. Engenio offers an enhanced feature called SANshare for storage partioning. There is an additional licensing cost required to obtain a SANshare license. SANshare allows hosts to be grouped separately and still access the same LUNs, thus allowing the IBM AIX 5L hosts to set the QERR mode page bit to 1 and not affect the other hosts accessing the LUN.
For each RAID unit, create one Host Group for all of the AIX 5L systems and the other hosts in the CXFS cluster. Set the Host Type to LINUX for the AIX nodes and to SGIAVT for the other nodes. (The Host Type of AIX has AVT status disabled.)
For information about installing and configuring the host bus adapter (HBA), see the IBM HBA documentation.
This section provides an overview of the steps that you or a qualified IBM service representative will perform on your AIX nodes prior to installing the CXFS software. It contains the following sections:
The following procedure provides an overview of the steps required to add a private network to the AIX system. A private network is required for use with CXFS. See “Use a Private Network” in Chapter 2.
You may skip some steps, depending upon the starting conditions at your site. For details about any of these steps, see the AIX documentation.
If your system is already operational and on the network, skip to step 2. If the AIX operating system has not been installed, install it in accordance with the AIX documentation.
Edit the /etc/hosts file so that it contains entries for every node in the cluster and their private interfaces.
The /etc/hosts file has the following format, where primary_hostname can be the simple hostname or the fully qualified domain name:
IP_address primary_hostname aliases
You should be consistent when using fully qualified domain names in the /etc/hosts file. If you use fully qualified domain names on a particular node, then all of the nodes in the cluster should use the fully qualified name of that node when defining the IP/hostname information for that node in the /etc/hosts file.
The decision to use fully qualified domain names is usually a matter of how the clients (such as NFS) are going to resolve names for their client server programs, how their default resolution is done, and so on.
Even if you are using the domain name service (DNS) or the network information service (NIS), you must add every IP address and hostname for the nodes to /etc/hosts on all nodes.
For example:
190.0.2.1 server1.company.com server1 190.0.2.3 stocks 190.0.3.1 priv-server1 190.0.2.2 server2-.company.com server2 190.0.2.4 bonds 190.0.3.2 priv-server2
You should then add all of these IP addresses to /etc/hosts on the other nodes in the cluster.
Note: Exclusive use of NIS or DNS for IP address lookup for the nodes will reduce availability in situations where the NIS or DNS service becomes unreliable. For more information, see “Understand Hostname Resolution and Network Configuration Rules” in Chapter 2 and the hosts, named, and nis man pages.
(Optional) Edit the /etc/netsvc.conf file so that local files are accessed before either NIS or DNS. That is, the hosts line in /etc/netsvc.conf must list local first. For example:
hosts = local,nis,bind
(The order of nis and bind is not significant to CXFS, but local must be first.)
Determine the name of the private interface by using the ifconfig command as follows, to list the available networks. For example:
# ifconfig -l en0 en1 lo0
However, if the second network interface (en1) does not appear, then the network interface must be set up in accordance with the AIX documentation.
You can set up an IP address by using ifconfig after restarting the system. If it is set up properly, the following information is output (line breaks added here for readability):
# ifconfig -a en0:flags=4e080863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,PSEG> inet 10.208.148.61 netmask 0xffffff00 broadcast 10.208.148.255 en1:flags=7e080863,10<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRPT,64BIT, CHECKSUM_OFFLOAD,CHECKSUM_SUPPORT,RSEG> inet 192.168.10.61 netmask 0xffffff00 broadcast 192.168.10.255 lo0:flags=e08084b<UP,BROADCAST,LOOPBACK,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT> inet 127.0.0.1 netmask 0xff000000 broadcast 127.255.255.255
(Optional) Edit the /.rhosts file if you want to use remote access or if you want to use the connectivity diagnostics with CXFS. Make sure that the mode of the .rhosts file is set to 600 (read and write access for the owner only).
Make sure that the /.rhosts file on each AIX node allows all of the nodes in the cluster to have access to each other. The connectivity tests execute a ping command from the local node to all nodes and from all nodes to the local node. To execute ping on a remote node, CXFS uses rsh as user root.
For example, suppose you have a cluster with three nodes: propack0, aix1, and aix2. The /.rhosts files could be as follows (where the prompt denotes the node name):
propack0# cat /.rhosts aix1 root aix2 root aix1# cat /.rhosts propack0 root aix2 root aix2# cat /.rhosts propack0 root aix1 root
For each private network on each AIX node in the pool, verify access with the AIX ping command. Enter the following, where nodeIPaddress is the IP address of the node:
/usr/sbin/ping -c 3 nodeIPaddress |
For example:
aix# /usr/sbin/ping -c 3 192.168.10.61 PING 192.168.10.61: (192.168.10.61): 56 data data bytes 64 bytes from 192.168.10.61 icmp_seq=0 ttl=255 time=0 ms 64 bytes from 192.168.10.61 icmp_seq=1 ttl=255 time=0 ms 64 bytes from 192.168.10.61 icmp_seq=2 ttl=255 time=0 ms ----192.168.10.61 PING Statistics---- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0/0/00 ms |
You should also execute a ping on the public networks. If that ping fails, follow these steps:
Verify that the network interface was configured up. For example:
aix# /usr/sbin/ifconfig en0 en0: flgs=4e08086<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,PSEG> inet 10.208.148.61 netmask 0xffffff00 broadcast 10.208.148.255
In the first output line above, UP indicates that the interface was configured up.
Verify that the cables are correctly seated. Repeat this procedure on each node.
The CXFS software initially will be installed and configured by SGI personnel. This section discusses the following:
Installing CXFS for AIX requires approximately 20 MB of space. To install the required software on an AIX node, SGI personnel will do the following:
Read the SGI InfiniteStorage Software Platform release notes, CXFS general release notes, and CXFS AIX release notes from the /docs directory on the ISSP DVD and any late-breaking caveats on Supportfolio.
Verify that the node has been upgraded to the supported AIX version according to the AIX documentation. Use the following command to display the currently installed system:
oslevel -r
For example, the following output indicates AIX version 5, revision 3, maintenance level 03:
aix# oslevel -r 5300-03
Transfer the client software that was downloaded onto a server-capable administration node during its installation procedure using ftp, rcp, or scp . The location of the tarball on the server will be as follows:
/usr/cluster/client-dist/CXFS_VERSION/aix/53ml3/noarch/SGIcxfs-aix5L
Install the CXFS software (the example output below is truncated):
aix# installp -a -d SGIcxfs-aix5L all +-----------------------------------------------------------------------------+ Pre-installation Verification... +-----------------------------------------------------------------------------+ Verifying selections...done Verifying requisites...done Results... SUCCESSES --------- Filesets listed in this section passed pre-installation verification and will be installed. Selected Filesets ----------------- SGIcxfs-aix5L 5.0.0.5 # CXFS CLIENT for AIX << End of Success Section >> FILESET STATISTICS ------------------ 1 Selected to be installed, of which: 1 Passed pre-installation verification ---- 1 Total to be installed +-----------------------------------------------------------------------------+ Installing Software... +-----------------------------------------------------------------------------+ installp: APPLYING software for: SGIcxfs-aix5L 5.0.0.5 . . . . . << Copyright notice for SGIcxfs-aix5L >> . . . . . . . ... Finished processing all filesets. (Total time: 4 secs). +-----------------------------------------------------------------------------+ Summaries: +-----------------------------------------------------------------------------+ Installation Summary -------------------- Name Level Part Event Result ------------------------------------------------------------------------------- SGIcxfs-aix5L 5.0.0.5 USR APPLY SUCCESS SGIcxfs-aix5L 5.0.0.5 ROOT APPLY SUCCESS
To verify that the CXFS software has been installed properly, use the lslpp command as follows:
aix# lslpp -L SGIcxfs-aix5L |
For example, the following output (showing a state of C, for “committed”) indicates that the CXFS package installed properly:
aix# lslpp -L SGIcxfs-aix5L Fileset Level State Type Description (Uninstaller) ---------------------------------------------------------------------------- SGIcxfs-aix5L 5.0.0.5 C F CXFS CLIENT for AIX State codes: A -- Applied. B -- Broken. C -- Committed. E -- EFIX Locked. O -- Obsolete. (partially migrated to newer version) ? -- Inconsistent State...Run lppchk -v. Type codes: F -- Installp Fileset P -- Product C -- Component T -- Feature R -- RPM Package |
I/O fencing is required on AIX nodes in order to protect data integrity of the filesystems in the cluster. The /etc/fencing.conf file enumerates the worldwide port name (WWPN) for all of the host bus adapters (HBAs) that will be used to mount a CXFS filesystem. The /etc/fencing.conf file must contain a simple list of WWPNs as 64-bit hexadecimal numbers, one per line. These HBAs will then be available for fencing.
If you want to use the /etc/fencing.conf file, you must update it whenever the HBA configuration changes, including the replacement of an HBA.
Do the following:
Follow the Fibre Channel cable on the back of the AIX host to determine the port to which it is connected in the switch. Ports are numbered beginning with 0. (For example, if there are 8 ports, they will be numbered 0 through 7.)
Use the telnet command to connect to the switch and log in as user admin (the password is password by default).
Execute the switchshow command to display the switches and their WWPNs. For example:
brocade04:admin> switchshow switchName: brocade04 switchType: 2.4 switchState: Online switchRole: Principal switchDomain: 6 switchId: fffc06 switchWwn: 10:00:00:60:69:12:11:9e switchBeacon: OFF port 0: sw Online F-Port 20:00:00:01:73:00:2c:0b port 1: cu Online F-Port 21:00:00:e0:8b:02:36:49 port 2: cu Online F-Port 21:00:00:e0:8b:02:12:49 port 3: sw Online F-Port 20:00:00:01:73:00:2d:3e port 4: cu Online F-Port 21:00:00:e0:8b:02:18:96 port 5: cu Online F-Port 21:00:00:e0:8b:00:90:8e port 6: sw Online F-Port 20:00:00:01:73:00:3b:5f port 7: sw Online F-Port 20:00:00:01:73:00:33:76 port 8: sw Online F-Port 21:00:00:e0:8b:01:d2:57 port 9: sw Online F-Port 21:00:00:e0:8b:01:0c:57 port 10: sw Online F-Port 20:08:00:a0:b8:0c:13:c9 port 11: sw Online F-Port 20:0a:00:a0:b8:0c:04:5a port 12: sw Online F-Port 20:0c:00:a0:b8:0c:24:76 port 13: sw Online L-Port 1 public port 14: sw No_Light port 15: cu Online F-Port 21:00:00:e0:8b:00:42:d8
The WWPN is the hexadecimal string to the right of the port number. For example, the WWPN for port 0 is 2000000173002c0b. (You must remove the colons from the WWPN reported in the switchshow output to produce the string to be used in the /etc/fencing.conf file.)
Edit or create the /etc/fencing.conf file on the AIX node and add the WWPN for the port determined in step 1. (Comment lines begin with a # character.) For example, if you determined that port 0 is the port connected to the switch, your /etc/fencing.conf file should appear as follows:
2000000173002c0b
To configure fencing, see the CXFS Administration Guide for SGI InfiniteStorage.
The /usr/cxfs_cluster/bin/cxfs_cluster script will be invoked automatically during normal system startup and shutdown procedures. This script starts and stops the processes required to run CXFS.
To start up cxfs_client manually, enter the following:
aix# /usr/cxfs_cluster/bin/cxfs_cluster start |
To stop cxfs_client manually, enter the following:
aix# /usr/cxfs_cluster/bin/cxfs_cluster stop |
To stop and then start cxfs_client manually, enter the following:
aix# /usr/cxfs_cluster/bin/cxfs_cluster restart |
This section contains the following:
To upgrade the CXFS software on an AIX system, do the following:
Make sure that no applications on the node are accessing files on a CXFS filesystem.
Determine the name of the CXFS package that is installed. For example:
aix# lslpp -L | grep cxfs SGIcxfs-aix5L 5.0.0.5 C F CXFS CLIENT for AIX
Uninstall the old version by using the following command:
installp -u packagename
For example, given a package name of SGIcxfs-aix5L:
aix# installp -u SGIcxfs-aix5L
Obtain the CXFS update software from Supportfolio according to the directions in CXFS Administration Guide for SGI InfiniteStorage.
Install the new version. See “Client Software Installation for AIX”.
You can modify the behavior of the CXFS client daemon (cxfs_client ) by placing options in the /usr/cxfs_cluster/bin/cxfs_client.options file. The available options are documented in the cxfs_client man page.
 | Caution: Some of the options are intended to be used internally by SGI only for testing purposes and do not represent supported configurations. Consult your SGI service representative before making any changes. |
CXFS supports guaranteed-rate I/O (GRIO) version 2 on the AIX platform. Application bandwidth reservations must be explicitly released by the application before exit. If the application terminates unexpectedly or is killed, its bandwidth reservations are not automatically released and will cause a bandwidth leak. If this happens, the lost bandwidth could be recovered by rebooting the client node.
An AIX client can mount a GRIO-managed filesystem and supports application- and node-level reservations. An AIX client will interoperate with the dynamic bandwidth allocator for all I/O outside of any reservation.
For more information, see “Guaranteed-Rate I/O (GRIO) and CXFS” in Chapter 1 and the Guaranteed-Rate I/O Version 2 Guide.
Following is an example of the /etc/failover2.conf file on AIX:
/dev/hdisk199 affinity=1 preferred /dev/hdisk135 affinity=1 /dev/hdisk231 affinity=2 /dev/hdisk167 affinity=2 |
For more information, see “XVM Failover and CXFS” in Chapter 1, the comments in the /etc/failover2.conf file, CXFS Administration Guide for SGI InfiniteStorage, and the XVM Volume Manager Administrator's Guide.
This section discusses the following:
Also see Chapter 11, “General Troubleshooting” and Appendix D, “Error Messages”.
If cxfs_info reports that cms is up but XVM or the filesystem is in another state, then one or more mounts is still in the process of mounting or has failed to mount.
The CXFS node might not mount filesystems for the following reasons:
The client may not be able to see all the LUNs. This is usually caused by misconfiguration of the HBA or the SAN fabric:
Check that the ports on the Fibre Channel switch connected to the HBA are active. Physically look at the switch to confirm the light next to the port is green, or remotely check by using the switchShow command.
Check that the HBA configuration is correct.
Check that the HBA can see all the LUNs for the filesystems it is mounting.
Check that the operating system kernel can see all the LUN devices.
If the RAID device has more than one LUN mapped to different controllers, ensure the node has a Fibre Channel path to all relevant controllers.
The cxfs_client daemon may not be running. See “The cxfs_client Daemon is Not Started on AIX ”.
The filesystem may have an unsupported mount option. Check the cxfs_client.log for mount option errors or any other errors that are reported when attempting to mount the filesystem.
The cluster membership (cms), XVM, or the filesystems may not be up on the node. Execute the /usr/cxfs_cluster/bin/cxfs_info command to determine the current state of cms, XVM, and the filesystems. If the node is not up for each of these, then check the /var/tmp/cxfs_client log to see what actions have failed.
Do the following:
If cms is not up, check the following:
Is the node is configured on the server-capable administration node with the correct hostname? See “Configuring Hostnames on Mac OS X” in Chapter 6.
Has the node been added to the cluster and enabled? See “Verifying the Cluster Status” in Chapter 10.
If XVM is not up, check that the HBA is active and can see the LUNs.
If the filesystem is not up, check that one or more filesystems are configured to be mounted on this node and check the /var/tmp/cxfs_client file for mount errors.
Confirm that the cxfs_client is not running. The following command would list the cxfs_client process if it were running:
aix# ps -ef | grep cxfs_client |
The cxfs_client daemon might not start for the following reasons:
The workstation is in 32-bit kernel mode, which is indicated if the following message is output to the console:
CXFS works only in the 64 bit kernel mode
In this case, you must change to 64-bit mode as follows:
Link the following libraries:
aix# ln -fs /usr/lib/boot/unix_64 /unix aix# ln -fs /usr/lib/boot/unix_64 /usr/lib/boot/unix
Create the boot image:
aix# bosboot -ad /dev/ipldevice
Reboot the system.
If the /var/tmp filesystem is full, CXFS cannot write logs to it and the CXFS filesystem will not be able to mount on the AIX node. In this case, you should clean out the /var/tmp filesystem.
If a disk is read from an AIX node and the following message is output, it means that the Fibre Channel switch has broken down:
no such device or address |
In this case, you should restart the Fibre Channel switch.
If the following message is output, then the genkex command does not exist:
genkex isn't found |
In this case, you must install the bos.perf.tools file set.
If an error occurs when a file is copied with the cp -p command and the following message is output, there is a problem with NFS:
There is not enough memory available now |
In this case, you must use maintenance level 5100-04+IY42428.
For more information, see:
If an ACL is not reflected when a file with an ACL is copied from JFS to CXFS using the cp -p command, there is a problem with the AIX software. (The ACL information for the file is indicated by the aclget command.) In this case, you must use maintenance level 5100-04.
For more information, see:
The /var/tmp/cxfs_client log file may become quite large over a period of time if the verbosity level is increased. To manually rotate this log file, use the -z option in the /usr/cxfs_cluster/bin/cxfs_client.options file.
See the cxfs_client man page and “Log Files on AIX”.
When reporting a problem about a CXFS AIX node to SGI, you should retain the following information:
Information about the AIX node system dump and system configuration:
aix# snap -a -o /dev/rmt0
aix# alog -o -t console
Current syslog file
The /var/tmp/cxfs_client CXFS log file
Moduler debugger output from the kdb command:
For panics or generated dumps, use the following commands and save the output:
aix# kdb /var/adm/ras/vmcore.xx[/unix] (0)> stat
For dumps from hangs:
aix# kdb /var/adm/ras/vmcore.xx[/unix] (0)> th* (to find the slot value of the working process or thread) (0)> sw slot_value (0)> stat
A list of the installed CXFS packages. Use the lslpp command as follows:
aix# lslpp -l SGIcxfs-aix5L
The version information of the operating system. Use the following oslevel commands:
aix# oslevel -r aix# oslevel -g | grep bos.64bit
A list of the loaded AIX kernel extensions. Use the genkex command.
If any of these AIX tools are not currently installed on your AIX node, you should install them.