CXFS supports a client-only node running the Mac OS X operating system. This chapter contains the following sections:
This section contains the following information about CXFS on Mac OS X:
In addition to the items listed in “Requirements” in Chapter 1, using a Mac OS X node to support CXFS requires the following:
For the latest information, see the CXFS Mac OS X release notes.
The following commands are shipped as part of the CXFS Mac OS X package:
| /usr/cluster/bin/autopsy |
| /usr/cluster/bin/cxfs_client |
| /usr/cluster/bin/cxfs_info |
| /usr/cluster/bin/cxfsdump |
| /usr/cluster/bin/fabric_dump |
| /usr/cluster/bin/install-cxfs |
| /usr/cluster/bin/uninstall-cxfs |
| /Library/StartupItems/cxfs/cxfs |
| /usr/sbin/grioadmin |
| /usr/sbin/grioqos |
| /usr/cluster/bin/xvm |
If a Mac OS X node panics, the OS will write details of the panic to /Library/Logs/panic.log. Running autopsy parses this file and adds symbolic backtraces where possible to make it easier to determine the cause of the panic. The autopsy script is automatically run as part of the cxfsdump script, so the recommended steps for gathering data from a problematic node are still the same. Run autopsy with the -man option to display the man page.
To display details of all visible devices on the Fibre Channel fabric, run the fabric_dump script. The output is useful for diagnosing issues related to mount problems due to missing LUNs. Run fabric_dump with the -man option to display the man page.
The cxfs_client and xvm commands are needed to include a client-only node in a CXFS cluster. The cxfs_info command reports the current status of this node in the CXFS cluster.
The installation package uses install-cxfs to install or update all of the CXFS files. You can use the uninstall-cxfs command to uninstall all CXFS files; uninstall is not an installation package option.
The /Library/StartupItems/cxfs/cxfs command is run by the operating system to start and stop CXFS on the Mac OS X node.
For more information on these commands, see the man pages. For additional information about the GRIO commands, see “Guaranteed-Rate I/O (GRIO) and CXFS” in Chapter 1 and “GRIO on Mac OS X”.
The cxfs_client command creates a /var/log/cxfs_client log file. To rotate this log file, use the -z option in the /usr/cluster/bin/cxfs_client.options file; see the cxfs_client man page for details.
The CXFS installation process (install-cxfs and uninstall-cxfs) appends to /var/log/cxfs_inst.log.
For information about the log files created on server-capable administration nodes, see the CXFS Administration Guide for SGI InfiniteStorage.
Also see the Mac OS X /var/log/system.log file.
CXFS for Mac OS X has the following limitations and considerations for both Leopard and Tiger:
Mac OS X is unable to safely memory-map a file on a filesystem whose block size is greater than 4 KB. This is a due to a bug in the Darwin kernel that may be fixed by Apple in a future OS update.
XVM volume names are limited to 31 characters and subvolumes are limited to 26 characters. For more information about XVM, see XVM Volume Manager Administrator's Guide.
Mac OS X does not support the CXFS mount scripts.
Using RDAC mode will output numerous error messages. RDAC mode does not permit XVM path failover. SGI supports SGIAVT mode for XVM failover version 2. For more information, see the information about XVM failover in CXFS Administration Guide for SGI InfiniteStorage.
See also Appendix B, “Filesystem and Logical Unit Specifications”.
CXFS for Mac OS X has the following limitations and considerations for Leopard only:
To view drive icons and see CXFS filesystems in the Finder under Leopard, enable the following:
Finder -> Preferences -> General -> Connected Servers
A Mac OS X node may use a combination of methods for obtaining the node's hostname, depending on if it is in a NetInfo domain or is standalone.
Normally, you specify the hostname by using the following menu selection:
System Preferences -> Sharing -> Computer Name
Although the HOSTNAME=-AUTOMATIC- entry does not exist in the /etc/hostconfig file, you can specify a hostname by using the HOSTNAME parameter in this file. The hostname specified for the machine will have the following domain by default:
.local |
For example, if the hostname was specified as cxfsmac1 , then you would see the following when requesting the hostname:
macosx# /bin/hostname cxfsmac1.local |
The full hostname including .local is the hostname that the CXFS software will use to determine its identity in the cluster, not cxfsmac1.
Therefore, you must configure the node as cxfsmac1.local or specify the fully qualified hostname in /etc/hostconfig . For example:
HOSTNAME=cxfsmac1.sgi.com |
Specifying the hostname in this way may impact some applications, most notably Bonjour, and should be researched and tested carefully. There are also known issues with the hostname being reported as localhost on some reboots after making such a change.
SGI recommends that you specify other hosts in the cluster in the Mac OS X node's /etc/hosts file.
To ensure that the correct access controls are applied to users on Mac OS X nodes when accessing CXFS filesystems, you must ensure that the user IDs (UIDs) and group IDs (GIDs) are the same on the Mac OS X node as on all other nodes in the cluster, particularly any server-capable administration nodes.
 | Note: A user does not have to have user accounts on all nodes in the cluster. However, all access control checks are performed by server-capable administration nodes, so any server-capable administration nodes must be configured with the superset of all users in the cluster. |
Users can quickly check that their UID and GID settings are correct by using the id command on both the Mac OS X node and the server-capable administration node. For example:
macosx% id uid=1113(fred) gid=999(users) groups=999(users), 20(staff) admin% id uid=1113(fred) gid=999(users) groups=999(users), 20(staff) |
If the UID and/or GID do not match, or if the user is not a member of the same groups, then the user may unexpectedly fail to access some files.
To change the user's UID, GID, or other groups on Tiger requires changes to the NetInfo domain, whether local or distributed. Do the following:
Applications -> Utilities -> NetInfo Manager
Select the domain (if not the local domain):
Domain -> Open....
Select the user in question:
users -> username
Modify the uid, gid, or group fields as required.
Note: Changing a user's primary UID and/or GID will also require modifying all files owned by the user to the new UID and GID. Ideally, users should be created with the correct values.
Leopard's Accounts Preference Pane hides a set of advanced options that you can use to customize user account settings. Do the following:
Control-click a name in the Accounts Preference Pane
Choose Advanced from the pop-up menu.
Select the item you want to change.
All CXFS files have POSIX mode bits (read, write, and execute) and optionally an access control list (ACL). For more information, see the chmod and chacl man pages on a server-capable administration node.
CXFS on Mac OS X supports both enforcement of ACLs and the editing of ACLs from the Mac OS X node.
To display ACLs on a Mac OS X node, use the ls -l command. For example, the + character after the file permissions indicates that there are ACLs for newfile:
macosx# ls -l newfile -rw-r--r--+ 1 userA ptg 4 Jan 18 09:49 newfile |
To list the ACLs in detail, use the -le options (line breaks shown here for readability):
macosx# ls -le newfile
-rw--wxr--+ 1 userA ptg 4 Jan 18 09:49 newfile
0: user:userA allow read,write,delete,append,readattr,writeattr,readextattr,writeextattr,
readsecurity,writesecurity,chown
1: user:userA deny execute
2: group:everyone deny read,readattr,readextattr,readsecurity
3: group:ptg allow read,execute,readattr,readextattr,readsecurity
4: group:ptg deny write,delete,append,writeattr,writeextattr,writesecurity,chown
5: group:everyone allow read,readattr,readextattr,readsecurity
6: group:everyone deny write,execute,delete,append,writeattr,writeextattr,writesecurity,chown |
POSIX ACLs (used by SGI ProPack for Linux) are very different from those available on Mac OS X. Therefore a translation occurs, which places some limitations on what can be achieved with Mac OS X ACLs. As shown in Table 6-1, POSIX supports only three types of access permissions; in contrast, Mac OS X supports many variations. This means that some granularity is lost when converting between the two systems.
Table 6-1. Mac OS X Permissions Compared with POSIX Access Permissions
POSIX | Mac OS X |
|---|---|
Read | Read data, read attributes, read extended attributes, read security |
Write | Write data, append data, delete, delete child, write attributes, write extended attributes, write security, add file, add subdirectory, take ownership, linktarget, check immutable |
Execute | Execute |
POSIX ACLs and the file permissions have a particular relationship that must be translated to work with Mac OS X ACLs. For example, the minimum ACL for a file is user, group, and other, as follows:
admin# chacl -l newfile newfile [u::rw-,g::r-x,o::r--] |
The ACL (user, group, and other) exactly matches the file permissions. Further, any changes to the file permissions will be reflected in the ACL, and vice versa. For example:
admin# chmod 167 newfile admin# chacl -l newfile newfile [u::--x,g::rw-,o::rwx] |
This is slightly complicated by the mask ACL, which if it exists takes the file's group permissions instead. For example:
admin# chacl -l newfile newfile [u::rw-,g::r-x,o::r--,m::rwx] |
With POSIX, it is not possible to have fewer than three ACL entries, which ensures the rules always match with the file permissions. On Mac OS X, ACLs and file permissions are treated differently. ACLs are processed first; if there is no matching rule, the file permissions are used. Further, each entry can either be an allow entry or a deny entry. Given these differences, some restrictions are enforced to allow translation between these systems. For example, the simplest possible Linux ACL:
admin# chacl -l newfile newfile [u::rw-,g::r-x,o::r--] |
And the comparative Mac OS X ACL:
macosx# ls -le newfile
-rw-r-xr--+ 1 userA ptg 4 Jan 18 09:49 newfile
0: user:userA allow read,write,delete,append,readattr,writeattr,readextattr,
writeextattr,readsecurity,writesecurity,chown
1: user:userA deny execute
2: group:ptg allow read,execute,readattr,readextattr,readsecurity
3: group:ptg deny write,delete,append,writeattr,writeextattr,writesecurity,chown
4: group:everyone allow read,readattr,readextattr,readsecurity
5: group:everyone deny write,execute,delete,append,writeattr,writeextattr,writesecurity,chown |
Each POSIX rule is translated into two Mac OS X rules. For example, the following user rules are equivalent:
Linux:
u::rw-
Mac OS X:
0: user:userA allow read,write,delete,append,readattr,writeattr, readextattr,writeextattr,readsecurity,writesecurity,chown 1: user:userA deny execute
However, because the mask rule limits the access that can be assigned to anyone except the owner, the mask is represented by a single deny rule. For example, the following are equivalent:
Linux:
linux# chacl -l newfile newfile [u::rw-,g::r-x,o::r--,m::-wx]
Mac OS X:
macosx# ls -le newfile -rw--wxr--+ 1 userA ptg 4 Jan 18 09:49 newfile 0: user:userA allow read,write,delete,append,readattr,writeattr,readextattr, writeextattr,readsecurity,writesecurity,chown 1: user:userA deny execute 2: group:everyone deny read,readattr,readextattr,readsecurity 3: group:ptg allow read,execute,readattr,readextattr,readsecurity 4: group:ptg deny write,delete,append,writeattr,writeextattr,writesecurity,chown 5: group:everyone allow read,readattr,readextattr,readsecurity 6: group:everyone deny write,execute,delete,append,writeattr,writeextattr, writesecurity,chown
The mask rule (m::-wx) is inverted into a simple deny rule (group:everyone deny read,readattr,readextattr,readsecurity ). If a mask rule exists, it is always rule number 2 because it applies to everyone except for the file owner.
To add, remove, or edit an ACL on a file or directory, use the chmod command, which allows you to change only a single rule at a time. However, it is not valid in POSIX to have a single entry in an ACL. Therefore the basic rules are created based on the file permissions. For example (line breaks shown here for readability):
macosx# ls -le newfile
-rw-rw-rw- 1 userA ptg 0 Jan 18 15:40 newfile
macosx# chmod +a "cxfs allow read,execute" newfile
macosx# ls -le newfile
-rw-rw-rw-+ 1 userA ptg 0 Jan 18 15:40 newfile
0: user:userA allow read,write,delete,append,readattr,writeattr,readextattr,
writeextattr,readsecurity,writesecurity,chown
1: user:userA deny execute
2: group:everyone deny execute
3: user:cxfs allow read,execute,readattr,readextattr,readsecurity
4: user:cxfs deny write,delete,append,writeattr,writeextattr,writesecurity,chown
5: group:ptg allow read,write,delete,append,readattr,writeattr,readextattr,
writeextattr,readsecurity,writesecurity,chown
6: group:ptg deny execute
7: group:everyone allow read,write,delete,append,readattr,writeattr,readextattr,
writeextattr,readsecurity,writesecurity,chown
8: group:everyone deny execute |
You should only ever add, modify, or remove the allow rules. The corresponding deny rule will be created, modified, or removed as necessary. The mask rule is the only deny rule that you should specify directly.
For example, to remove a rule by using chmod:
macosx# chmod -a# 3 newfile
macosx# ls -le newfile
-rw-rw-rw-+ 1 userA ptg 0 Jan 18 15:40 newfile
0: user:userA allow read,write,delete,append,readattr,writeattr,readextattr,
writeextattr,readsecurity,writesecurity,chown
1: user:userA deny execute
2: group:everyone deny execute
3: group:ptg allow read,write,delete,append,readattr,writeattr,readextattr,
writeextattr,readsecurity,writesecurity,chown
4: group:ptg deny execute
5: group:everyone allow read,write,delete,append,readattr,writeattr,readextattr,
writeextattr,readsecurity,writesecurity,chown
6: group:everyone deny execute
7: group:everyone allow read,write,delete,append,readattr,writeattr,readextattr,
writeextattr,readsecurity,writesecurity,chown
8: group:everyone deny execute |
If you remove rules leaving only the user, group, and other rules, ACLs will be removed completely. For example:
macosx# chmod -a# 2 newfile macosx# ls -le newfile -rw-rw-rw- 1 userA ptg 0 Jan 18 15:40 newfile |
Adding rules to an existing ACL is complicated slightly because the ordering required by CXFS is different from the order used on Mac OS X. You may see the following error:
macosx# chmod +a "cxfs allow execute" newfile chmod: The specified file newfile does not have an ACL in canonical order, please specify a position with +a# : Invalid argument |
However, because an order will be enforced regardless of where the rule is placed, insert at any position and the rules will be sorted appropriately. For example:
macosx# chmod +a# 6 "sshd allow execute" newfile
macosx# ls -le newfile
-rw-rw-rw-+ 1 userA ptg 0 Jan 18 15:40 newfile
0: user:userA allow read,write,delete,append,readattr,writeattr,readextattr,writeextattr,
readsecurity,writesecurity,chown
1: user:userA deny execute
2: group:everyone deny execute
3: user:cxfs allow execute
4: user:cxfs deny read,write,delete,append,readattr,writeattr,readextattr,writeextattr,
readsecurity,writesecurity,chown
5: user:sshd allow execute
6: user:sshd deny read,write,delete,append,readattr,writeattr,readextattr,writeextattr,
readsecurity,writesecurity,chown
7: group:ptg allow read,write,delete,append,readattr,writeattr,readextattr,writeextattr,
readsecurity,writesecurity,chown
8: group:ptg deny execute
9: group:everyone allow read,write,delete,append,readattr,writeattr,readextattr,
writeextattr,readsecurity,writesecurity,chown
10: group:everyone deny execute |
You can also edit an existing rule by using chmod. Assuming the above file and permissions, you could allow the user to read files with the following command:
macosx# chmod =a# 3 "cxfs allow execute,read" newfile
macosx# ls -le newfile
-rw-rw-rw-+ 1 userA ptg 0 Jan 18 15:40 newfile
0: user:userA allow read,write,delete,append,readattr,writeattr,readextattr,writeextattr,
readsecurity,writesecurity,chown
1: user:userA deny execute
2: group:everyone deny execute
3: user:cxfs allow read,execute,readattr,readextattr,readsecurity
4: user:cxfs deny write,delete,append,writeattr,writeextattr,writesecurity,chown
5: user:sshd allow execute
6: user:sshd deny read,write,delete,append,readattr,writeattr,readextattr,writeextattr,
readsecurity,writesecurity,chown
7: group:ptg allow read,write,delete,append,readattr,writeattr,readextattr,writeextattr,
readsecurity,writesecurity,chown
8: group:ptg deny execute
9: group:everyone allow read,write,delete,append,readattr,writeattr,readextattr,
writeextattr,readsecurity,writesecurity,chown
10: group:everyone deny execute |
Adding a second rule for the same user or group is not permitted with POSIX ACLs. If you attempt to do this, the permissions will be merged. It is important to get the rule number correct when editing a rule.
It is possible to define default ACLs to a directory, so that all new files or directories created below are assigned a set of ACLs automatically. The semantics are handled differently between Linux and Mac OS X, so the functionality is limited to mimic what is available in POSIX. In POSIX, the default ACL is applied at creation time only; if the default rule subsequently changes, it is not applied to a directory's children. The equivalent behavior on Mac OS X is achieved by the only_inherit and limit_inherit flags.
For example, a default ACL might look like this on Linux:
admin# chacl -l test test [u::rwx,g::r--,o::---/u::rw-,g::rw-,o::r--,u:501:r--,m::rwx] |
On Mac OS X, a default ACL might look like the following:
macosx# ls -lde test
drwxr-----+ 2 userA ptg 78 Jan 18 15:39 test
0: user:userA allow list,add_file,search,delete,add_subdirectory,delete_child,
readattr,writeattr,readextattr,writeextattr,readsecurity,writesecurity,chown
1: user:userA deny
2: group:ptg allow list,readattr,readextattr,readsecurity
3: group:ptg deny add_file,search,delete,add_subdirectory,delete_child,writeattr,
writeextattr,writesecurity,chown
4: group:everyone allow
5: group:everyone deny list,add_file,search,delete,add_subdirectory,delete_child,
readattr,writeattr,readextattr,writeextattr,readsecurity,writesecurity,chown
6: user:userA allow list,add_file,delete,add_subdirectory,delete_child,readattr,
writeattr,readextattr,writeextattr,readsecurity,writesecurity,chown,file_inherit,
directory_inherit,only_inherit
7: user:userA deny search,file_inherit,directory_inherit,only_inherit
8: group:everyone deny file_inherit,directory_inherit,only_inherit
9: user:cxfs allow list,readattr,readextattr,readsecurity,file_inherit,
directory_inherit,only_inherit
10: user:cxfs deny add_file,search,delete,add_subdirectory,delete_child,writeattr,
writeextattr,writesecurity,chown,file_inherit,directory_inherit,only_inherit
11: group:ptg allow list,add_file,delete,add_subdirectory,delete_child,readattr,
writeattr,readextattr,writeextattr,readsecurity,writesecurity,chown,file_inherit,
directory_inherit,only_inherit
12: group:ptg deny search,file_inherit,directory_inherit,only_inherit
13: group:everyone allow list,readattr,readextattr,readsecurity,file_inherit,
directory_inherit,only_inherit
14: group:everyone deny add_file,search,delete,add_subdirectory,delete_child,writeattr,
writeextattr,writesecurity,chown,file_inherit,directory_inherit,only_inherit |
The default rules are flagged with the inheritance flags ( file_inherit,directory_inherit,only_inherit). Editing these rules is similar to editing an access rule, except the inherit flag is included. For example:
macosx# mkdir newdir
macosx# chmod +a "cxfs allow read,only_inherit" newdir
macosx# ls -led newdir
drwxr-xr-x+ 2 userA ptg 6 Jan 20 11:20 newdir
0: user:userA allow list,add_file,search,delete,add_subdirectory,delete_child,
readattr,writeattr,readextattr,writeextattr,readsecurity,writesecurity,chown
1: user:userA deny
2: group:ptg allow list,search,readattr,readextattr,readsecurity
3: group:ptg deny add_file,delete,add_subdirectory,delete_child,writeattr,
writeextattr,writesecurity,chown
4: group:everyone allow list,search,readattr,readextattr,readsecurity
5: group:everyone deny add_file,delete,add_subdirectory,delete_child,writeattr,
writeextattr,writesecurity,chown
6: user:userA allow list,add_file,search,delete,add_subdirectory,delete_child,
readattr,writeattr,readextattr,writeextattr,readsecurity,writesecurity,chown,
file_inherit,directory_inherit,only_inherit
7: user:userA deny file_inherit,directory_inherit,only_inherit
8: group:everyone deny add_file,delete,add_subdirectory,delete_child,writeattr,
writeextattr,writesecurity,chown,file_inherit,directory_inherit,only_inherit
9: user:cxfs allow list,readattr,readextattr,readsecurity,file_inherit,
directory_inherit,only_inherit
10: user:cxfs deny add_file,search,delete,add_subdirectory,delete_child,writeattr,
writeextattr,writesecurity,chown,file_inherit,directory_inherit,only_inherit
11: group:ptg allow list,search,readattr,readextattr,readsecurity,file_inherit,
directory_inherit,only_inherit
12: group:ptg deny add_file,delete,add_subdirectory,delete_child,writeattr,
writeextattr,writesecurity,chown,file_inherit,directory_inherit,only_inherit
13: group:everyone allow list,search,readattr,readextattr,readsecurity,
file_inherit,directory_inherit,only_inherit
14: group:everyone deny add_file,delete,add_subdirectory,delete_child,writeattr,
writeextattr,writesecurity,chown,file_inherit,directory_inherit,only_inherit |
The base ACL is created if its not specified and removing the default ACL is a matter of removing rules until only the base rules are present, at which point the ACL will be removed.
CXFS for Mac OS X supports Apple Computer, Inc. host bus adapters (HBAs).
| Note: The procedures in this section may be performed by you or by a qualified service representative. You must be logged in as root to perform the steps listed in this section. |
This section discusses the following:
Do the following:
Install the Apple HBA into a spare PCI, PCI-X, or PCI Express slot in the Mac OS X node, according to the manufacturer's instructions. Do not connect the HBA to the Fibre Channel switch at this time.
Note: Apple HBAs are normally shipped with copper SFPs and copper cables, so additional optic SFPs and optic cables may be required.
Reboot the node.
Do the following:
Install the configuration utility from the CD distributed with the Apple HBA. To do this, copy Mac OS X Utilities/Fibre Channel Utility from the CD to your Application directory.
Run the Fibre Channel Utility after it is copied to the node. The tool will list the HBA on the left-hand side of the window. Select the Apple FC card item to display the status of the ports via a pull-down menu. Initially, each port will report that it is up (even though it is not connected to the switch), and the speed and port topology will configure automatically.
Connect one of the HBA ports to the switch via a Fibre Channel cable. After a few seconds, close and relaunch the Fibre Channel Utility. Select the Apple FC card item and then the connected port from the drop-down list to display the speed of the link.
Repeat these steps for the second HBA port if required.
(Optional) If necessary, use Apple's /sbin/fibreconfig tool to modify port speed and topology. See the man page for details.
The CXFS fabric_dump tool can also be of use in verifying Fibre Channel fabric configuration. See “CXFS Commands on Mac OS X”.
The Mac OS X node does its own path management for paths that go to the same RAID controller and thus only presents one /dev device to userspace per RAID controller. Even if multiple paths exist to a RAID controller, you will only see one /dev device.
Therefore, the Fibre Channel Utility does not support masking logical units (LUNs) on specific ports. However, if the first port can see all of the LUNs, the default is that all I/O will go through a single port. To avoid this, configure the switch so that each port can see a different set of LUNs. You can achieve this by zoning the switch or by using multiple switches, with different controllers and HBA ports to each switch.
This section provides an overview of the steps that you or a qualified Apple service representative will perform on your Mac OS X nodes prior to installing the CXFS software. It contains the following sections:
The following procedure provides an overview of the steps required to add a private network to the Mac OS X system. A private network is required for use with CXFS. See “Use a Private Network” in Chapter 2.
You may skip some steps, depending upon the starting conditions at your site. For details about any of these steps, see the Mac OS X system documentation.
Install Mac OS X and configure the machine's hostname (see “Configuring Hostnames on Mac OS X”) and IP address on its public network interface.
Decide if the Mac OS X node will be part of a NetInfo domain or a standalone machine. If part of a NetInfo domain, configure the node into the domain before proceeding further.
Add the IP addresses and hostnames of other machines in the cluster to the NetInfo database and/or the /etc/hosts file. You should be consistent about specifying the hostname or the fully qualified domain name for each host. A common convention is to name the CXFS private network address for each host as hostname-priv.
Install a second network interface card if necessary as per the manufacturer's instructions.
Configure the second network interface by using the following menu selection:
System Preferences -> Network -> Show
Select the second network interface (most likely PCI Ethernet Slot 1), and specify the IP address, subnet mask, and router. The private network interface should not require a DNS server because the private network address of other cluster nodes should be explicitly listed in the NetInfo database and/or in the /etc/hosts file. Relying on a DNS server for private network addresses introduces another point of failure into the cluster and must be avoided.
Confirm the configuration using ifconfig to list the network interfaces that are up:
macosx# ifconfig -u
In general, this should include en0 (the onboard Ethernet) and en1 (the additional PCI interface), but the names of these interfaces may vary.
For more information, see the ifconfig man page.
Verify each interface by using the ping command to connect to the public and private network addresses of the other nodes that are in the CXFS pool.
For example:
macosx# grep cxfsmac2 /etc/hosts 134.14.55.115 cxfsmac2 macosx# ping -c 3 134.14.55.115 PING 134.14.55.115 (134.14.55.115): 56 data bytes 64 bytes from 134.14.55.115: icmp_seq=0 ttl=64 time=0.247 ms 64 bytes from 134.14.55.115: icmp_seq=1 ttl=64 time=0.205 ms 64 bytes from 134.14.55.115: icmp_seq=2 ttl=64 time=0.197 ms --- 134.14.55.115 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.197/0.216/0.247 ms |
CXFS does not support the energy-saving mode on Mac OS X. If this mode is enabled, the Mac OS X node will lose CXFS membership and unmount the CXFS filesystem whenever it is activated.
Select the following to disable the energy-saving mode:
System Preferences -> Energy Saver -> Put the computer to sleep when it is inactive for -> Never
The CXFS software will be initially installed and configured by SGI personnel. This section provides an overview of those procedures. You can use the information in this section to verify the installation.
| Note: CXFS software can also be installed from the command line. For more information about command line installation using the installer command, see the Mac OS X documentation. |
Installing the CXFS client software for Mac OS X requires approximately 30 MB of space.
To install the required software on a Mac OS X node, SGI personnel will do the following:
Read the SGI InfiniteStorage Software Platform release notes, and CXFS general release notes in the /docs directory on the ISSP DVD and late-breaking caveats on Supportfolio.
Verify that the node is running the supported Mac OS X operating system according to the Mac OS X installation guide. Use the following command to display the currently installed system:
macosx# uname -r
This command should return a value of 8.8.0 or later for Tiger or 9.0.0 or later for Leopard.
As root or a user with administrative privileges, transfer the client software that was downloaded onto a server-capable node during its installation procedure using ftp, rcp, or scp. The location of the disk image on the server will be as follows:
/usr/cluster/client-dist/CXFS_VERSION/macosx/MAC_VERSION/noarch/cxfs.dmg
Note: You must transfer the disk image to the root or your own home directory in order to make it visible with the Finder tool. Double-click the transferred cxfs.dmg file to mount the disk image
Double click cxfs.pkg to begin the installation.
Click continue when you see the following message:
message : This package contains a program that determines if the software can be installed. Are you sure you want to continue
Click continue when you see the following message:
The installer will guide you through the steps necessary to install CXFS for Mac OS X. To get started, click Continue
This will launch the installation application, which will do the following:
Display the CXFS Mac OS X release note. Read the release note and click continue.
Display the license agreement. Read the agreement and click agree if you accept the terms.
Perform a standard installation of the software on the root drive volume.
Caution: Do not choose Change install location.
Continue Installation at the following message:
Installation of this software requires you to restart your computer when the installation is done. Are you sure you want to install the software now?
After the install succeeds, click the highlighted Restart button to reboot your machine.
I/O fencing is required on Mac OS X nodes in order to protect data integrity of the filesystems in the cluster. The cxfs_client software automatically detects the world wide port names (WWPNs) of any supported host bus adapters (HBAs) for Mac OS X nodes that are connected to a switch that is configured in the cluster database. These HBAs are available for fencing.
However, if no WWPNs are detected, the following messages will be logged to the /var/log/cxfs_client file:
hba_wwpn_list warning: No WWPN found from IO Registry
cis_get_hbas warning: Not able to find WWN (err=Device not
configured). Falling back to "/etc/fencing.conf".
cis_config_swports_set error fetching hbas |
If no WWPNs are detected, you can manually specify the WWPNs in the fencing file.
| Note: This method does not work if the WWPNs are partially discovered. |
The /etc/fencing.conf file enumerates the WWPNs for all of the HBAs that will be used to mount a CXFS filesystem. There must be a line for the HBA WWPN as a 64-bit hexadecimal number.
| Note: The WWPN is that of the HBA itself, not any of the devices that are visible to that HBA in the fabric. |
If used, /etc/fencing.conf must contain a simple list of WWPNs, one per line. You must update it whenever the HBA configuration changes, including the replacement of an HBA.
Do the following:
Set up the switch and HBA. See the release notes for supported hardware.
Follow the Fibre Channel cable on the back of the node to determine the port to which it is connected in the switch. Ports are numbered beginning with 0. (For example, if there are 8 ports, they will be numbered 0 through 7.)
Use the telnet command to connect to the switch and log in as user admin. (On Brocade switches, the password is password by default).
Execute the switchshow command to display the switches and their WWPN numbers.
For example:
brocade04:admin> switchshow switchName: brocade04 switchType: 2.4 switchState: Online switchRole: Principal switchDomain: 6 switchId: fffc06 switchWwn: 10:00:00:60:69:12:11:9e switchBeacon: OFF port 0: sw Online F-Port 20:00:00:01:73:00:2c:0b port 1: cu Online F-Port 21:00:00:e0:8b:02:36:49 port 2: cu Online F-Port 21:00:00:e0:8b:02:12:49 port 3: sw Online F-Port 20:00:00:01:73:00:2d:3e port 4: cu Online F-Port 21:00:00:e0:8b:02:18:96 port 5: cu Online F-Port 21:00:00:e0:8b:00:90:8e port 6: sw Online F-Port 20:00:00:01:73:00:3b:5f port 7: sw Online F-Port 20:00:00:01:73:00:33:76 port 8: sw Online F-Port 21:00:00:e0:8b:01:d2:57 port 9: sw Online F-Port 21:00:00:e0:8b:01:0c:57 port 10: sw Online F-Port 20:08:00:a0:b8:0c:13:c9 port 11: sw Online F-Port 20:0a:00:a0:b8:0c:04:5a port 12: sw Online F-Port 20:0c:00:a0:b8:0c:24:76 port 13: sw Online L-Port 1 public port 14: sw No_Light port 15: cu Online F-Port 21:00:00:e0:8b:00:42:d8
The WWPN is the hexadecimal string to the right of the port number. For example, the WWPN for port 0 is 2000000173002c0b (you must remove the colons from the WWPN reported in the switchshow output to produce the string to be used in the fencing file).
Edit or create /etc/fencing.conf and add the WWPN for the port determined in step 2. (Comment lines begin with #.)
For dual-ported HBAs, you must include the WWPNs of any ports that are used to access cluster disks. This may result in multiple WWPNs per HBA in the file; the numbers will probably differ by a single digit.
For example, if you determined that port 0 is the port connected to the switch, your fencing file should contain the following:
# WWPN of the HBA installed on this system # 2000000173002c0b
To configure fencing, see the CXFS Administration Guide for SGI InfiniteStorage.
The /Library/StartupItems/cxfs/cxfs script will be invoked automatically during normal system startup and shutdown procedures. This script starts and stops the cxfs_client daemon.
To start cxfs_client manually, enter the following:
macosx# sudo /Library/StartupItems/cxfs/cxfs start |
To stop cxfs_client manually, enter the following:
macosx# sudo /Library/StartupItems/cxfs/cxfs stop |
To stop and start cxfs_client manually, enter the following:
macosx# sudo /Library/StartupItems/cxfs/cxfs restart |
To prevent the automatic startup of cxfs_client on boot, move the /Library/StartupItems/cxfs directory out of /Library/StartupItems.
This section contains the following:
Before updating CXFS software, ensure that no applications on the node are accessing files on a CXFS filesystem. You can then run the new CXFS software package, which will automatically upgrade all CXFS software.
You can modify the behavior of the CXFS client daemon (cxfs_client) by placing options in the /usr/cluster/bin/cxfs_client.options file. The available options are documented in the cxfs_client man page.
| Caution: Some of the options are intended to be used internally by SGI only for testing purposes and do not represent supported configurations. Consult your SGI service representative before making any changes. |
To see if cxfs_client is using the options in cxfs_client.options , enter the following:
ps -axwww | grep cxfs |
For example:
macosx# ps -axwww | grep cxfs 611 ?? 0:06.17 /usr/cluster/bin/cxfs_client -D trace -z |
CXFS supports guaranteed-rate I/O (GRIO) version 2 on the Mac OS X platform. Application bandwidth reservations must be explicitly released by the application before exit. If the application terminates unexpectedly or is killed, its bandwidth reservations are not automatically released and will cause a bandwidth leak. If this happens, the lost bandwidth could be recovered by rebooting the client node.
For more information, see “Guaranteed-Rate I/O (GRIO) and CXFS” in Chapter 1 and the Guaranteed-Rate I/O Version 2 Guide.
Following is an example of the /etc/failover2.conf file on Mac OS X:
/dev/rxvm-200400a0b80cd5fe-000 affinity=1 preferred /dev/rxvm-200500a0b80cd5fe-000 affinity=2 /dev/rxvm-200400a0b80cd5fe-001 affinity=2 /dev/rxvm-200500a0b80cd5fe-001 affinity=1 preferred |
The device is the node's WWN plus the LUN number.
| Note: Even if multiple paths exist to a RAID controller, you will only see one /dev device. The Mac OS X node does its own path management for paths that go to the same RAID controller and thus only presents one /dev device to userspace per RAID controller. See “Configuring Two or More Apple HBA Ports”. |
For more information, see “XVM Failover and CXFS” in Chapter 1, the comments in the /etc/failover2.conf file, CXFS Administration Guide for SGI InfiniteStorage, and the XVM Volume Manager Administrator's Guide.
This section discusses the following:
For general troubleshooting information, see Chapter 11, “General Troubleshooting” and Appendix D, “Error Messages”
Confirm that the cxfs_client is not running. The following command would list the cxfs_client process if it were running:
macosx# ps -auxww | grep cxfs_client |
Check the cxfs_client log file for errors.
Restart cxfs_client as described in “Start/Stop cxfs_client for Mac OS X” and watch the cxfs_client log file for errors.
On Mac OS X nodes, the following error message in the system.log file indicates that the volume name is too long and must be shortened so that the Mac OS X node can recognize it:
devfs: volumename name slot allocation failed (Errno=63) |
The /var/log/cxfs_client log file may become quite large over a period of time if the verbosity level is increased.
To manually rotate this log file, use the -z option in the /usr/cluster/bin/cxfs_client.options file.
See the cxfs_client.options man page and “Log Files on Mac OS X”.