When you install the CXFS software, there are some system file considerations you must take into account. The network configuration is critical. Each node in the cluster must be able to communicate with every other node in the cluster by both logical name and IP address without going through any other network routing; proper name resolution is key. SGI recommends static routing.
This section discusses the following:
After completing these step discussed in this chapter, see Chapter 9, “Initial Setup of the Cluster”. For details about specific configuration tasks, see Chapter 10, “CXFS GUI”. For information about upgrades, see “CXFS Release Versions and Rolling Upgrades” in Chapter 12.
The optional /etc/exports file on each node describes the filesystems that are being exported to NFS clients.
If the /etc/exports file contains a CXFS mount point, then when the system is booted NFS will export the empty mount point because the exports are done before CXFS is running. When CXFS on the node joins membership and starts mounting filesystems, the clconfd-pre-mount script searches the /etc/exports file looking for the mountpoint that is being mounted. If found, the script unexports the mountpoint directory because if it did not the CXFS mount would fail. After successfully mounting the filesystem, the clconfd-post-mount script will search the /etc/exports file and export the mount point if it is found in the /etc/exports file.
For more information, see “CXFS Mount Scripts” in Chapter 12.
This section discusses system files on administration nodes:
The /etc/services file on each CXFS administration contains entries for sgi-cad and sgi-crsd . The port numbers assigned for these processes must be the same in all nodes in the pool.
The following shows an example of /etc/services entries for sgi-cad and sgi-crsd:
sgi-crsd 7500/udp # Cluster reset services daemon sgi-cad 9000/tcp # Cluster Admin daemon |
The cad.options file on each server-capable administration node contains the list of parameters that the cluster administration daemon reads when the cad process is started. The file is located as follows:
/etc/cluster/config/cad.options |
cad provides cluster information.
The following options can be set in the cad.options file:
| --append_log | Append cad logging information to the cad log file instead of overwriting it. | |
| --log_file filename | cad log filename. Alternately, this can be specified as -lf filename. | |
| -vvvv | Verbosity level. The number of v characters indicates the level of logging. Setting -v logs the fewest messages; setting -vvvv logs the highest number of messages. |
The default file has the following options:
-lf /var/cluster/ha/log/cad_log --append_log |
The following example shows an /etc/config/cad.options file that uses a medium-level of verbosity:
-vv -lf /var/cluster/ha/log/cad_nodename --append_log |
The default log file is /var/cluster/ha/log/cad_log. Error and warning messages are appended to the log file if log file is already present.
The contents of the /etc/config/cad.options file cannot be modified using cxfs_admin or the GUI.
If you make a change to the cad.options file at any time other than initial configuration, you must restart the cad processes in order for these changes to take effect. You can do this by rebooting the nodes or by entering the following command:
server-admin# service cxfs_cluster restart |
If you execute this command on a running cluster, it will remain up and running. However, the GUI will lose connection with the cad daemon; the GUI will prompt you to reconnect.
The fs2d.options file on each server-capable administration node contains the list of parameters that the fs2d daemon reads when the process is started. (The fs2d daemon manages the distribution of the cluster database (CDB) across the server-capable administration nodes in the pool.) The file is located as follows:
/etc/cluster/config/fs2d.options |
Table 8-1 shows the options can that can be set in the fs2d.options file.
Table 8-1. fs2d.options File Options
If you use the default values for these options, the system will be configured so that all log messages of level info or less, and all trace messages for transaction events, are sent to the /var/cluster/ha/log/fs2d_log file. When the file size reaches 10 MB, this file will be moved to its namesake with the .old extension and logging will roll over to a new file of the same name. A single message will not be split across files.
If you make a change to the fs2d.options file at any time other than the initial configuration time, you must restart the fs2d processes in order for those changes to take effect. You can do this by rebooting the server-capable administration nodes or by entering the following command:
server-admin# service cxfs_cluster restart |
If you execute this command on a running cluster, it should remain up and running. However, the GUI will lose connection with the cad daemon; the GUI will prompt you to reconnect.
The following example shows an /etc/config/fs2d.options file that directs logging and tracing information as follows:
All log events are sent to: /var/log/messages
Tracing information for RPCs, updates, and transactions are sent to /var/cluster/ha/log/fs2d_ops1.
When the size of this file exceeds 100,000,000 bytes, this file is renamed to /var/cluster/ha/log/fs2d_ops1.old and a new file /var/cluster/ha/log/fs2d_ops1 is created. A single message is not split across files.
(Line breaks added for readability.)
-logevents all -loglevel trace -logdest syslog -trace rpcs -trace updates -trace transactions -tracefile /var/cluster/ha/log/fs2d_ops1 -tracefilemax 100000000 |
The following example shows an /etc/config/fs2d.options file that directs all log and trace messages into one file, /var/cluster/ha/log/fs2d_chaos6, for which a maximum size of 100,000,000 bytes is specified. -tracelog directs the tracing to the log file.
(Line breaks added for readability.)
-logevents all -loglevel trace -trace rpcs -trace updates -trace transactions -tracelog -logfile /var/cluster/ha/log/fs2d_chaos6 -logfilemax 100000000 -logdest logfile. |
You can use the clconfd.options file on each server-capable administration node to contain a list of nondefault parameters that the clconfd daemon will read when the process is started. To use this feature, create the following file:
/etc/cluster/config/clconfd.options |
Table 8-2 shows the options that can be set in the fs2d.options file.
Table 8-2. clconfd.options File Options
Option | Description |
|---|---|
-c CDBfile | Reads the cluster database configuration from the specified CDBfile file. The default file is /var/cluster/cdb/cdb.db. |
-d debugfile | Enables printing hafence debug information to the specified file debugfile . The default is to print no information. |
-h | Prints a help message for clconfd.options. |
-l | Runs clconfd in the foreground. (For SGI development debugging purposes only. Do not use this option unless directed to do so by SGI support.) The default is to run clconfd in the background. |
-s loglevel | Specifies the log level to use for logging to standard error. The default is 0 (no logging). For information about log levels, see “Configure Log Groups with the GUI” in Chapter 10. |
-R | Disables real-time scheduling. By default, real-time scheduling is enabled. |
For example, to print hafence debug information to the file /tmp/hafence.log, add the following line to the clconfd.options file:
-d /tmp/hafence.log |
If you make a change to the clconfd.options file at any time other than the initial configuration time, you must restart the clconfd processes in order for those changes to take effect. You can do this by rebooting the server-capable administration nodes or by entering the following command:
server-admin# service cxfs restart |
This section discusses enabling and disabling GRIO the following for server-capable administration nodes:
To enable GRIO on a server-capable administration node, do the following:
Turn the grio2 flag on with the chkconfig(8) command:
server# chkconfig grio2 on |
Reboot the system.