The information in this appendix will help you migrate from a cluster with IRIX server-capable administration nodes to a cluster with SGI Foundation Software for Linux server-capable administration nodes. It discusses the following:
You must ensure that you have the correct network configuration for the new Linux server-capable administration nodes; see Chapter 6, “Preinstallation Steps”. For assistance, contact SGI Managed Services.
If you are migrating from a cluster with IRIX metadata servers to a cluster with SGI Foundation Software metadata servers, you should understand the differences between IRIX and Linux system administration. The details of these differences are beyond the scope of this guide, but a brief overview includes:
Installation tools
Mount options
Paths
Location of kernel system-tunable parameters
For more information, see the operating system documentation.
See also:
This section discusses the following:
CXFS does not support SGIRDAC mode. To convert from SGIRDAC to SGIAVT, do the following:
Install the latest supported firmware on the RAID.
Determine the IP address for one of the controllers on each RAID box.
Make a script settype.scr that contains the following line:
set storageArray defaultHostType="modename";
Note: The capitalization and punctuation in the above line are required. To switch to SGIAVT mode, use the following line:
set storageArray defaultHostType="SGIAVT";
For the InfiniteStorage 220, use the CLI client to set the host type to SGIAVT:
smicli -w SA_WWID -c 'set storageArray defaultHostType="SGIAVT";'
To determine the value for SA_WWID, invoke the following:
smicli -d -w
For example:
# smicli -d -w unit1 600a0b80002459d40000000045003fbc localhost |---> SA_WWID
Run the following for one of the controllers per RAID box:
/opt/tpssm/client/tpssmcli RAID_IPaddress -f settype.scr
For example:
# /opt/tpssm/client/tpssmcli 192.168.0.1 -f settype.scr Performing syntax check... Syntax check complete. Executing script... Script execution complete. tpssmcli completed successfully.
Linux does not support version 1 naming on filesystems. You must examine the parameters for each filesystem by using the following command on the active metadata server for each filesystem:
xfs_growfs -n mountpoint |
For example, the following shows that the /stripe1 filesystem has version-2 naming:
irixMDS# xfs_growfs -n /stripe1
meta-data=/stripe1 isize=256 agcount=16, agsize=1663360 blks
= sectsz=512 attr=0, parent=0
data = bsize=4096 blocks=26611968, imaxpct=25
= sunit=128 swidth=768 blks, unwritten=1
= mmr=0
naming =version 2 bsize=4096 mixed-case=Y
log =internal bsize=4096 blocks=13056 version=1
= sectsz=512 sunit=0 blks lazy-count=0
realtime =none extsz=4096 blocks=0, rtextents=0 |
If you have a filesystem that displays version 1 naming, you must recreate it on a Linux server-capable administration node so that it will have version 2 naming. Do the following:
Dump the filesystem data to backup media by using the xfsdump(1M) command.
Warning: If this step is not performed, all data currently on the filesystems will become inaccessible and will be permanently lost. Unmount the CXFS filesystems clusterwide by using the cxfs_admin command.
Recreate the filesystem by using the mkfs.xfs(8) command on a Linux server-capable administration node. Preserve any nondefault filesystem parameters by specifying the applicable options to the mkfs.xfs command.
Restore the filesystem data from backup media by using the xfsrestore(8) command.
For more information, see the man pages for the above commands and the following:
IRIX Admin: Disks and Filesystems
XFS for Linux Administration
The maximum block size on Linux nodes varies by architecture:
For SGI Altix ia64 server-capable administration nodes, the maximum block size is 16K.
For SGI Altix XE x86_64 server-capable administration nodes, the maximum block size is 4K.
If you have filesystems with larger block sizes than the above, you must recreate them by performing a dump and a restore, changing the block size so that it does not exceed the maximum for the architecture of your systems.
| Note: SGI recommends a block size of 4K for CXFS filesystems in a multiOS cluster because this is the only block size that is supported on all CXFS hardware platforms. |
Under Linux, project quotas are applied to the files below a given directory. You can view the project quotas only from the active CXFS metadata server, not from CXFS client-only nodes. See “Quotas Differences” in Chapter 1.
| Note: The following procedure assumes that the filesystems in the cluster you want to migrate do not have block sizes greater than the system page size and that they are not real-time filesystems or external log filesystems. These types of filesystems are supported on IRIX but not on SGI Foundation Software. |
Following is a procedure for using cxfs_admin to migrate from a cluster with IRIX server-capable administration nodes to a cluster with SGI Foundation Software for Linux server-capable administration nodes. For details about using cxfs_admin, see Chapter 11, “cxfs_admin Command”.
| Note: The cluster cannot be used for product work during this process. |
As a precaution, save the current cluster database contents by using the build_cmgr_script command. See “Creating a cmgr Script Automatically” in Appendix F.
Ensure you have CXFS 5.0 and XVM 5.0 licenses installed on the Linux server-capable administration nodes. See:
From an IRIX server-capable administration node:
Unmount the CXFS filesystems clusterwide by using the cxfs_admin command. For example, to unmount the CXFS V9500 filesystem clusterwide:
cxfs_admin:mycluster> unmount V9500
Use the IRIX mount and umount commands to mount and unmount the filesystems locally, which will ensure that the XFS log plays back cleanly. For example, mount and unmount the V9500 filesystem locally:
irixadmin# mount /dev/cxvm/V9500 /mnt irixadmin# umount /mnt
Disable each IRIX server-capable administration node by using the cxfs_admin command. For example, to disable the node named irixnode1:
cxfs_admin:mycluster> disable irixnode1
Repeat the disable command for each IRIX server-capable administration node.
Create a Linux server-capable administration node in the cluster. This will allow the cluster database information to be transferred to the Linux nodes.
Note: This migration situation is the only time that mixed OS server-capable administration nodes are allowed in the same cluster. For example:
cxfs_admin:mycluster> create node name=linuxnode1 type=server_admin os=Linux private_net=10.0.10.1 enabled=true hostname=linuxnode1.domain.com
From the Linux server-capable administration node created in step d above, do the following:
Delete each IRIX server-capable administration node by using the cxfs_admin command. For example, to delete the node named irixnode1:
cxfs_admin:mycluster> delete irixnode1
Repeat the delete command for each IRIX server-capable administration node.
Create any additional Linux server-capable administration nodes in the cluster. For example, to create a node named linuxnode2:
cxfs_admin:mycluster> create node name=linuxnode2 type=server_admin os=Linux private_net=10.0.10.2 enabled=true hostname=linuxnode2.domain.com
Repeat the create command for each Linux server-capable administration node.
Modify the CXFS filesystems to assign the new Linux server-capable administration nodes as potential metadata servers. For example:
cxfs_admin:mycluster> modify filesystem name=V9500 servers=linuxnode1,linuxnode2
Mount the CXFS filesystems by using the cxfs_admin command. For example, to mount the V9500 filesystem:
cxfs_admin:mycluster> mount V9500
Check cluster configuration:
cxfs_admin:mycluster> status
You could also use the cxfs-config command to verify the status.
Dispose of the former IRIX server-capable administration nodes properly:
Stop CXFS processes on each IRIX server-capable administration node:
irixadmin# service grio2 stop (if running GRIOv2) irixadmin# service cxfs stop irixadmin# service cluster stop
Repeat these commands on each IRIX server-capable administration node.
Do one of the following:
Transform the nodes into IRIX client-only nodes and add them to the cluster. Execute the stop commands above and follow the procedure in “Transforming a Server-Capable Administration Node into a Client-Only Node” in Chapter 12.
Remove the IRIX server-capable administration nodes physically from the network.
| Note: The following procedure assumes that the filesystems in the
cluster you want to migrate do not have block sizes greater than the system
page size and that they are not real-time filesystems or external log
filesystems. These types of filesystems are supported on IRIX but not
on SGI Foundation Software.
You should also be aware the differences between basic IRIX and Linux system administration. See “An Overview of Differences Between IRIX and Linux System Administration ”. |
The example in this chapter uses cmgr, but you could perform a similar procedure using cxfs_admin or the GUI; for a brief procedure using cxfs_admin, see “Migration Procedure Using cxfs_admin”. The example begins with a cluster named performance having a two IRIX server-capable administration nodes named rum and snake and a Solaris client-only node named ray:
rum # clconf_info Event at [2004-02-13 07:57:17] Membership since Thu Feb 12 15:15:26 2004 ____________ ______ ________ ______ ______ Node NodeID Status Age CellID ____________ ______ ________ ______ ______ snake 1 up 2 1 rum 2 up 2 2 ray 3 up 1 0 ____________ ______ ________ ______ ______ 1 CXFS FileSystems /dev/cxvm/V9500 on /cxfs/V9500 enabled server=(snake) 2 client(s)=(ray,rum) status=UP |
Do the following:
As a precaution, save the current cluster database contents by using the build_cmgr_script command. See “Creating a cmgr Script Automatically” in Appendix F.
Ensure you have CXFS 5.0 and XVM 5.0 licenses installed on the Linux server-capable administration nodes. See:
Unmount the CXFS filesystems cluster-wide within CXFS. For example:
cmgr> admin cxfs_unmount cxfs_filesystem V9500 cxfs_unmount operation successful
Mount and unmount the filesystems locally, which will ensure that the XFS log plays back cleanly. For example:
# mount /dev/cxvm/V9500 /mnt # umount /mnt
Stop CXFS services on all nodes. For example on the IRIX node rum:
cmgr> stop cx_services for cluster performance CXFS services have been deactivated in cluster performance
Note: If you use cxfs_admin, you must issue a command for each node. Define the server-capable administration node with the SGI Foundation Software operating system type. For example on the IRIX node rum:
Note: This migration situation is the only time that mixed OS server-capable administration nodes are allowed in the same cluster. cmgr> define node bang Enter commands, you may enter "done" or "cancel" at any time to exit Hostname[optional] ? bang Is this a FailSafe node <true|false> ? false Is this a CXFS node <true|false> ? true Operating System <IRIX|Linux32|Linux64|AIX|HPUX|Solaris|MacOSX|Windows> ? Linux64 Node Function <server_admin|client_admin|client_only> ? server_admin Node ID[optional] ? 64 Partition ID[optional] ? (0) Do you wish to define failure hierarchy[y/n]:y Hierarchy option 0 <System|FenceReset|Fence|Reset|Shutdown>[optional] ? Fence Hierarchy option 1 <System|FenceReset|Fence|Reset|Shutdown>[optional] ? Reset type <powerCycle|reset|nmi> ? (powerCycle) Do you wish to define system controller info[y/n]:n Number of Network Interfaces ? (1) NIC 1 - IP Address ? bang-p NIC 1 - Heartbeat HB (use network for heartbeats) <true|false> ? true NIC 1 - (use network for control messages) <true|false> ? true NIC 1 - Priority <1,2,...> ? 1 Successfully defined node bang
Add the SGI Foundation Software server-capable administration node to the cluster. For example on the IRIX node rum:
cmgr> modify cluster performance Enter commands, you may enter "done" or "cancel" at any time to exit Is this a FailSafe cluster <true|false> ? (false) Is this a CXFS cluster <true|false> ? (true) Cluster Notify Cmd [optional] ? Cluster Notify Address [optional] ? Cluster CXFS mode <normal|experimental>[optional] ? (normal) Cluster ID ? (1) Current nodes in cluster performance: Node - 1: ray Node - 2: snake Node - 3: rum No networks in cluster performance Add nodes to or remove nodes/networks from cluster performance Enter "done" when completed or "cancel" to abort performance ? add node bang performance ? done Added node <bang> to cluster <performance> Successfully modified cluster performance
Modify the CXFS filesystems to remove the IRIX server-capable administration nodes as metadata servers and add the new SGI Foundation Software server-capable administration node as metadata server. For example, on the IRIX node rum:
cmgr> modify cxfs_filesystem V9500 (Enter "cancel" at any time to abort) Device ? (/dev/cxvm/V9500) Mount Point ? (/cxfs/V9500) Mount Options[optional] ? Use Forced Unmount ? <true|false> ? (false) Grio Qualififed Bandwidth[optional] ? Grio managed filesystem ? <true|false>[optional] ? Default Local Status ? (enabled) MODIFY CXFS FILESYSTEM OPTIONS 0) Modify Server. 1) Add Server. 2) Remove Server. 3) Add Enabled Node. 4) Remove Enabled Node. 5) Add Disabled Node. 6) Remove Disabled Node. 7) Show Current Information. 8) Cancel. (Aborts command) 9) Done. (Exits and runs command) Enter option:2 Current servers: CXFS Server 1 - Rank: 0 Node: rum CXFS Server 2 - Rank: 1 Node: snake Server Node ? rum 0) Modify Server. 1) Add Server. 2) Remove Server. 3) Add Enabled Node. 4) Remove Enabled Node. 5) Add Disabled Node. 6) Remove Disabled Node. 7) Show Current Information. 8) Cancel. (Aborts command) 9) Done. (Exits and runs command) Enter option:2 Current servers: CXFS Server 1 - Rank: 1 Node: snake Server Node ? snake 0) Modify Server. 1) Add Server. 2) Remove Server. 3) Add Enabled Node. 4) Remove Enabled Node. 5) Add Disabled Node. 6) Remove Disabled Node. 7) Show Current Information. 8) Cancel. (Aborts command) 9) Done. (Exits and runs command) Enter option:1 No current servers Server Node ? bang Server Rank ? 1 0) Modify Server. 1) Add Server. 2) Remove Server. 3) Add Enabled Node. 4) Remove Enabled Node. 5) Add Disabled Node. 6) Remove Disabled Node. 7) Show Current Information. 8) Cancel. (Aborts command) 9) Done. (Exits and runs command) Enter option:9 Successfully modified cxfs_filesystem V9500
After you complete this step, the filesystems would show the following information:
cmgr> show cxfs_filesystem V9500 Name: V9500 Device: /dev/cxvm/V9500 Mount Point: /cxfs/V9500 Forced Unmount: false Global Status: disabled Default Local Status: enabled Server Name: bang Rank: 1
Remove the IRIX server-capable administration nodes from the cluster. For example, switching to the SGI Foundation Software node bang:
cmgr> modify cluster performance Enter commands, you may enter "done" or "cancel" at any time to exit Is this a FailSafe cluster <true|false> ? (false) Is this a CXFS cluster <true|false> ? (true) Cluster Notify Cmd [optional] ? Cluster Notify Address [optional] ? Cluster CXFS mode <normal|experimental>[optional] ? (normal) Cluster ID ? (1) Current nodes in cluster performance: Node - 1: ray Node - 2: snake Node - 3: rum Node - 4: bang Add nodes to or remove nodes/networks from cluster performance Enter "done" when completed or "cancel" to abort performance ? remove node rum performance ? remove node snake performance ? done Successfully modified cluster performance
Delete the IRIX server-capable administration nodes from the pool. For example, from the SGI Foundation Software node bang:
cmgr> delete node rum Deleted node (rum). cmgr> delete node snake Deleted node (snake).
Start CXFS services for all nodes in the cluster. For example, from the SGI Foundation Software node bang:
cmgr> start cx_services for cluster performance CXFS services have been activated in cluster performance
Mount the CXFS filesystems. For example, from the SGI Foundation Software node bang:
cmgr> admin cxfs_mount cxfs_filesystem V9500 cxfs_mount operation successful
If you are running other storage software products on server-capable administration nodes, confirm that you have installed either TPSSM or SMI, as appropriate for your hardware.
After completing this procedure, the cluster information is as follows:
[root@bang root]# clconf_info Event at [2004-02-13 08:44:18] Membership since Fri Feb 13 08:44:13 2004 ____________ ______ ________ ______ ______ Node NodeID Status Age CellID ____________ ______ ________ ______ ______ ray 3 up 1 0 bang 64 up 1 3 ____________ ______ ________ ______ ______ 1 CXFS FileSystems /dev/cxvm/V9500 on /cxfs/V9500 enabled server=(bang) 1 client(s)=(ray) status=UP |
For more information about using the GUI, see the following:
The following sections discuss possible problems you may encounter after migrating from an IRIX cluster to an SGI Foundation Software cluster:
Messages such as the following indicate that the filesystem was not cleanly unmounted from the IRIX metadata server:
Jan 29 22:06:07 4A:cxfs2 kernel: XFS: nil uuid in log - IRIX style log Jan 29 22:06:07 5A:cxfs2 kernel: Starting XFS recovery on filesystem: xvm-0 (dev: xvm-0) Jan 29 22:06:07 4A:cxfs2 kernel: XFS: dirty log written in incompatible format - can't recover |
To resolve this problem, you must return to the IRIX node and then mount and umount the filesystem locally on the IRIX node in order to replay the dirty log messages (as in step 4 above in “Migration Procedure Using cmgr”).
 | Caution: Do not steal the XVM volumes to the local host. Mounting /dev/cxvm/volname locally on /mnt is sufficient. |
If you have DMF filesystems and have dmi as a mount option,you must edit the/etc/sysconfig/sysctlfile to turn on DMAPI probing in order to mount CXFS filesystems. Change the bottom line from:
DMAPI_PROBE="no" |
to:
DMAPI_PROBE="yes" |