Procedure 3-6. To install RHEL 6 on an SGI ICE X SAC

1. Connect to the system admin controller (SAC) by one of the following methods:

   • Through the intelligent platform management interface (IMPI) tool

   • Through the console attached to the SGI ICE X system

   • Through a separate keyboard, video display terminal, and mouse

2. Use a text editor, such as vi or vim, to open file /etc/sysconfig/network-scripts/ifcfg-eth0 .

3. Add lines for the IPADDR, NETMASK , and NETWORK values appropriate for your public (house) network to file /etc/sysconfig/network-scripts/ifcfg-eth0 .

   For example:

   IPADDR=128.162.244.88 NETMASK=255.255.255.0 NETWORK=128.162.244.0

   
   

4. Save and close file /etc/sysconfig/network-scripts/ifcfg-eth0 .

5. Use a text editor to create file /etc/sysconfig/network .

6. Add the following three lines to file /etc/sysconfig/network :

   NETWORKING=yes HOSTNAME=SAC_hostname GATEWAY=gateway_IP_address

   
   

   For SAC_hostname, type the hostname you want to assign to the SAC.

   For gateway_IP_address, type the IP address of the gateway for your house network.

   For example:

   NETWORKING=yes HOSTNAME=my-system-admin GATEWAY=128.162.244.1

   
   

7. Save and close file /etc/sysconfig/network.

8. Use a text editor to open file /etc/hosts.

9. Add a line in the following format to file /etc/hosts :

   SAC_IP SAC_FQDN SAC_hostname

   
   

   The variables in the preceding line are as follows:

   • For SAC_IP, type the IP address of the SAC.

   • For SAC_FQDN, type the fully qualified domain name (FQDN) of the SAC.

   • For SAC_hostname, type the hostname of the SAC.

   For example, add the following line:

   128.162.244.88 my-system-admin.domain-name.mycompany.com my-system-admin

   
   

10. Save and close file /etc/hosts.

11. Type the following command to set the SAC hostname:

    # hostname SAC_hostname

    
    

    For SAC_hostname, type the hostname of the SAC.

    For example:

    # hostname my-system-admin

    
    

12. Use a text editor to create file /etc/resolv.conf .

13. Add lines to file /etc/resolv.conf that specify the search domain and the domain name service (DNS) servers at your site.

    Later in the configuration process, when you run the cluster configuration tool, the tool uses the DNS servers you specify in this step for its defaults.

    Specify lines with the following format:

    search search_domain nameserver name_server_IP nameserver name_server_IP

    
    

    The following is an example resolv.conf file:

    search mydomain.com nameserver 192.168.0.1 nameserver 192.168.0.25

    
    

14. Type the following nscd(8) command to force the invalidation of the name service cache daemon:

    # nscd -i hosts

    
    

15. Type the following commands, in the order shown, to restart services:

    # /etc/init.d/network restart # /etc/init.d/rpcbind start # /etc/init.d/nfslock start

    
    

16. Type the following command to retrieve the SAC's current time zone information:

    # strings /etc/localtime | tail -1 CST6CDT,M3.2.0,M11.1.0

    
    

    The previous output shows the SAC set to US Central time. If the output you see is not correct for this SGI ICE X system, perform the following steps:

    • Type the following command to change to the directory that contains the time zone configuration files:

      # cd /usr/share/zoneinfo

      
      

    • Select a file from that directory that describes the time zone for the SAC.

    • Type the following commands to enable the new time zone configuration file.

      For example:

      # /bin/cp -l /usr/share/zoneinfo/time_zone_file /etc/localtime.$$ # /bin/mv /etc/localtime.$$ /etc/localtime

      
      

      For time_zone_file, type the name of the time zone file that you need from the /usr/share/zoneinfo directory.

      For example, type the following commands to change the SAC's time zone to US Pacific time:

      # /bin/cp -l /usr/share/zoneinfo/PST8PDT /etc/localtime.$$ # /bin/mv /etc/localtime.$$ /etc/localtime

      
      

    • Type the following command to confirm the time zone:

      # strings /etc/localtime | tail -1 PST8PDT,M3.2.0,M11.1.0

      
      

17. (Conditional) Edit file /etc/ntp.conf to direct requests to the network time protocol (NTP) server at your site.

    Complete the following steps if you want to direct requests to your site's NTP server instead of to the public time servers of the pool.ntp.org project:

    • Use a text editor to open file /etc/ntp.conf .

    • Insert a pound character (#) into column 1 of each of each line that includes rhel.pool.ntp.org.

      ---

      Note: Do not edit or remove entries that serve the cluster networks.

      ---

      
      

    • At the end of the file, add a line that points to your site's NTP server.

      The following is an example of a correctly edited file:

      # Use public servers from the pool.ntp.org project. # Please consider joining the pool (http://www.pool.ntp.org # server 0.rhel.pool.ntp.org # server 1.rhel.pool.ntp.org # server 2.rhel.pool.ntp.org server ntp.mycompany.com

      The preceding output has been truncated at the right for inclusion in this guide.

    • Type the following command to restart the NTP server:

      # /etc/init.d/ntpd restart

      
      

18. (Conditional) Type a tilde character (~) and then a period character (.) to exit from the IPMI tool.

    Perform this step if you connected to the system through the IPMI tool.

19. (Optional) Configure the system so that you can perform the installation from a VGA screen and can perform later operations from a serial console.

    If you want to enable this capability, perform the following steps:

    • Use a text editor to open file /boot/grub/menu.lst .

    • Search the file for the word kernel at the beginning of a line.

    • Add the following to the kernel line: console=type.

      For example:

      kernel /boot/vmlinuz-2.6.16.56-0.12-smp root=/dev/disk/by-label/sgiroot console=ttyS1,38400n8 splash=silent showopts

      
      

    • Add the console=type parameter to the end of every kernel line. By default, this is set to ttyS1,38400n8. You might have ttys2, for example.

    Later, if you want to access the SAC from only a VGA, you can remove the console= parameters.

20. Proceed to the following:

    “Running the Cluster Configuration Tool”

Procedure 3-7. To install SLES 11 on an SGI ICE X SAC

1. Connect to the system admin controller (SAC) by one of the following methods:

   • Through the intelligent platform management interface (IMPI) tool

   • Through the console attached to the SGI ICE X system

   • Through a separate keyboard, video display terminal, and mouse

2. On the Language and Keyboard Layout screen, complete the following steps:

   • Select your language

   • Select your keyboard layout

   • Select Next.

3. On the Welcome screen, select Next.

4. On the Hostname and Domain Name screen, complete the following steps:

   • Type the hostname for this SGI ICE X system.

   • Type the domain name.

   • Clear the box next to Change Hostname via DHCP. The box appears with an X in it by default, but you need to clear this box.

   • Select Assign Hostname to Loopback IP. Put an X in this box.

   • Select Next.

5. On the Network Configuration screen, complete the following steps:

   • Select Change. A pop-up window appears.

   • On the pop-up window, choose Network Interfaces.

6. On the Network Settings screen, complete the following steps:

   • Highlight the first network interface card that appears underneath Name.

   • Select Edit.

7. On the Network Card Setup screen, specify the SAC's house/public network interface.

   Figure 3-3 shows the Network Card Setup screen.

   Figure 3-3. Network Card Setup Screen
   
   

   Complete the following steps:

   • Select Statically Assigned IP Address. SGI recommends a static IP address, not DHCP, for the SAC.

   • In the IP Address field, type the system's IP address.

   • In the Subnet Mask field, type the system's subnet mask.

   • In the Hostname field, type the system's fully qualified domain name (FQDN). SGI requires you to type an FQDN, not the system's shorter hostname, into this field. For example, type mysystem-admin.mydomainname.com. Failure to supply an FQDN in this field causes the configure-cluster command to fail.

   • Select Next.

   You can specify the default route, if needed, in a later step.

8. On the Network Settings screen, complete the following steps:

   • Select Hostname/DNS.

   • In the Hostname field, type the system's fully qualified domain name (FQDN).

   • In the Domain Name field, type the domain name for your site.

   • Put an X in the box next to Assign Hostname to Loopback IP.

   • In the Name Servers and Domain Search List , type the name servers for your house network.

   • Back at the top of the screen, select Routing .

     The Network Settings > Routing screen appears.

   • In the Default Gateway field, type your site's default gateway.

   • Select OK.

9. On the Network Configuration screen, click Next.

   The Saving Network Configuration screen appears and saves your configuration.

10. On the Clock and Time Zone screen, complete the following steps:

    • Select your region.

    • Select your time zone.

    • (Optional) In the Hardware Clock Set To field, choose Local Time or accept the default of UTC.

    • Select Next.

    This step synchronizes the time in the BIOS hardware with the time in the operating system. Your choice depends on how the BIOS hardware clock is set. If the clock is set to GMT, which corresponds to UTC, your system can rely on the operating system to switch from standard time to daylight savings time and back automatically.

11. On the Password for System Administrator “root” screen, complete the following steps:

    • In the Password for root User field, type the password you want to use for the root user.

      This password becomes the root user's password for all the nodes on the ICE X system. These nodes are as follows:

      • SAC

      • Rack leader controller (RLC)

      • Service nodes

      • Compute nodes (blades)

    • In the Confirm password field, type the root user's password again.

    • In the Test Keyboard Layout field, type a few characters.

      For example, if you specified a language other than English, type a few characters that are unique to that language. If these characters appear in this plain text field, you can use these characters in passwords safely.

    • Select Next.

12. On the User Authentication Method screen, select one of the authentication methods and select Next.

    Typically, users accept the default (Local).

13. On the New Local User screen, create additional user accounts or select Next.

    If you do not create additional users, select Yes on the Empty User Login warning pop-up window, and select Next.

14. On the Installation Completed screen, select Finish.

15. Type a tilde character (~) and then a period character (.) to exit from the IPMI tool.

16. Log into the SAC, open file /etc/hosts within a text editor, and verify that the SAC's fully qualified domain name (FQDN) and hostname are entered correctly.

    For example, the following /etc/hosts file entry contains the correct data in the three required fields and is correct for a SAC with an IP address of 100.100.100.100, an FQDN of mysystem-admin.mydomain.com, and a hostname of mysystem-admin:

    100.100.100.100 mysystem-admin.mydomain.com mysystem-admin

    
    

    Make sure that the /etc/hosts file on the SAC contains the required information. If it does not, edit the /etc/hosts file to contain the three required fields as the preceding example shows.

17. Confirm that the system is working as expected.

    If necessary, restart YaST2 to correct settings.

18. (Optional) Configure the system so that you can perform the installation from a VGA screen and can perform later operations from a serial console.

    If you want to enable this capability, perform the following steps:

    • Use a text editor to open file /boot/grub/menu.lst .

    • Search the file for the word kernel at the beginning of a line.

    • Add the following to the kernel line: console=type.

      For example:

      kernel /boot/vmlinuz-2.6.16.56-0.12-smp root=/dev/disk/by-label/sgiroot console=ttyS1,38400n8 splash=silent showopts

      
      

    • Add the console=type parameter to the end of every kernel line. By default, this is set to ttyS1,38400n8. You might have ttys2, for example.

    Later, if you want to access the SAC from only a VGA, you can remove the console= parameters.

19. Proceed to the following:

    “Running the Cluster Configuration Tool”

Procedure 3-10. To verify your environment

1. Verify that your SGI ICE X system supports the following:

   • The Intel Intelligent Power Node Manager (IPNM)-capable compute nodes. Consult your SGI technical support representative if you are unsure.

   • The SMC DCM plugin

   If you are unsure about the preceding requirements, contact your SGI representative or review your hardware specifications. You can configure DCM at any time, but it is easier to configure at this point in the installation process because you have not yet installed the SGI Management Center (SMC) license key.

2. Verify that your SGI Management Center (SMC) license includes both of the following keys:

   • A premium license

   • A power opt license

   If you are installing a completely new SGI ICE X system, there is no SMC license installed on the system at this time, and you need information about what your license includes, contact your SGI representative to verify your licenced features.

   If you are enabling the DCM feature on a production SGI ICE X system that already has an SMC license installed upon it, use the lk_verify command to verify your system's license key. The following example shows these keys:

   # lk_verify 1 /etc/lk/keys.dat:007 product=SGIMC, version=1.000, count=0, begDate=1374482126, \ expDate=1375765199, licenseID=0, key=Iym9+MynzDO2R0ABCcqJjgrXYqgu6rwp, \ info='SMC 1.7 PREM SVR/CL',attr='ST=CL NC=0 LC=24FF', \ vendor='Silicon Graphics International',ref_id='279071' Verdict: SUCCESS. 2 /etc/lk/keys.dat:018 product=SGIMC, version=1.000, count=0, begDate=1373396727, \ expDate=1381208399, licenseID=b9e99f78, key=xybn6FAs4rx2UoC7WzclQElAQ7DVbc+6, \ info='SGIMC POWER OPT',attr='LC=0200', vendor='Silicon Graphics International', \ ref_id='278444' Verdict: SUCCESS.

   
   

   In the preceding lk_verify output, license 1 is the premium license, and license 2 shows the power option in the info and attr fields.

3. (Conditional) Back up the current DCM configuration files on the system admin controller (SAC) and on all rack leader controller (RLC) nodes.

   Perform this step if you already have the DCM SDK installed on this SGI ICE X system and you are reinstalling the DCM SDK. This can be the case if you are troubleshooting an incomplete installation.

   The steps are as follows:

   • Log into the SAC or into one of the RLCs as the root user.

   • Type the following command to stop the DCM daemon:

     # service mcdcm stop

     
     

   • Type the following command to create a scratch directory and write the existing DCM configuration files to that directory:

     # mkdir scratch; cp -a /opt/sgi/mcdcm/conf /scratch/mcdcm-conf

     
     

   • Type the following command to remove the DCM package:

     # rpm -e mcdcm

     
     

   • Type the following command to remove any existing initialization scripts:

     # rm -rf /etc/init.d/dcm* /etc/init.d/mcdcm*

     
     

   • Type the following command to remove the mcdcm daemon:

     # rm -rf /opt/sgi/mcdcm

     
     

   • Repeat the preceding steps on the other computers in the SGI ICE X cluster.

4. Type the following commands to install the SMC DCM package:

   # cinstallman --yum-node --node admin install mcdcm # cinstallman --yum-node --node r1lead install mcdcm

   
   

   If you have multiple RLCs, repeat the preceding command on other RLCs as needed.

5. On the SAC, type the following command to verify the installation:

   # cat /opt/sgi/mcdcm/logs/dcmcmdstatus.log

   
   

   The command returns information such as the following if the installation was successful:

   Success. You can now start the database server using: [...] waiting for server to start.... done server started

   
   

6. On the SAC, type the following commands to change localhost to admin in the system files:

   # sed -i 's/localhost/admin/g' \ /opt/sgi/mcdcm/conf/web.config.xml # sed -i 's/localhost/admin/g' \ /opt/sgi/mcdcm/external/apache-tomcat/webapps/DCMWsdl/internal.wsdl # sed -i 's/localhost/admin/g' \ /opt/sgi/mcdcm/external/apache-tomcat/webapps/DCMWsdl/dcm.wsdl

   
   

   Note that the preceding commands use the backslash character ( \) because the command continues to a second line.

7. On each RLC, type the following commands to change localhost to admin in the system files:

   # pdsh -g leader 'sed -i "s/localhost/${HOSTNAME}/g" \ /opt/sgi/mcdcm/conf/web.config.xml' # pdsh -g leader 'sed -i "s/localhost/${HOSTNAME}/g" \ /opt/sgi/mcdcm/external/apache-tomcat/webapps/DCMWsdl/internal.wsdl' # pdsh -g leader 'sed -i "s/localhost/${HOSTNAME}/g" \ /opt/sgi/mcdcm/external/apache-tomcat/webapps/DCMWsdl/dcm.wsdl'

   
   

   Note that the preceding commands use the backslash character ( \) because the command continues to a second line.

8. Type the following commands to start the SMC DCM daemons on the SAC and on the RLCs:

   # service mcdcm start # pdsh -g leader service mcdcm start

   
   

9. Type the following commands on the SAC and on the RLCs, and observe the output:

   # ps -ef | grep jsvc | grep -i "startup.Bootstrap" # ps -ef | grep jsvc | grep -i "DaemonImpl"

   
   

   In the output, make sure that the com.intel.dcm.daemon.DaemonImpl and org.apache.catalina.startup.Bootstrap daemons started.

10. On the SAC, and on each RLC, type the following command to make sure that the DCM daemon is started:

    # grep "process started" /opt/sgi/mcdcm/logs/DatacenterManager.log Intel(R) Data Center Manager 2.x.xxxxx process started!

    
    

11. On the SAC, type the following command to install the sgimc-dcm daemon:

    # cinstallman --yum-node --node admin install sgimc-dcm

    
    

12. On the SAC, use a text editor to open file /opt/sgi/sgimc/etc/dcm.properties , search for the string dcm.bmc, and modify the existing lines to be as follows:

    dcm.bmc.user = ADMIN dcm.bmc.password = ADMIN

    
    

    The preceding lines are case sensitive.

13. On the SAC, use a text editor to open file /opt/sgi/sgimc/etc/tempo.properties , search for the string dcm.enabled, and modify the existing line (or add it) to be as follows:

    dcm.enabled = true

    
    

14. Proceed to one of the following:

    • If you need to install SMC license keys, for example on a new system, proceed to the following:

      “(Conditional) Installing the SGI Management Center License Key ”

    • If the correct SMC license keys are already installed on your system, proceed to the following:

      “Completing the Intel Data Center Power Manager (DCM) Software Development Kit (SDK) Configuration”

Procedure 3-11. To license the SMC software

1. Use a text editor to open file /etc/lk/keys.dat.

2. Copy and paste the license key exactly as it was given to you.

   There can be line breaks in the license key.

3. Save the file.

4. Type the following command to restart the SMC daemon:

   # service mgr restart

   
   

5. Type the following command to start SMC:

   # mgrclient

   
   

6. Proceed to “Completing the Intel Data Center Power Manager (DCM) Software Development Kit (SDK) Configuration”.

Procedure 3-12. To complete the Intel DCM enablement procedure

1. Log into the system admin controller (SAC) and type the following command to restart the mgr daemon:

   # service mgr restart

   
   

2. On the SAC, use a text editor to open file /opt/sgi/sgimc/etc/perfhost_options.profile , and search for tempo.dcm.

   Verify that lines similar to the following appear in the perfhost_options.profile file:

   tempo.dcmBmc.r1i0n1=192.168.160.101 tempo.dcmBmc.r1i0n0=192.168.160.100

   
   

3. On the SAC, verify the contents of file /opt/sgi/sgimc/etc/site-site.profile .

   At this time, the site-site.profile file should contain a line for each node that declares the rack leader controller (RLC) for each node. These lines are similar to the following:

   PlatformManagementService.hostsettings.r1i0n0:BEGIN1,CPOWER,false,localhost,4,END1,BEGIN2,CONSERVER,false, localhost,admin,END2,BEGIN3,DCM,false,r1lead,400,0,,,4,IPNM2,END3,1,1,0,2,0 PlatformManagementService.hostsettings.r1i0n1:BEGIN1,CPOWER,false,localhost,4,END1,BEGIN2,CONSERVER,false, localhost,admin,END2,BEGIN3,DCM,false,r1lead,400,0,,,4,IPNM2,END3,1,1,0,2,0

   
   

   If the site-site.profile file does not contain a line for each node, you need to add these lines manually. This can be the case if you installed the SMC license and are now using this procedure to enable DCM. You can use a loop, such as the following, to add these lines:

   for node in `cnodes --compute --rack 1`; do echo PlatformManagementService.hostsettings.$node:BEGIN1,CPOWER,false,localhost,4,END1,BEGIN2,CONSERVER,false, localhost,admin,END2,BEGIN3,DCM,false,r1lead,400,0,,,4,IPNM2,END3,1,1,0,2,0 >> /opt/sgi/sgimc/etc/site-site.profile done

   
   

4. (Optional) Add entries to the /opt/sgi/sgimc/etc/site-site.profile file for the RLCs and service nodes.

   Perform this step if you want to enable DCM on nondefault nodes such as the RLCs and service nodes.

   Edit the following configuration files to define DCM for the RLCs and service nodes:

   • Open file site-site.profile and add IP address information, such as the following examples show, to the file:

     tempo.dcmBmc.r1lead=172.24.0.2 tempo.dcmBmc.service0=192.168.160.109

     
     

   • Open file /opt/sgi/sgimc/etc/site-site.profile and add information such as the following:

     PlatformManagementService.hostsettings.r1lead:BEGIN1,CPOWER,false,localhost,4,END1,BEGIN2,CONSERVER,false, localhost,admin,END2,BEGIN3,DCM,false,admin,400,0,,,4,IPNM2,END3,1,1,0,2,0 PlatformManagementService.hostsettings.service0:BEGIN1,CPOWER,false,localhost,4,END1,BEGIN2,CONSERVER,false, localhost,admin,END2,BEGIN3,DCM,false,admin,400,0,,,4,IPNM2,END3,1,1,0,2,0

     
     

5. Type the mgrclient command to start the SGI Management Center GUI.

6. Click Edit > Preferences > Platform Management .

   In the window that appears, select or set the following:

   • To the right of Management Device 3:, click Configure.

     Select DCM as the third platform device.

   • Verify the defaults and change defaults if needed. The defaults are as follows:

     DCM Primary Host Name: localhost Derated Power: 400 Nameplate Power 450 Unmanaged Power 100 Cooling Power Multipler: 0.5 Power Cost Per KWh: 1 Device Connector Type: IPNM2 Platform Management User: ADMIN

     
     

     For information about the defaults, see the SGI Management Center System Administrator's Guide.

   • Select the policy button.

   • Click OK.

   • Click OK.

7. On the SAC, type the following command to restart the mgr daemon:

   # service mgr restart

   
   

8. Wait for the changes to be applied to your SGI ICE X system.

   On a large cluster, these changes can take 30 minutes or more to complete.

9. Type the mgrclient command to restart the SGI Management Center GUI.

10. In the SGI Management Center GUI, click Admin > Instrumentation > Power.

    Visually inspect the information on the Power tab to verify that it presents data for the cluster and for the individual nodes.

11. Proceed to the following:

    “Synchronizing the Software Repository, Installing Software Updates, and Cloning the Images”

Procedure 3-14. To download the MPSS package

1. Open a browser, and navigate to the following website:

   http://software.intel.com/mic-developer

2. Click the Tools & Downloads tab.

3. Click the Software Drivers: Intel Manycore Platform Software Stack (Intel MPSS) link.

   Follow the instructions on the website to download the Linux software version for your operating system platform. The download comes in the form of a tar(1) file.

4. Use the instructions from Intel to build the RPM files that you need.

   A later procedure explains how to transfer these files to the SGI ICE X system and build new images.

5. Proceed to one of the following topics:

   • “Creating Images for the Compute Nodes That Include MIC Devices ”

   • “Creating Images for the Service Nodes That Include MIC Devices ”

   Perform both of the preceding procedures if you have MIC devices on both compute nodes and service nodes.

Procedure 3-15. To create compute node images for compute nodes with MIC devices

1. On the SAC, use the mkdir(1) command, in the following format, to create a directory for the RPM repository:

   mkdir -p /tftpboot/intel/mpss_repository_directory

   
   

   For mpss_repository_directory, type a name for the directory that is to contain the MPSS repository. For convenience, make sure to include an identifier for the MPSS release level you downloaded.

   For example:

   # mkdir -p /tftpboot/intel/mpss_u3-2.1.6720-19

   
   

2. Use operating system commands to copy the RPM files you downloaded to the /tftpboot/intel/mpss_repository_directory directory on the SAC node.

   For example, use cp(1), ftp(1), rsync(1), scp(1), or another method.

3. Use the crepo command, in the following format, to specify a custom repository for the MPSS RPMs:

   crepo --add rpm_repo_directory -custom rpm_repo_name

   
   

   The variables in this command are as follows:

   • For rpm_repo_directory, specify the full path to the directory that contains the RPM files.

   • For rpm_repo_name, create a name for the image. You can specify the same name for both rpm_repo_directory and rpm_repo_name. After the image is built, the cinstallman --show-images command returns this label in the Image Name column of its output.

   For example:

   # crepo --add /tftpboot/intel/mpss_u3-2.1.6720-19 --custom mpss_u3-2.1.6720-19

   
   

4. Type the following command to confirm that the MPSS image is in the correct repository:

   # crepo --show

   
   

5. Use the crepo command, in the following format, to add the custom repository to the generated RPM list in /etc/opt/sgi/rpmlists:

   crepo --select rpm_repo_name

   
   

   For rpm_repo_name, create a name for the image. You can specify the same name for both rpm_repo_directory and rpm_repo_name. After the image is built, the cinstallman --show-images command returns this label in the Image Name column of its output. This is the same rpm_repo_name that you specified in the following step:

   Step 3

   For example, the following command adds the custom repository and displays the content of the respository:

   # crepo --select mpss_u3-2.1.6720-19 Selecting: mpss_u3-2.1.6720-19 Updating: /etc/opt/sgi/rpmlists/generated-compute-rhel6.5.rpmlist Updating: /etc/opt/sgi/rpmlists/generated-lead-rhel6.5.rpmlist Updating: /etc/opt/sgi/rpmlists/generated-service-rhel6.5.rpmlist

   
   

6. Type the following command to confirm that you selected the new repository that contains the MPSS RPMs:

   # crepo --show * mpss_u3-2.1.6720-19 : /tftpboot/intel/mpss_u3-2.1.6720-19 * Red-Hat-Enterprise-Linux-6.5 : /tftpboot/distro/rhel6.5 * SGI-MPI-1.7-rhel6 : /tftpboot/sgi/SGI-MPI-1.7-rhel6 * SGI-Management-Center-1.9-rhel6 : /tftpboot/sgi/SGI-Management-Center-1.9-rhel6 * SGI-Foundation-Software-2.9-rhel6 : /tftpboot/sgi/SGI-Foundation-Software-2.9-rhel6 * SGI-Tempo-2.9-rhel6 : /tftpboot/sgi/SGI-Tempo-2.9-rhel6

   
   

   The asterisk character (*) in column 1 indicates an image that is selected.

7. Type the following command to display the images that are available for installation on the compute nodes:

   # cinstallman --show-images Image Name BT Path lead-rhel6.5 0 /var/lib/systemimager/images/lead-rhel6.5 service-rhel6.5 0 /var/lib/systemimager/images/service-rhel6.5 compute-rhel6.5 1 /var/lib/systemimager/images/compute-rhel6.5

   
   

8. Use the cinstallman command, in the following format, to clone the current operating system image:

   cinstallman --create-image --clone --source current_image --image new_image

   
   

   The variables in this command are as follows:

   • For current_image, type the name of the operating system image you want to use that is on the system right now. Choose one that appears in the output from the from the cinstallman --show-images command in the Image Name list. For example, compute-rhel6.5.

   • For new_image, type a new name for the operating system image that is to include the MPSS file RPMs. SGI recommends that you include information in the new name that can enable you to identify this image as the operating system image that includes MPSS software. For example, compute-rhel6.5-mic-6720-19 identifies the new image as a RHEL image that contains a revision of the MPSS MIC software.

   For example:

   # cinstallman --create-image --clone --source compute-rhel6.5 --image compute-rhel6.5-mic-6720-19

   
   

9. Type the following command to display the images and confirm that the new image appears in the list:

   # cinstallman --show-images Image Name BT Path lead-rhel6.5 0 /var/lib/systemimager/images/lead-rhel6.5 service-rhel6.5 0 /var/lib/systemimager/images/service-rhel6.5 compute-rhel6.5 1 /var/lib/systemimager/images/compute-rhel6.5 compute-rhel6.5-mic-6720-19 1 /var/lib/systemimager/images/compute-rhel6.5-mic-6720-19

   
   

10. Use the cinstallman command, in the following format, to install the sgi-mic-compute package and the MPSS RPMs:

    cinstallman --yum-image --image image_name install sgi-mic-compute

    
    

    For image_name, specify the new_image that you created in the following step:

    Step 8

    For example:

    # cinstallman --yum-image --image compute-rhel6.5-mic-6720-19 install sgi-mic-compute

    
    

11. (Conditional) Enable the SLES operating system to load the MPSS package.

    Perform this step if you are installing the MPSS packages on a SLES platform.

    Complete the following steps:

    • Open file /etc/modprobe.d/unsupported-modules with a text editor.

    • Add the following line at the end of the file:

      allow_unsupported_modules 1

      
      

    • Save and close the file.

12. Type the following command to display all the images:

    # cimage --list-images image: compute-rhel6.5 kernel: 2.6.32-358.el6.x86_64 image: compute-rhel6.5-mic-6720-19 kernel: 2.6.32-358.el6.x86_64

    The preceding output shows the newly installed image, compute-rhel6.5-mic-6720-19.

13. Use the cimage command, in the following format, to set the default image for the compute nodes:

    cimage --set-default --file_system compute-rhel6.5-mic-6720-19 kernel

    
    

    The variables in this command are as follows:

    • For file_system, type either nfs or tmpfs, according to your site practice.

    • For kernel, type the kernel associated with the new image, as shown in the output from the cimage --list-images command in the following step:

      Step 12

    For example:

    # cimage --set-default --compute-rhel6.5-mic-6720-19 2.6.32-358.el6.x86_64

    
    

14. Type the following command to confirm that the new image is the default image:

    # cimage --show-default compute-rhel6.5-stout7-mic-6720-15 2.6.32-358.el6.x86_64 tmpfs

    
    

15. Proceed to one of the following:

    • If your SGI ICE X system contains MIC devices on the service nodes, proceed to “Creating Images for the Service Nodes That Include MIC Devices ”.

    • If your SGI ICE X system does not contain MIC devices on the service nodes, proceed to “Configuring the Switches”.

Procedure 3-16. To create service node images for service nodes with MIC devices

1. On the SAC, use the mkdir(1) command, in the following format, to create a directory for the RPM repository:

   mkdir -p /tftpboot/intel/mpss_repository_directory

   
   

   For mpss_repository_directory, type a name for the directory that is to contain the MPSS repository. For convenience, make sure to include an identifier for the MPSS release level you downloaded.

   For example:

   # mkdir -p /tftpboot/intel/mpss_u3-2.1.6720-19

   
   

2. Use operating system commands to copy the RPM files you downloaded to the /tftpboot/intel/mpss_repository_directory directory on the SAC node.

   For example, use cp(1), ftp(1), rsync(1), scp(1), or another method.

3. Use the crepo command, in the following format, to specify a custom repository for the MPSS RPMs:

   crepo --add rpm_repo_directory -custom rpm_repo_name

   
   

   The variables in this command are as follows:

   • For rpm_repo_directory, specify the full path to the directory that contains the RPM files.

   • For rpm_repo_name, create a name for the image. You can specify the same name for both rpm_repo_directory and rpm_repo_name. After the image is built, the cinstallman --show-images command returns this label in the Image Name column of its output.

   For example:

   # crepo --add /tftpboot/intel/mpss_u3-2.1.6720-19 --custom mpss_u3-2.1.6720-19

   
   

4. Type the following command to confirm that the MPSS image is in the correct repository:

   # crepo --show

   
   

5. Use the crepo command, in the following format, to add the custom repository to the generated RPM list in /etc/opt/sgi/rpmlists:

   crepo --select rpm_repo_name

   
   

   For rpm_repo_name, create a name for the image. You can specify the same name for both rpm_repo_directory and rpm_repo_name. After the image is built, the cinstallman --show-images command returns this label in the Image Name column of its output. This is the same rpm_repo_name that you specified in the following step:

   Step 3

   For example, the following command adds the custom repository and displays the content of the respository:

   # crepo --select mpss_u3-2.1.6720-19 Selecting: mpss_u3-2.1.6720-19 Updating: /etc/opt/sgi/rpmlists/generated-compute-rhel6.5.rpmlist Updating: /etc/opt/sgi/rpmlists/generated-lead-rhel6.5.rpmlist Updating: /etc/opt/sgi/rpmlists/generated-service-rhel6.5.rpmlist

   
   

6. Type the following command to confirm that you selected the new repository that contains the MPSS RPMs:

   # crepo --show * mpss_u3-2.1.6720-19 : /tftpboot/intel/mpss_u3-2.1.6720-19 * Red-Hat-Enterprise-Linux-6.5 : /tftpboot/distro/rhel6.5 * SGI-MPI-1.7-rhel6 : /tftpboot/sgi/SGI-MPI-1.7-rhel6 * SGI-Management-Center-1.9-rhel6 : /tftpboot/sgi/SGI-Management-Center-1.9-rhel6 * SGI-Foundation-Software-2.9-rhel6 : /tftpboot/sgi/SGI-Foundation-Software-2.9-rhel6 * SGI-Tempo-2.9-rhel6 : /tftpboot/sgi/SGI-Tempo-2.9-rhel6

   
   

   The asterisk character (*) in column 1 indicates an image that is selected.

7. Type the following command to display the images that are available for installation on the service nodes:

   # cinstallman --show-images Image Name BT Path lead-rhel6.5 0 /var/lib/systemimager/images/lead-rhel6.5 service-rhel6.5 0 /var/lib/systemimager/images/service-rhel6.5 compute-rhel6.5 1 /var/lib/systemimager/images/compute-rhel6.5

   
   

8. Use the cinstallman command, in the following format, to clone the current operating system image:

   cinstallman --create-image --clone --source current_image --image new_image

   
   

   The variables in this command are as follows:

   • For current_image, type the name of the operating system image you want to use. Choose one that appears in the output from the cinstallman --show-images command in the Image Name list. For example, service-rhel6.5.

   • For new_image, type a new name for the operating system image that is to include the MPSS file RPMs. SGI recommends that you include information in the new name that can enable you to identify this image as the operating system image that includes MPSS software. For example, service-rhel6.5-mic-6720-19 identifies the new image as a RHEL image that contains a revision of the MPSS MIC software.

   For example:

   # cinstallman --create-image --clone --source service-rhel6.5 --image service-rhel6.5-mic-6720-19

   
   

9. Type the following command to display the images and confirm that the new image appears in the list:

   # cinstallman --show-images Image Name BT Path lead-rhel6.5 0 /var/lib/systemimager/images/lead-rhel6.5 service-rhel6.5 0 /var/lib/systemimager/images/service-rhel6.5 compute-rhel6.5 1 /var/lib/systemimager/images/compute-rhel6.5 service-rhel6.5-mic-6720-19 1 /var/lib/systemimager/images/service-rhel6.5-mic-6720-19

   
   

10. Use the cinstallman command, in the following format, to install the sgi-mic-service package and the MPSS RPMs:

    cinstallman --yum-image --image image_name install sgi-mic-service

    
    

    For image_name, specify the new_image that you created in the following step:

    Step 8

    For example:

    # cinstallman --yum-image --image service-rhel6.5-mic-6720-19 install sgi-mic-service

    
    

11. (Conditional) Enable the SLES operating system to load the MPSS package.

    Perform this step if you are installing the MPSS packages on a SLES platform.

    Complete the following steps:

    • Open file /etc/modprobe.d/unsupported-modules with a text editor.

    • Add the following line at the end of the file:

      allow_unsupported_modules 1

      
      

    • Save and close the file.

12. Type the following command to display all the images:

    # cimage --list-images image: compute-rhel6.5 kernel: 2.6.32-358.el6.x86_64 image: compute-rhel6.5-mic-6720-19 kernel: 2.6.32-358.el6.x86_64 image: service-rhel6.5-mic-6720-19 kernel: 2.6.32-358.el6.x86_64

    
    

    The preceding output shows the newly installed image, service-rhel6.5-mic-6720-19.

13. Use the cimage command, in the following format, to set the default image for the service nodes:

    cimage --set-default --file_system service-rhel6.5-mic-6720-19 kernel

    
    

    The variables in this command are as follows:

    • For file_system, type either nfs or tmpfs, according to your site practice.

    • For kernel, type the kernel associated with the new image, as shown in the output from the cimage --list-images command in the following step:

      Step 12

    For example:

    # cimage --set-default --service-rhel6.5-mic-6720-19 2.6.32-358.el6.x86_64

    
    

14. Type the following command to confirm that the new image is the default image:

    # cimage --show-default service-rhel6.5-stout7-mic-6720-15 2.6.32-358.el6.x86_64 tmpfs

    
    

15. Proceed to the following:

    “Configuring the Switches”

Procedure 3-17. To verify your switches

1. Visually inspect your system.

   Note the types of switches you have and their identifiers. Switches are configured in stacks of two switches per stack. In each stack, the top switch is typically the master switch, and the bottom switch is typically the slave switch. At a minimum, you have one primary switch stack. The system admin controller (SAC) connects to the master switch in the primary switch stack. You might have additional secondary switch stacks.

   Within each stack, each switch is labeled MSW xx. In the primary switch stack, the master switch is labeled MSW0A, and the slave switch is labeled MSW0B. In the first secondary switch stack, the master switch is labeled MSW1A, and the slave switch is labeled MSW1B. The A and B on the switch labels identify the master switch and slave switch in the stack. The switchconfig set command operates on a switch stack (not just one switch), so you need to note only the characters on the label that precede the A and B when you provide information to the switchconfig set command. Use the following table to determine the value you need to use for switch on the switchconfig commands:

   switch		Label
   mgmtsw0		MSW0A or MSW0B
   mgmtsw1		MSW1A or MSW1B
   mgmtsw2		MSW2A or MSW2B
   mgmtsw3		MSW3A or MSW3B
   mgmtsw4		MSW4A or MSW4B
   mgmtsw5		MSW5A or MSW5B
   mgmtsw6		MSW6A or MSW6B
   mgmtsw7		MSW7A or MSW7B
   mgmtsw8		MSW8A or MSW8B
   mgmtsw9		MSW9A or MSW9B

2. Make sure that only the SAC is plugged in and that all rack circuit breakers are powered off.

   If you have a MAC file, connect all other nodes and switches to a power source, but do not power them on. That is, make sure that all chassis management controllers (CMCs), all rack leader controllers (RLCs), all service nodes, all switches, and so on, are not powered on.

   If you do not have a MAC file, unplug all other nodes and switches. The SAC should be the only component that is plugged in.

3. (Conditional) Remove the cascade cables that connect the slaves switches together.

   Perform this step only if you have two or more switch stacks.

   Locate the cascading cables that connect the switch stacks to each other, and unplug the cascading cable end from the lower switch in the neighboring switch stack. On the front of the switch stack, unplug the cascading cables that plug into the ports on the slave switches. When you unplug one end of each cable, you prevent a switching loop.

   ---

   Note: Do not unplug the stacking cables in the rear of the switch stack. The installation procedure instructs you to plug or unplug only the cascading cables in the front of the switch stack.

   ---

   For example, if you have one secondary switch stack, locate the cascading cable that runs from the slave switch on the primary switch to the slave switch on the secondary switch. Unplug that cable from the port on the slave switch.

4. Proceed as follows:

   • If you have a MAC file, proceed to the following:

     “Configuring Management Switches With a MAC File”

   • If you do not have a MAC file, proceed to the following:

     “Configuring Management Switches Without a MAC File”

Procedure 3-18. To configure switches -- with a MAC file

1. Through an ssh connection, log in as root to the SAC, and write the MAC file to a location on your SAC.

   For example, write it to /var/tmp/mac_file.

2. Plug in all the switches.

3. (Conditional) Specify a site-specific IP address for the head gateway.

   Perform this step if you need to set a site-specific, nondefault IP address for the primary switch stack (head gateway).

   By default, SGI sets the IP address for the primary switch stack (head gateway) to 172.23.0.254. Type the following command to set a site-specific IP address:

   # cadmin -set-head-gateway IP_addr

   
   

   For IP_addr, specify your site-specific IP address.

4. Use the discover command, in the following format, to configure the primary switch stack (the switches labeled MSW0A and MSW0B), which is attached to the SAC:

   # discover --mgmtswitch 0,gateway=yes --macfile path

   
   

   For path, type the full path to the location of the MAC file.

   This step assigns an IP address to the primary switch stack. The primary switch stack becomes the head gateway.

5. (Conditional) Type additional discover commands for each secondary switch stack.

   Complete this step if you have more than one switch.

   The formats for these additional commands are as follows:

   # discover --mgmtswitch num --macfile path

   
   

   The arguments are as follows:

   Argument	Specification
   num	The identifier for the switch stack.
   path	The full path to the location of the MAC file.

   For example:

   # discover --mgmtswitch 1 --macfile /var/tmp/mac_file

   
   

6. (Conditional) On the front of the switch stack, plug the cascading cables into the ports on the slave switches.

   Perform this step only if you have two or more switch stacks.

   This step is the opposite of the following step, in which you unplugged the cascading cables from the ports on the slave switches:

   Step 3

7. Type the following command to retrieve information about the switches that you discovered, and examine the output for errors:

   # cnodes --mgmtswitch

   
   

8. Use the ssh(1) command to open a console window to the SAC.

9. Type the following command, and monitor the power-on process in the SAC's console window:

   # tail -f /var/log/cmcdetectd.log

   
   

10. Flip the power breakers on the SGI ICE X's CMCs, one rack at a time.

    The cmcdetectd daemon runs on the SAC. It configures the top level switches so that the CMCs are on the appropriate rack VLAN. After you power on rack one, the cmcdetectd daemon detects the rack and adds the rack to the switch. After the CMCs for rack one appear on the switch, power on the CMCs for rack two.

11. Use the switchconfig command, in the following format, to list the CMCs associated with each switch:

    switchconfig list -s switch

    
    

    For switch, specify the system ID for a switch, for example mgmtsw0, mgmtsw1, and so on.

    Issue one of these commands for each switch in your configuration and examine the output. If the output includes all the CMCs in your SGI ICE X system, you can proceed with your configuration. If the output contains errors or does not include all expected CMCs, contact your SGI representative for troubleshooting information.

12. Save the configuration to the nonvolatile memory (flash) on the switches.

    ---

    Note: This step is very important. In the event of a power outage or other interruption, the switch stack boots with the saved configuration.

    ---

    Type the switchconfig command in the following format:

    # switchconfig save_running_config -s mgmtsw0[,mgmtsw1,mgmtsw2,...]

    
    

    Include the parameters mgmtsw1, mgmtsw2 , and so on, only if there are switches in addition to the primary switch (mgmtsw0).

13. Use the switchconfig command, in the following format, to back up the switch configuration to a file on the SAC:

    switchconfig pull_switch_config -s switch_ID -f file [--debug]

    The arguments are as follows:

    Argument	Specification
    switch_ID	The switch system ID. For example, this could be mgmtsw0, mgmtsw1, mgmtsw2, and so on. The output from the cnodes command includes this information.
    file	The name of the file to receive the switch configuration information. The command writes the file to the /tftpboot/ file.cfg. If your file specification ends in .cfg, the command does not append another .cfg string to the file name.

    The --debug parameter is optional. When specified, the command writes debugging information to /var/log/switchconfig .

    For example, the following command writes the configuration file for mgmtsw0 to file /tftpboot/mgmtsw0_startup1.cfg on the SAC:

    switchconfig pull_switch_config -s mgmtsw0 -f mgmtsw0_startup1 --debug

    
    

    Issue one of these commands per switch.

    In the future, if you need to replace the switch, you can save configuration time if you push out this configuration file from the SAC to the new switch.

14. After all management switches have been configured, proceed as follows::

    • If you have MCells, proceed to the following:

      “(Conditional) Configuring the Cooling Racks and Cooling Distribution Units (CDUs) on the MCell Network's Switch Ports”

    • If you do not have MCells, proceed to the following:

      “Configuring the Rack Leader Controllers (RLCs) and Service Nodes with the discover Command”

Procedure 3-19. To configure switches -- without a MAC file

1. (Conditional) Specify a site-specific IP address for the head gateway.

   Perform this step if you need to set a site-specific, nondefault IP address for the primary switch stack (head gateway).

   By default, SGI sets the IP address for the primary switch stack (head gateway) to 172.23.0.254. Type the following command to set a site-specific IP address:

   # cadmin -set-head-gateway IP_addr

   
   

   For IP_addr, specify your site-specific IP address.

2. Through an ssh connection, log in to the SAC as the root user, and type the following command:

   # discover --mgmtswitch 0,gateway=yes

   
   

3. When prompted, connect the primary switch stack to a power source.

   To complete this step, plug in the master switch and then the slave switch so that the entire primary switch stack is powered up. In this way, the master switch boots just a few seconds before the slave switch.

   The discover command configures the MAC address of the switch after you connect the primary switch stack to a power source.

4. (Conditional) Configure each secondary switch stack.

   Perform this step if you have secondary switch stacks.

   Complete the following steps for each secondary switch stack:

   • Plug in the switch stack when prompted by the system.

   • Type a discover command for the switch in the following format:

     # discover --mgmtswitch num

     
     

     For num, type the identifier for the switch.

     For example:

     # discover --mgmtswitch 1

     
     

5. (Conditional) On the front of the switch stack, plug the cascading cables into the ports on the slave switches.

   Perform this step only if you have secondary switch stacks.

   This step is the opposite of the following step, in which you unplugged the cascading cables from the ports on the slave switches:

   Step 3

6. Type the following command to retrieve information about the switches that you configured, and examine the output for errors:

   # cnodes --mgmtswitch

   
   

7. Use the ssh(1) command to open a console window to the SAC.

8. Type the following command, and monitor the power-on process in the SAC's console window:

   # tail -f /var/log/cmcdetectd.log

   
   

9. Flip the power breakers on the SGI ICE X's CMCs, one rack at a time.

   The cmcdetectd daemon runs on the SAC. It configures the top level switches so that the CMCs are on the appropriate rack VLAN. After you power on rack one, the cmcdetectd daemon detects the rack and adds the rack to the switch. After the CMCs for rack one appear on the switch, power on the CMCs for rack two.

10. Use the switchconfig command, in the following format, to list the CMCs associated with each switch:

    switchconfig list -s switch

    
    

    For switch, specify the system ID for a switch, for example mgmtsw0, mgmtsw1, and so on.

    Issue one of these commands for each switch in your configuration and examine the output. If the output includes all the CMCs in your SGI ICE X system, you can proceed with your configuration. If the output contains errors or does not include all expected CMCs, contact your SGI representative for troubleshooting information.

11. Save the configuration to the nonvolatile memory (flash) on the switches.

    ---

    Note: This step is very important. In the event of a power outage or other interruption, the switch stack boots with the saved configuration.

    ---

    Type the switchconfig command in the following format:

    # switchconfig save_running_config -s mgmtsw0[,mgmtsw1,mgmtsw2,...]

    
    

    Include the parameters mgmtsw1, mgmtsw2 , and so on, only if there are switches in addition to the primary switch (mgmtsw0).

12. For each switch you configured, use the switchconfig command, in the following format, to back up the switch configuration to a file on the SAC:

    switchconfig pull_switch_config -s switch_ID -f file [--debug]

    The arguments are as follows:

    Argument	Specification
    switch_ID	The switch system ID. For example, this could be mgmtsw0, mgmtsw1, mgmtsw2, and so on. The output from the cnodes command includes this information.
    file	The name of the file to receive the switch configuration information. The command writes the file to the /tftpboot/ file.cfg. If your file specification ends in .cfg, the command does not append another .cfg string to the file name.

    The --debug parameter is optional. When specified, the command writes debugging information to /var/log/switchconfig .

    For example, the following command writes the configuration file for mgmtsw0 to file /tftpboot/mgmtsw0_startup1.cfg on the SAC:

    switchconfig pull_switch_config -s mgmtsw0 -f mgmtsw0_startup1 --debug

    
    

    Issue one of these commands per switch.

    In the future, if you need to replace a switch, you can save configuration time if you push out this configuration file from the SAC to the new switch.

13. Use the discover command, in the following format, to save the MAC addresses to a MAC file:

    # discover --show-macfile > path

    For path, type the full path to the location of the MAC file. For example, /var/tmp/mac_file.

    You can use this MAC file if you ever have to configure the switches again.

14. After all management switches have been configured, proceed as follows:

    • If you have MCells, proceed to the following:

      “(Conditional) Configuring the Cooling Racks and Cooling Distribution Units (CDUs) on the MCell Network's Switch Ports”

    • If you do not have MCells, proceed to the following:

      “Configuring the Rack Leader Controllers (RLCs) and Service Nodes with the discover Command”

Procedure 3-20. To configure MCell switches

1. Gather information about the MCell switches in your SGI ICE X system.

   Visually inspect your system. Note the switches identifiers, and note the port identifiers.

2. Log in as the root user to the system admin controller (SAC).

3. Type following command to retrieve information about the virtual local area networks (VLANs) that are configured at this time:

   # cattr list -g mcell_vlan global mcell_vlan : 3

   
   

   The preceding output shows that the MCell VLAN is VLAN 3.

4. Use the switchconfig set command, in the following format, to configure the ports on which the CDUs and the CRCs are connected to the MCells:

   switchconfig set -v num=vlan_num -b=none -d vlan_num -p ports -s switch

   
   

   Type an individual switchconfig set command for each switch on the SGI ICE X system network.

   The arguments are as follows:

   Argument	Specification
   vlan_num	The VLAN number of the MCell network. For vlan_num, use the output from the cattr list command as shown earlier in this procedure. The default is 3, and SGI recommends that you do not change this value. This argument appears in two places in the switchconfig command.
   ports	Specify the target ports. The command configures both the target ports and the corresponding redundant ports.
   switch	The ID number of the management switch to which the CDU or CDC is attached. For example: mgmtsw0. To determine this value, you need to visually inspect the switch, as follows: Locate each CDU or CDC. The following are example labels for CDUs: DU01, DU02, and so on. Follow the cable that connects each CDU or CDC to a switch. The following is an example label for a cable that connects each CDU to a switch: DU01-LAN1 | 101MSW0A-36. Note the label on the switch. Make sure that the labels on the cables correspond to the labels on the switch ports.

   Example 1. The following command configures VLAN 3 on management switch 0 for target ports 1/31, 1/32, and 1/33 and for redundant ports 2/31, 2/32, and 2/33:

   switchconfig set -v num=3 -b=none -d 3 -p 1/31,1/32,1/33 -s mgmtsw0

   
   

   Example 2. The following command configures VLAN 3 on management switch 0 for target ports 2/31, 2/32, and 2/33 and for redundant ports 1/31, 1/32, and 1/33:

   switchconfig set -v num=3 -b=none -d 3 -p 2/31,2/32,2/33 -s mgmtsw0

   
   

   ---

   Note: If you make a mistake in your configuration, you can disable the ports from the VLANs you configured. The following example command removes the configuration of VLAN 3 from the target ports and the redundant ports: switchconfig unset -p 1/31,1/32,1/33 -s mgmtsw0 -v num=3

   ---

   
   

5. Repeat the following step for each CDU and each CRC attached to your system:

   Step 4

   If you encounter errors, issue a switchconfig set command again.

6. Save the configuration to the nonvolatile memory (flash) on the switches.

   ---

   Note: This step is very important. In the event of a power outage or other interruption, the switch stack boots with the saved configuration.

   ---

   Type the switchconfig command in the following format:

   switchconfig save_running_config -s mgmtsw0[,mgmtsw1,mgmtsw2,...]

   
   

   Include the parameters mgmtsw1, mgmtsw2 , and so on, only if there are switches in addition to the primary switch (mgmtsw0).

7. Type the following command to back up the switch configuration to a file on the SAC:

   switchconfig pull_switch_config -s switch_ID -f file

   The arguments are as follows:

   Argument	Specification
   switch_ID	The switch system ID. For example, this could be mgmtsw0, mgmtsw1, mgmtsw2, and so on. The output from the cnodes command includes this information.
   file	The name of the file to receive the switch configuration information. The command writes the file to the /tftpboot/ file.cfg. If your file specification ends in .cfg, the command does not append another .cfg string to the file name.

   For example, the following command writes the configuration file for mgmtsw0 to file /tftpboot/mgmtsw0_startup1.cfg on the SAC:

   switchconfig pull_switch_config -s mgmtsw0 -f mgmtsw0_startup1 [--debug]

   
   

   The --debug parameter is optional. When specified, the command writes debugging information to /var/log/switchconfig .

   In the future, if you need to replace the switch, you can save configuration time if you push out this configuration file from the SAC to the new switch.

8. After all switches have been discovered, proceed to the following:

   “Configuring the Rack Leader Controllers (RLCs) and Service Nodes with the discover Command”