This chapter includes the following topics:
The SGI Integrated Compute Environment (ICE) X systems provide an integrated compute node (blade) environment that can include thousands of compute nodes. The SGI Tempo software for SGI ICE X systems enables you to provision, install, configure, and manage your system. This manual describes how to configure the software on the system for use at your site. This manual does not include an SGI ICE X hardware description. For hardware information, see the SGI ICE X System Hardware User Guide.
Figure 1-1 shows a simple SGI ICE X system.
SGI ICE X system processes are distributed across several controllers and service nodes. Each node has a specific role. These components are as follows:
One system admin controller (SAC). The site system administrator can log into the SAC as the root user to monitor the system, to modify the master software images, and to perform general system administration tasks.
One rack leader controller (RLC) per compute node (blade) rack. You can have multiple compute nodes in the racks, and you can have multiple racks. Each RLC manages a set of compute nodes in a particular rack.
One or more rack leader controllers (RLCs). The RLCs manage the compute nodes (blades) in the rack enclosures.
One or more service nodes. End users can log into a service node to run jobs. There can be several service nodes, and they can host single services or multiple services. The service nodes can host one or more of the following types of services:
Login services. These services allow an end user to log in and then, for example, run or monitor MPI jobs.
Batch scheduling services. You can install schedulers such as Altair's PBS Professional or TORQUE.
I/O gateway. On a small system, you can combine the I/O gateway, login services, and batch scheduling on the same node. The I/O gateway services connect the SGI ICE X system to your house network. You can configure one or more of the following protocols on the service node: network file system (NFS), network address translation (NAT), network information service (NIS).
Storage. A storage service node is a network attached (NAS) appliance bundle that provides InfiniBand attached storage for the system.
Object storage server. This service is used in Lustre File Storage configurations.
Metadata server. This service is used in Lustre File Storage configurations.
The software distribution includes the master system image for the SAC. During installation and configuration process, the installation software creates the master system images for the RLCs, the service nodes, and the compute nodes. As you customize the system for your site, you can modify the node-specific system images on the SAC and push the updated images to the RLCs, to the service nodes, or to the compute nodes.
The SGI ICE X system includes several virtual local area networks (VLANs). Figure 1-2 is a logical representation of the SGI ICE X Ethernet network that shows three VLANs in an example SGI ICE X system.
In Figure 1-2, the head network is VLAN 1. The SAC and service nodes are attached to the customer LAN.
The system components are attached to one or more of the following two internal networks:
The high-performance network, which is designed for computation. This InfiniBand network facilitates communication to all compute nodes from the service nodes. It connects the following:
The compute nodes to each other. The IB network connects all the compute nodes (blades) to each other. The compute nodes are not part of the head network.
The service nodes to the compute nodes.
The head network, which is the administrative network. This Ethernet network is designed for communication between the SAC, RLCs, and service nodes. These components communicate to each other directly within the head network. The head network includes the following Ethernet connections:
The SAC to the Ethernet switches.
The Ethernet switches to the SAC, RLCs, and service nodes.
The service nodes to the Ethernet switches.
One or two separate InfiniBand networks segregate traffic within the SGI ICE X system in a way that optimizes computing performance. When there are two InfiniBand networks, communication is segregated by network, as follows:
ib0, which is typically used for Message Passing Interface (MPI) communication.
ib1, which is typically used for storage traffic.
Users can log into the SAC and into the service nodes directly. If you need to reach the RLC, you need to log into the SAC first, and then ssh(1) into the RLC.
The RLCs and compute nodes are attached to a separate Ethernet network. An Ethernet network connects each RLC to each of the compute nodes that the RLC controls. Communication to the compute nodes always goes through the RLC. Only the RLC for a particular rack can communicate with the compute nodes on its own rack. Each compute node has its own Ethernet IP address within its rack. If you have multiple racks, compute node IP addresses are duplicated across the racks.