If you later want to add an execution node to the cluster, you can manually install packages on the node by using the rpm(8) command or you can use the installace utility as follows to install software on just that node:
In interactive mode, bypass installation on the head node by entering a dot (.) for the head node name. When prompted for the execution nodes, enter only the name of the new node. See “Interactive Mode Example”.
In batch mode, bypass installation on the head node by leaving the Head_node= field blank. Include only the name of the new node in the Execution_nodes= field. See “Batch Mode Example”
For more information about installace, see “Interactive Mode Instructions” in Chapter 4, and “Batch Mode Instructions ” in Chapter 4.
After the software is installed, perform the following configuration tasks on the new execution node and head node, according to the products you are using.
Do the following on the new execution node:
Ethernet MPICH:
Add all cluster node names to the /usr/share/machines.LINUX file
Myrinet GM-MPICH:
Add all cluster node names to the /usr/util/machines/machines.LINUX file
PBS:
PCP:
Start the pmcd daemon:
# /etc/rc.d/init.d/pcp start |
Lconsole:
Do the following on the head node:
All products:
Edit the /etc/ace/nodes file to include the name of the new execution node.
Myrinet GM-MPICH:
Edit the gm configuration files ($HOME/.gmpi/confand /etc/ace/gmpi.conf by default) according to the directions in Chapter 7, “Configure gm-mpich”.
Add the node name to the /usr/util/machines/machines.LINUX file.
Tell users to copy the an existing gm configuration file (such as $HOME/.gmpi/conf).
Ethernet MPICH:
Add the node name to the /usr/share/machines.LINUX file.
PBS:
The following example shows user entries in bold:
root@ace07 /root]# /bin/mount /dev/cdrom /mnt/cdrom
[root@ace07 /root]# cd /mnt/cdrom
[root@ace07 cdrom]# ./installace
ACE nodes file (/etc/ace/nodes) already exists.
If you continue with the install, you will be redefining your cluster.
CONTROL-C if you want to stop this install.
Advanced Cluster Environment v. 1.4 for Linux i686 includes the
following products:
pbs PBS batch-queuing system
pcp Performance Co-Pilot
mpich MPICH message-passing library - ethernet
gm-mpich Myricom's MPICH message-passing library
gm Myricom's GM driver
lconsole Command-line serial console
hoover Hoover cluster manager & serial console
Enter "ethernet" to select:
mpich pbs pcp lconsole
Enter "myrinet" to select:
gm gm-mpich pbs pcp lconsole
Or enter the individual products separated by white space.
Please enter all desired products separated by white space.
ACE> eth
Note: When providing a hostname, use the fully qualified name
such as "foo.domain.com". If the hostname is resolved on all
nodes, you can abbreviate it to "foo".
The installace utility can modify hosts.equiv files on your nodes.
These files are used by MPICH, GM-MPICH, and PBS.
You can choose to provide the information required to edit these files
or avoid this action by entering the appropriate number:
1) Enter the hostnames individually
2) Specify a file containing the list of hostnames
3) Specify a pattern for hostnames (NAME1.domain.com)
4) Install software only on this node
Enter choice (1, 2, 3, or 4).
ACE> 1
Enter the hostname of the head node.
The head node is the host from which users will launch
execution of parallel jobs.
Enter the hostname of the head node.
ACE> .
Enter the hostnames for each execution node in the
cluster. After you have entered all hostnames,
Enter '.' or <control-D> on a new line to end the list.
ACE> ace06
ACE> .
The install process can update the /etc/hosts.equiv file on all
the hosts in the cluster. The /etc/hosts.equiv will allow access
to the hosts. If /etc/hosts.equiv file is not being used, each user
must update their .rhosts file to allow rcp/rsh type access to all
cluster nodes. Enter one of the following:
1) Add the specified hostnames to the existing /etc/hosts.equiv files.
If the file does not exist, create it.
2) Replace the current /etc/hosts.equiv file with a new one
containing only the specified cluster hostnames. The current
/etc/hosts.equiv will be saved as /etc/hosts.equiv.old.
3) Do not change the existing /etc/hosts.equiv file.
Enter choice (1, 2, or 3).
ACE> 1
Checking rsync/rcp/rsh access ...
Checking disk space requirements ...
Installing selected ACE products on node(s)
ace-tools ........... [yes]
ace-docs ............ [yes]
pcp ................. [yes]
pbs ................. [yes]
mpich ............... [yes]
gm .................. [no]
gm-mpich ............ [no]
lconsole ............ [yes]
hoover............... [no]
Installing execution node software on ace06
Installing package mpich-1.2.1-1 on ace06 ...
MPICH postinstall script
Installing package pcp-2.1.9-12 on ace06 ...
Installing package pcp-pro-2.1.5-2 on ace06 ...
Installing package pcp-ace-1.3.0-4 on ace06 ...
Creating MPI wrapper
...Done, output in /var/tmp/pcp-ace-1.3.0-4-mpi_wrapper
Package textutils-2.0e-3 already installed on ace06, skipping.
Installing package pbs-mom-2.2-11RH6 on ace06 ...
PBS mom preinstall script
PBS mom postinstall script
Configuring selected ACE products on node(s) ...
Configuring PBS on ace06 ...
PBS has not been configured on the head node for
this new execution node.
You must do the following to properly configure PBS:
1) Add the name of the head node to ace06:/usr/spool/pbs/server_name
2) Edit ace06:/usr/spool/pbs/mom_priv/config to add the following:
$logevent 0x1ff
$clienthost 3) Edit the head node /usr/spool/pbs/server_priv/nodes file and
add the execution node ace06.
Starting PCP daemons ...
Starting PCP on ace06 ...
Post configuration ...
Adding to the /etc/hosts.equiv file on all nodes ...
This ACE release also contains third-party evaluation
software. See /mnt/cdrom/ace-demos/README.TXT for
more information.
The installation/configuration process has been completed! |
This section shows an example of a modified install.conf file and output.
The following example shows user changes (from the template /mnt/cdrom/install.conf file) in bold:
# # This file contains Advanced Cluster Environment (ACE) # configuration information that can be used by installace. # # This file must be /bin/sh parsable. # # # Advanced Cluster Environment for Linux, includes the # following products: # # pbs PBS batch-queuing system # pcp Performance Co-Pilot # gm Myricom's GM driver # gm-mpich Myricom's enhanced MPICH message-passing library # mpich MPICH message-passing library # lconsole Command-line serial console # hoover Hoover cluster manager & serial console # # Enter "ethernet" to select: # mpich pbs pcp lcdinc # # Enter "myrinet" to select: # gm gm-mpich pbs pcp lcdinc # # List all desired products (separated by commas or white space) # in the 'Products' variable. Products="eth" # # ACE needs to know about all the nodes in the cluster. # There are two ways to specify this information: # 1) Create a separate file containing all the nodes in the cluster. # The first node listed must be the head node. # If you want to use this option, list the full pathname in # he 'Nodes_filename' variable below. # # 2) Define the head and execution nodes in this configuration file. # To do this, fill in the 'Head_node' and the 'Execution_nodes' # variables below. # # # List the full path to the file containing all cluster nodes. # The first node listed must be the head node. # Leave empty if you want to specify nodes below. # Nodes_filename="" # List the head node. The head node is the host from which # users will submit PBS requests and launch the execution of parallel # jobs. Use the fully qualified name such as "foo.domain.com". # # If you have left 'Nodes_filename' variable empty and you do not # fill in a node for 'Head_node', only the software appropriate to # execution nodes will be installed. In this case, you must manually # configure PBS on the head node for the execution nodes. Head_node="" # # NOTE: If you have left 'Nodes_filename' variable empty and # you do not list a node for 'Head_node', only the software # appropriate to execution nodes will be installed. In this case, # you must manually configure PBS for each new execution node # by doing the following: # # 1) Add the name of the head node to /usr/spool/pbs/server_name # on the execution nodes. # 2) Edit /usr/spool/pbs/mom_priv/config on the execution nodes # to add the following: # $logevent 0x1ff # $clienthost # 3) Edit the /usr/spool/pbs/server_priv/nodes file on the head node # and add the name of the execution nodes. # # List all execution nodes in the cluster. They can be comma or # white space, including newline, separated. # Use the fully qualified name such as "foo.domain.com". # # If you want the head to also be an execution node, you # must also list it in the 'Execution_nodes' variable. # Execution_nodes="ace06" # # The install process can update the /etc/hosts.equiv file on all # the nodes in the cluster. The /etc/hosts.equiv file allows access # to the nodes. The MPICH, PBS, and GM products use rsh/rcp to access # other nodes. If /etc/hosts.equiv is not updated, all users will # have to update their .rhosts file on all nodes. # Set the 'Hosts_equiv_file' variable below to one of # the following values: # # "add") Add the specified nodes to the existing /etc/hosts.equiv files. # If /etc/hosts.equiv does not exist, installace will create one. # "replace") Replace the current /etc/hosts.equiv file with a new one # containing only the specified nodes. The current # /etc/hosts.equiv will be saved as /etc/hosts.equiv.old. # "nochange") Do not change the existing /etc/hosts.equiv file. # If the variable is not set, the default will be "nochange". Hosts_equiv_file="add" |
The following shows the output when using the above file:
[root@ace08 cdrom]# ./installace /tmp/install.conf
ACE nodes file (/etc/ace/nodes) already exists.
If you continue with the install, you will be redefining your cluster.
CONTROL-C if you want to stop this install.
Using install config file '/tmp/install.conf'.
Checking rsync/rcp/rsh access ...
Checking disk space requirements ...
Installing selected ACE products on node(s)
ace-tools ........... [yes]
ace-docs ............ [yes]
pcp ................. [yes]
pbs ................. [yes]
mpich ............... [yes]
gm .................. [no]
gm-mpich ............ [no]
lconsole ............ [yes]
hoover .............. [no]
Installing execution node software on ace06
Installing package mpich-1.2.1-1 on ace06 ...
MPICH postinstall script
Installing package pcp-2.1.9-12 on ace06 ...
Installing package pcp-pro-2.1.5-2 on ace06 ...
Installing package pcp-ace-1.3.0-4 on ace06 ...
Creating MPI wrapper
...Done, output in /var/tmp/pcp-ace-1.3.0-4-mpi_wrapper
Package textutils-2.0e-3 already installed on ace06, skipping.
Installing package OpenPBS-mom-2-3-mom on ace06 ...
PBS mom preinstall script
PBS mom postinstall script
Configuring selected ACE products on node(s) ...
Configuring PBS on ace06 ...
PBS has not been configured on the head node for
this new execution node.
You must do the following to properly configure PBS:
1) Add the name of the head node to ace06:/usr/spool/pbs/server_name
2) Edit ace06:/usr/spool/pbs/mom_priv/config to add the following:
$logevent 0x1ff
$clienthost 3) Edit the head node /usr/spool/pbs/server_priv/nodes file and
add the execution node ace06.
Starting PCP daemons ...
Starting PCP on ace06 ...
Post configuration ...
Adding to the /etc/hosts.equiv file on all nodes ...
This ACE release also contains third-party evaluation
software. See /mnt/cdrom/ace-demos/README.TXT for
more information.
The installation/configuration process has been completed! |