The nice value is the execution priority of the processes that form the batch request. A user can explicitly set the nice value (through the Job Limits selection of the Configure menu on the NQE GUI Submit window or as a cqsub or qsub command option) or NQS can set the nice value implicitly, based on the queue in which the request resides. Increasingly negative nice values increase the relative execution priority of a process.

---

Warning: Do not specify a negative nice increment for a queue because it may cause serious CPU scheduling problems, such as timeouts in system daemons or interference with kernel scheduling.

---

A batch request can specify the full path name of a shell to interpret its shell script. If a user does not specify a shell, the user's login shell is used.

When NQS initiates a request, it spawns a second shell.

The second shell invoked is always /bin/sh unless another shell is specified by the user. The two-shell method allows stdin read-ahead by commands such as remsh and cat.

To define a shell for the first shell invoked, users can use the General Options selection of the Configure menu on the NQE GUI Submit window or use the cqsub -s or qsub -s command.

Users can define a shell other than /bin/sh to interpret their batch request by including the following line as the first line of the file (before any #QSUB directives):

#! shell_path  (such as /bin/csh)

You can include shell options on this line.

NQS sets the QSUB_SHELL environment variable to the shell the user is in when executing a request. NQS sets the SHELL environment variable to the initial shell executed by NQS.

The initial shell is not necessarily the same shell as the second shell. The second shell will always be /bin/sh, unless the user includes #! shell_path as the first line of the batch request.

The following example uses the cqsub command and shows how to verify the shells that an NQS request is using. User jjj uses csh as a login shell. The batch request file job1 contains only one line, as follows:

ps -ef | fgrep jjj

The request is submitted by using the following command:

cqsub -s /bin/csh job1

The following output shows that the request was executed under /bin/sh because jjj did not modify the first line of the file as required:

jjj 11874 11873 2 15:18:17 ?      0:00 fgrep jjj
jjj 11355 11354 0 14:54:37 pts/0    0:00 -csh
jjj 11873 11871 2 15:18:17 ?        0:00 /bin/sh
/nqebase/nqeversion/poe/database/spool/scripts/++++2++cd4X+++
jjj 11871 11870 22 15:18:16 ?        0:00 -csh
jjj 11875 11874 9 15:18:17 ?        0:00 ps -ef

To run the request under csh, you must modify the batch request file as follows:

#!/bin/csh
ps -ef | fgrep jjj

The cqsub -s /bin/csh job1 command produces the following output:

jjj 11969 11967 19 15:23:06 ?  0:00 /bin/csh
/nqebase/nqeversion/pendulum/database/spool/scripts/++++3++cd4X+++
jjj 11355 11354 0 14:54:37 pts/0 0:00 -csh
jjj 11967 11966 5 15:23:05 ?   0:00 -csh
jjj 11974 11969 9 15:23:08 ?   0:00 ps -ef

If you want to use a one-shell invocation method, you can set the NQSCHGINVOKE environment variable to true or yes. The environment variable is set on a per-request basis; you must export environment variables by using the General Options selection of the Configure menu on the NQE GUI Submit window or by using the qsub -x option. If NQSCHGINVOKE is set to true, and you do not request a shell (by using the General Options selection of the Configure menu on the NQE GUI Submit window or by using cqsub -s or qsub -s), NQS invokes the request owner's UDB shell on UNICOS and UNICOS/mk systems or their login shell on UNIX systems.

To set a fixed shell path and invocation method, edit the nqeinfo file to define the following two variables:

NQE_NQS_SHELLPATH
NQE_NQS_SHELLINVOCATION

The default values are a null string for the shell path and 2 for the invocation method.

The allowed value for NQE_NQS_SHELLPATH is any valid shell path name. The default null string for NQE_NQS_SHELLPATH uses the user's login shell. If you specify a shell path name for NQE_NQS_SHELLPATH, the shell you specify is used for batch request processing.

The value specified by using the General Options selection of the Configure menu on the NQE GUI Submit window or specified by using the cqsub -s or qsub -s option, if used, overrides the system-level shell path configured in the nqeinfo file.

The allowed value for NQE_NQS_SHELLINVOCATION is 1 or 2. A value of 1 for NQE_NQS_SHELLINVOCATION uses a one-shell invocation method for all NQS requests. A value of 2 uses the default two-shell invocation method.

The value of any NQSCHGINVOKE environment variable received with a request overrides the system-level invocation method configured in the nqeinfo file.

During NQS startup, the values of these two variables are verified. If a stat() system call of a non-null shell path name fails, or if the value of the invocation method is not either 1 or 2, the nqsdaemon aborts its startup. Messages describing the problem are written to the NQS console file and/or the log file.

---

Note: Alternate user validation applies only for NQS requests submitted using the cqsub command.

---

By default, an NQE job is submitted for execution under the user name executing the cqsub command. By using the cqsub -u command, or by setting the User Name field in the NQE GUI Submit display, a user can submit a request for execution under a different user name.

The user account on the client workstation is referred to as the originating client user account, and the user account on the NQS server is referred to as the target user account.

The nqeinfo file variable NQE_NQS_NQCACCT controls which user account is validated on the NQS_SERVER when a user submits a job for execution under a different user name. The value of this variable on the NQS_SERVER host determines which user name is used for validation on the NQS_SERVER.

The value ORIGIN means that the originating client user account is validated on the NQS_SERVER. This means that the client user must have an account on the NQS_SERVER with the same name (but not necessarily the same UID) as the originating client user account.

The value TARGET means that the target user account is validated on the NQS_SERVER. It is not necessary for the client user to have an account on the NQS_SERVER with the same name as the client user account.

If the NQE_NQS_NQCACCT variable is not set on the NQS_SERVER, it defaults to the value ORIGIN.

If NQE_NQS_NQCACCT is set to ORIGIN, and password validation is set on the NQS_SERVER, then both the client user account on the NQS_SERVER and the target user account on the NQS server must have the same password or the password validation will fail.

The intraqueue priority determines the order in which requests are initiated. After a request is accepted into a batch queue, the NQS scheduler calculates the intraqueue priority each time it scans the requests in the queue. The order in which queues are considered does not change; only the order in which requests are selected from a queue changes. The eligible request with the highest intraqueue priority is selected next for initiation. (Eligible means the request meets all quotas such as run limit, user limit, and group limit). For additional information, see “Defining Job Scheduling Parameters” in Chapter 5.

A request or the individual processes of a request can impose a set of limits on the use of a resource. Typical limits are maximum file size, CPU time, and maximum memory size. A user can explicitly set up resource limits by using NQE GUI options or cqsub command options, or NQS can set the resource limits implicitly, based on the batch queue in which the request executes.

The limits supported by NQS are described in “Defining Per-request and Per-process Limits” in Chapter 5.

Batch queues accept and process batch requests. Each batch queue has an associated set of request and process resource limits. If a request is to be accepted in a particular batch queue, no resource limit specified by the request may exceed the corresponding limit specified by the target batch queue. If a batch request fails to specify a resource limit that is enforced, NQS sets the limit for the request to the corresponding default limit of the queue. The qmgr commands define the queue default values.

---

Note: For UNICOS and UNICOS/mk systems, NQS assigns the limit for the request to either the default limit of the queue or the user database (UDB) limit, whichever is more restrictive.

---

If the qmgr manager lowers the request and process resource of a queue, all requests that are already in the queue with quotas that exceed the new limits remain in the queue through a "grandfather" clause.

Every batch queue also has a set of associated queue limits, which are described in “Queue Attributes”.

Pipe queues handle the routing and delivery of requests from one queue to another. They serve as pipelines, transporting requests to other local and remote queue destinations.

Pipe queues do not have associated quota limits (although they can have attributes). Pipe queues have a set of associated queue destinations, on both local and remote machines, to which they route requests. These destinations can be batch queues or other pipe queues.

NLB pipe queues used for load balancing do not have destinations associated with them.

Each request in a pipe queue is routed to one of the queue's destinations. Destinations are considered in the order in which the administrator configures them. If a request cannot be accepted by a destination queue (for example, if its resource requirements exceed those allowed for membership in the queue), the next destination is considered. If no destination accepts the request, the request is deleted, and the user receives a mail message.

If a destination is inaccessible (for example, if the network connection is broken or if the loadonly limit for a queue has been reached), NQS tries to requeue the request at specified intervals (defined by the qmgr command set default destination_retry wait) for a specified length of time (defined by the qmgr command set default destination_retry time). If these attempts fail, NQS abandons the attempted request and the user receives a mail message.

Each pipe queue has an associated program spawned to handle every request initiated from the queue for routing and delivery. In the context of the client/server network connection, this program is referred to as the pipeclient process.

Queue, queue complex, and global limits (defined beginning with “Queue Attributes”) limit the number of requests that can run concurrently. You can use qmgr commands to set a finite value that limits the maximum number of requests that can run at one time. Most limits can be set to unlimited. When a queue is created, these limits are defined as unspecified; unspecified means that the queue has the default value for that limit.

You do not have to set every limit. A comprehensive set of limits is provided so that you can configure the workload to meet your site's needs.

By default, requests are sent to NQS. You may change the default and have requests sent to the NQE database, or your users may select the destination of their individual requests. For additional information about configuring the NQE database, see Chapter 9, “NQE Database”.

When a user submits a batch request to NQS, the request is sent to nqsdaemon. The nqsdaemon checks the request qualifiers (if there are any) against the defined queue limits and attributes. Based on these checks, nqsdaemon accepts or rejects the request. If the queue is a pipe queue, when nqsdaemon determines that a request should be routed, it spawns the pipeclient program to route the request from the pipe queue to another pipe or batch queue. Figure 2-1 shows this process.

Figure 2-1. Request Processing for Local Submission

---

Note:: The concept of local submission does not apply for requests submitted to the NQE database.

---

A user can submit a request to NQS or to the NQE database. The following sections describe the process of submitting a request to each destination.

Client Submission to NQS

When a user submits a request to NQS using the NQE GUI Submit window or the NQE cqsub command, no queues are involved on the local host. The request is sent to the netdaemon process on the server machine as specified by the NQS_SERVER variable. The netdaemon creates a child process (netserver) to handle the request. The child process tries to queue the request with nqsdaemon. netserver ensures that the user has an account on the target host and that the user name on the local machine is authorized to run requests on the target machine.

The target nqsdaemon then checks the request qualifiers against the defined queue limits and attributes and, based on these checks, accepts or rejects the request. Figure 2-2 shows this process.

Figure 2-2. Request Processing for Client Submission to NQS

Client Submission to NQE Database

When a user submits a request to the NQE database using the NQE GUI Submit window or the NQE cqsub command, the NQE scheduler determines when and where the request will execute. The NQE database routes a copy of the request to the lightweight server (LWS) on the node running the NQS server. The LWS verifies validation, sends a copy of the request to the NQS batch queue for processing, and obtains exit status of completed requests.

Figure 2-3 shows request processing for client submission to the NQE database.

Figure 2-3. Request Processing for Client Submission to the NQE Database

If the request is sent from one NQS system to a remote NQS system, pipeclient sends the request to the remote netdaemon. The rest of the processing is the same as it is for client submission to NQS. Figure 2-4 shows this process.

---

Note:: The concept of remote submission does not apply for requests submitted to the NQE database.

---

Figure 2-4. Request Processing for Remote Submission

When nqsdaemon determines that a request is eligible to run, it spawns a shepherd process. The shepherd process sets up the environment for the request and runs it. It sends mail to the user if it cannot execute the request. The shepherd process waits for the request to complete and returns output file(s) to the specified or default file destination.

NQS users can control or delete their own requests by using the NQE GUI Status window Actions menu options or by using the cqdel or the qdel command. NQS operators and managers can control requests with the following qmgr commands (see “Manipulating Requests” in Chapter 6, for a description of these commands and examples of their use):

Managers and operators can also delete requests from any user by using the cqdel or qdel command.

The cqstatl command, the qstat command, and the qmgr commands show long queues and show queues display the current status of NQS queues.

The queue attributes described in the following sections apply to both batch and pipe queues.

The queue attributes described in the following sections apply only to batch queues.

loadonly

You may not submit requests directly to a loadonly queue. Requests may come only from a pipe queue. In other words, a loadonly queue is a pipeonly queue that has the additional restriction of being able to accept only a limited number of requests.

A batch queue can be declared loadonly, meaning that it can accept only a limited number of requests. The number of requests that can be queued is restricted to the run limit of that queue. Therefore, if no other limits exist, a loadonly queue accepts only the number of requests that it can run. Specifically, the algorithm is the number of requests queued + number of requests running + number of requests waiting + number of requests arriving <= run limit.

This algorithm can be used for load sharing between multiple machines or across multiple queues on a single machine. In the example in Figure 2-5, requests are submitted to the pipe queues [email protected] and [email protected]; they are routed from these queues to the central pipe queue, which queries all three batch queues on both hot and ice. The central pipe queue would try to route a small request to [email protected] first, next try [email protected], and continue down the list until a place in a queue became available. Without using loadonly queues, the queue [email protected] may accept the request, and the request may remain queued.

Figure 2-5. Queue Configuration That Uses theloadonly Attribute

The following sections describe the concepts used in the Cray Network Load Balancer (NLB).

The NQE database provides the following:

Figure 2-6 shows the basic concept of the NQE database in the NQE:

Figure 2-6. Basic Concept of the NQE Database

The components of the NQE database are described in the following sections.

A file transfer service is a program that provides FTA with access to a network system that uses a specific file transfer protocol.

A unique FTA domain_name is assigned to each of the file transfer services that are available to FTA. A domain name is subsequently used to identify a specific transfer service.

FTA defines specific types of users, as follows:

Network peer-to-peer authorization ( NPPA) lets users transfer files without sending a password across the network. It requires FTA on the local system and support for network peer-to-peer authorization on the remote system. It can be used to authorize both batch and interactive file transfers.

A file transfer request is an object containing information that describes the file transfer operations to be performed by FTA on behalf of a user. Each request is a separate file, which is located in the FTA queue directory.

A queue directory is a file system directory that contains file transfer request files.