Chapter 11. Monitoring Queues

This chapter describes how to use the cqstatl command and the qstat command to monitor NQS queues. This information does not apply to requests submitted to the NQE database. The following topics are covered:


Note: If you do not have an NQE license, you cannot access the NQE GUI and the cqstatl command. You can access only the qstat command from an NQS server.

If the UNICOS multilevel security (MLS) feature or the UNICOS/mk security enhancements are enabled on your system and NQS is configured to enforce mandatory access control (MAC), your active label must dominate the job submission label for you to receive status information. To display the job submission and execution label information for a specific job, use the cqstatl -f or qstat -f command. NQS managers and operators bypass the MAC checks.

Displaying Queue Summaries

NQS uses both batch and pipe queues. To display a summary of a specific type of queue, use one of the following cqstatl or qstat command options:

Option

Description

-b

A summary of all batch queues

-p

A summary of all pipe queues

To display summary information about all NQS queues at your NQS server, use either the cqstatl command or the qstat command; for example:

cqstatl

The display produced by this command includes information about pipe and batch queues.

Often this is a large display and will scroll off your screen. To control scrolling, redirect the output from cqstatl or qstat to the more(1) command; for example:

cqstatl | more

For example, the following display shows a summary of all NQS queues:

% cqstatl
-----------------------------
NQS BATCH QUEUE SUMMARY
-----------------------------
QUEUE NAME              LIM TOT ENA STS QUE RUN  WAI HLD ARR EXI
--------------------- --- --- --- --- --- ---  --- --- --- ---
nqebatch                5  11 yes  on   9   2    0   0   0   0
--------------------- --- --- --- --- --- ---  --- --- --- ---
latte                   5  11           9   2    0   0   0   0
--------------------- --- --- --- --- --- ---  --- --- --- ---
----------------------------
NQS PIPE QUEUE SUMMARY
----------------------------
QUEUE NAME            LIM TOT ENA STS QUE ROU  WAI HLD ARR DEP  DESTINATIONS
--------------------- --- --- --- --- --- ---  --- --- --- ---  -------------
nqenlb                  1   0 yes  on   0   0    0   0   0   0
--------------------- --- --- --- --- --- ---  --- --- --- ---  -------------
latte                   5   0           0   0    0   0   0   0
--------------------- --- --- --- --- --- ---  --- --- --- ---  -------------

Batch Queue Summary

The last line in the batch queue summary display shows the name of the server and the total for the server. The individual columns have the following meanings:

Column name 

Description

QUEUE NAME 

The name of the queue.

LIM 

The maximum number of requests that can execute in this queue simultaneously. When this limit is reached, other requests in the queue will remain queued until a request already in the queue completes execution.

TOT 

The total number of requests currently in the queue.

ENA 

The availability of the queue (that is, whether the NQE administrator has enabled the queue). If the queue is enabled (if ENA is yes), the queue can accept requests.

STS 

The status of the queue (that is, whether the NQE administrator has started the queue). If the queue has been started (STS is on), the queue will accept and queue requests. If the queue has not been started (STS is off), the queue will accept and queue requests, but it will not execute them.

QUE 

The number of requests in the queue that are queued and ready to execute. Queued requests have not begun execution because the queue has not been started (STS is off) or because starting any of the requests would exceed a system limit. For a description of system limits, see “NQS System Limits” in Chapter 4.

RUN 

The number of executing requests in the queue.

WAI 

The number of requests in the queue that are waiting to be executed. Requests can be waiting for a specified time. They also can wait for a license.

HLD 

The number of requests the NQE administrator has put into the hold state.

ARR 

The number of requests currently arriving from other queues.

EXI 

The number of requests currently terminating their processing.

Pipe Queue Summary

The last line in the pipe queue summary display shows the name of the server, the maximum number of requests that can be processed in all pipe queues at one time, and the total number of requests in each state for all pipe queues. The individual columns have the following meanings:

Column name 

Description

QUEUE NAME 

The name of the pipe queue.

LIM 

The maximum number of requests that can be processed in this queue at any one time. A request can still be placed in the queue when this limit is reached, but it will not be processed until the processing of a request in the queue is complete.

TOT 

The total number of requests currently in the queue.

ENA 

The availability of the queue (that is, whether the NQE administrator has enabled the queue). If the queue is enabled (if ENA is yes), the queue can accept requests.

STS 

The status of the queue (that is, whether the NQE administrator has started the queue). If the queue has been started (STS is on), the queue will accept and route requests. If the queue has not been started (STS is off), the queue will accept requests, but it will not route them.

QUE 

The number of requests in the queue that are awaiting processing.

ROU 

The number of requests in the queue that are being routed to another queue.

WAI 

The number of requests in the queue that are waiting to be processed at a specific time.

HLD 

The number of requests the NQE administrator has put into the hold state.

ARR 

The number of requests arriving from other queues.

DEP 

The number of requests on their way to another queue.

DESTINATIONS 

A list of the destination queues for this pipe queue.

Displaying Queue Details

To display full details about NQS queues, you can use either the cqstatl -f command or the qstat -f command ; for example:

cqstatl -f

As with the summary display, you can use the -b and -p options on the cqstatl or qstat command line to restrict the display to a particular type of queue. See “Displaying Queue Summaries”.

To restrict the detailed display to a particular queue, you can use one of the following formats:

cqstatl -f queue

qstat -f queue

Pipe Queue Details

The following screen shows an example of a detailed pipe queue display for a pipe queue called nqepipe:

The fields in this display have the following meanings:

% cqstatl -f nqenlb
-------------------------------------
NQS PIPE QUEUE: [email protected]    Status:      ENABLED/INACTIVE
-------------------------------------
                                         Priority:    63
<ENTRIES>
     Total:          0

     Running:        0       Queued:         0       Waiting:        0
     Holding:        0       Arriving:       0       Departing:      0
<DESTINATIONS>
     [email protected]
<SERVER>
     /usr/craysoft/nqe/bin/pipeclient CRI_DS

<ACCESS>
     Route: Unrestricted            Users: Unrestricted

<CUMULATIVE TIME>
     System Time:    0.00 secs       User Time:      0.00 secs

<ATTRIBUTES>
     C90
     chemdept
     nastran

Field 

Description

Initial heading 

The queue type (BATCH or PIPE), the name of the queue, and the NQS server at which it is located.

Status 

The current status of the queue can be one of the following:

Status

Description

ENABLED

Requests can be accepted into the queue.

DISABLED

Requests cannot be accepted, although NQS is running at the NQS server. The NQS operator has disabled the queue.

LOADED

Requests cannot be accepted, although NQS is running at the NQS server. The queue is a loadonly queue that has reached its request limit. When a request leaves the queue, it will return to an ENABLED status.

CLOSED

Requests cannot be accepted because NQS is not running at the NQS server.

Priority 

Interqueue priority. Determines the order in which NQS looks at the queues for work.

ENTRIES 

The total number of requests in the queue and the number of requests in the following states: running, holding, queued, arriving, waiting, and exiting.

DESTINATIONS 

A list of the destination queues to which requests in this queue may be sent. The order of the names in this chapter is the order in which NQS considers the queues for forwarding a request. If no destinations are listed, the queue is a destination-selection queue used for load balancing.

SERVER 

The name of the NQS pipeclient process used to handle routing of requests into and out of this pipe queue. The string CRI_DS indicates that the queue is a destination-selection queue used for load balancing.

ACCESS 

Indicates any restrictions on requests entering the queue.

The Route: subheading can have two values, as follows:

Unrestricted

A request can be submitted to this queue directly.

Pipeonly

A request can enter this queue only from another pipe queue. You cannot submit a request directly to this queue.

The Users: subheading indicates whether any user or group restrictions exist for this queue. The subheading can have two values, as follows:

Unrestricted

Any user's request can enter the queue.

Restricted

Requests from only specified users and groups can enter the queue. Look under the <ACCESS> heading for a list of the valid user names and user groups whose requests can enter the queue.

CUMULATIVE TIME 

The system and user time accumulated by all requests in the queue since NQS was initiated.

ATTRIBUTES 

A list of queue attributes. Queue attributes are strings supplied by the qmgr set attribute command. The attributes determine which requests are accepted into this queue. If a request specifies an attribute that the queue does not have, that request is not accepted. If a queue has no attributes, this field is not displayed and the queue will accept requests with any list of attributes.

Batch Queue Details

The following example shows a detailed display of an NQS batch queue called nqebatch:

% cqstatl -f nqebatch
-------------------------------------
NQS BATCH QUEUE: [email protected]        Status:          ENABLED/INACTIVE
-------------------------------------
                                                Priority:       30
<ENTRIES>
        Total:          11
        Running:        2       Queued:         9       Waiting:        0
        Holding:        0       Arriving:       0       Exiting:        0
<RUN LIMITS>
        Queue:          5       User: unspecified       Group: unspecified
<COMPLEX MEMBERSHIP>
<LOCAL SCHEDULER EXTENSIONS>
        Miser Queue: unspecified      Scheduling Window: 0:0.0
<RESOURCE USAGE>
                                   LIMIT                ALLOCATED
        Memory Size             unspecified (unlimited)   524288kw 0kb
        Quick File Space        unspecified (unlimited)        0kw 0kb
        MPP Processor Elements  unspecified (unlimited)        0   0
<RESOURCE LIMITS>
                                PER-PROCESS             PER-REQUEST
        type a Tape Drives                              unspecified (0)
        type b Tape Drives                              unspecified (0)
        type c Tape Drives                              unspecified (0)
        type d Tape Drives                              unspecified (0)
        type e Tape Drives                              unspecified (0)
        type f Tape Drives                              unspecified (0)
        type g Tape Drives                              unspecified (0)
        type h Tape Drives
        Core File Size          unspecified (256mw)
        Data Size               unspecified (256mw)
        Permanent File Space    unspecified (100mb)     unspecified (0b)
        Memory Size             unspecified (256mw)     unspecified (256mw)
        Nice Increment                 0
        Quick File Space        unspecified (0b)        unspecified (0b)
        Stack Size              unspecified (256mw)
        CPU Time Limit          unspecified (720000sec)   unspecified (720000sec)
        Temporary File Space    unspecified (0b)        unspecified (0b)
        Working Set Limit       unspecified (256mw)
        MPP Processor Elements                          unspecified (0)
        MPP Time Limit          unspecified (10sec)     unspecified (10sec)
        Shared Memory Limit                             unspecified (0mw)
        Shared Memory Segments                          unspecified (0)
        MPP Memory Size         unspecified (256mw)     unspecified (256mw)
<ACCESS>
        Route: Unrestricted            Users: Unrestricted
<CUMULATIVE TIME>
        System Time:    605.70 secs     User Time:      764.03 secs

See “Pipe Queue Details”, for a description of most of the fields in this display. The batch queue display does not contain the DESTINATIONS field.

This display contains the following four fields that the pipe queue display does not contain:

Field 

Description

RUN LIMITS 

The limit on the maximum number of concurrently executing requests for the entire queue, for one user, and for one user group.

COMPLEX MEMBERSHIP 

The names of the queue complexes of which this queue is a member.

LOCAL SCHEDULER EXTENSIONS 

Information about any local scheduling extensions that have been enabled for the queue; typically, the name of a Miser scheduler queue and the defined scheduling window.

RESOURCE USAGE 

The potential maximum and the cumulative usage of resources by requests currently executing in the queue. The default values are displayed in parentheses.

RESOURCE LIMITS 

The maximum per-process and per-request resource values that can be requested by a request to enter the queue. The default values are displayed in parentheses.

Displaying Batch Queue Limits

To display a list of the limits that the NQE administrator defined for all NQS batch queues, you can use either the cqstatl -l command or the qstat -l command (lowercase L); for example:

cqstatl -l

The following screen shows an example of this summary display (pendulum is the NQS server). The last line in the display shows the global limits for the NQS server:

% cqstatl -l
----------------------------
NQS BATCH QUEUE LIMITS
----------------------------
QUEUE NAME                RUN       MEMORY          QUICKFL     USR GRP
----------------------- --- --- ------- ------- ------- ------- --- ---
nqebatch                  5/0        --/0            --/0        --  --
----------------------- --- --- ------- ------- ------- ------- --- ---
pendulum                  5/0        **/0            --/0         2  --
----------------------- --- --- ------- ------- ------- ------- --- ---

Some columns in this display have two entries separated by /. The first entry is the limit set for the queue. The second entry is the current use. The -- symbols mean that no limit has been specified explicitly for the queue. The ** symbols mean that the item is unlimited.

The columns in this display have the following meanings:

Column name

Description

QUEUE NAME

The name of the batch queue.

RUN

The number of requests that can execute simultaneously (the queue run limit), followed by the number that are currently executing.

MEMORY

The maximum amount of memory that all requests in the queue can use at one time, followed by the amount currently being used. A value of 0 for the first entry means that the amount of memory available to this queue is unlimited. Values are expressed in units of 1024 words.

QUICKFL

The maximum amount of quickfile secondary data segments (SDS) space that a request can use, followed by the amount currently being used. Values are expressed in units of 1024 words.

USR

The maximum number of requests that one user can have executing in the queue at any one time (the queue user run limit).

GRP

The maximum number of requests that one user group can have executing in the queue at any one time (the queue group run limit).

Monitoring Remote Queues

NQE can route your requests to NQS queues at a server other than your NQS server (as defined by NQS_SERVER). You can display information about remote NQS queues by doing one of the following:

  • Use either the cqstatl -h command or the qstat -h command and supply the network host name of an NQS server. For example, the following command displays a summary status of all NQS queues at a server called hot:

    cqstatl -h hot

  • Include the host when you specify a queue, as follows:

    queue@target_host

    For example, the following command displays full details about the NQS queue single at the NQS server hot:

    cqstatl -f [email protected]

If password validation is in force, you must include the cqstatl -P option or set the NQS_PASSWORD_NEEDED environment variable to ensure that you are prompted for a password.

If both password and file validation are in force, you do not have to set the environment variable or specify the cqstatl -P command. Your validation files are checked as described in “Validation File Examples” in Chapter 2.

If validation files are checked, the cqstatl or qstat command is successful only if your user name is included in a validation file at the NQS server. For more information on file validation, see “File Validation” in Chapter 2.

If the UNICOS MLS feature or the UNICOS/mk security enhancements are enabled on your system, you cannot display information from a remote host if the execution host has a workstation access list (WAL) entry for the host of origin that restricts your access to NQS services.