Chapter 8. Using Job Dependency

NQE job dependency lets you specify an interdependency among events in shell scripts or NQS requests. For example, you can ensure that one request runs after another so that the second can use output from the first. This chapter discusses job dependency. The following topics are covered:

Using Job Dependency

Job dependency lets a client script or NQS request post, read, and wait for events. Events are associated with the following information:

  • Group. Each event is a member of a group. A group of requests or shell scripts can communicate by using events. The group is identified by a name that must be globally unique. Group names are limited to 1023 characters. They must begin with a letter and contain accepted C identifier characters.

  • Name. Each event has a name that must be unique within its group. Names are limited to 1023 characters.

  • Value. Optionally, events can have values. Client commands can set this value when posting an event and can obtain the value when reading an event. Values are limited to 4095 characters.

  • Other information, such as the NQS originating user name, host, the time of events, and so on.

exit events are unique events for tracking NQS requests. NQS sends an event when a request completes and has the NQE_GROUP environment variable set. The event group name is obtained from NQE_GROUP. The event name has the form EXIT_$QSUB_REQNAME. The name is derived from the prefix EXIT_ (that NQS automatically adds) and from the request's name as specified (if you are using the command line interface, the cqsub -r or qsub -r option is used to specify a request's name). This naming method lets the event be tested without using the request ID. The originator of the request owns its exit event.

The following criteria determine access to events:

  • The user name that posts an event owns the event.

  • Anyone who knows the group name can read an event.

  • Only root can read or delete all events.

Using cevent

To specify events and their associated actions, use the cevent(1) command. The cevent command options let you post (-p), read (-r), clear (-c), and list (-l) events.

To post an event with the name TAPE_HERE under the group mygroup, use the following command:

cevent -p -g mygroup TAPE_HERE

You can then refer to the name TAPE_HERE when you read events.

The following csh command reads the event TAPE_A and stores it in a variable:

gottape=`cevent -g mygroup -r TAPE_A'

The following ksh command reads the event TAPE_A and stores it in a variable:

set -A gottape $(cevent -g mygroup -r TAPE_A)

When you read events, you can specify a time interval to wait for the event to occur (the -w option) and the time interval between queries to the Network Load Balancer (NLB) database to see whether the event has occurred (the -z option). The NLB stores information about events and reports the information when it is queried.

The cevent -l option lists event values to standard output (stdout).

To list the events in a group named mygroup, use the following command:

% cevent -g mygroup -l

Time:                     Group:      Name:      Value:
Mon Jan 26 10:14:14 1998  mygroup     n2         specified
Fri Jan 23 08:42:30 1998  mygroup     n1         <NONE>

The output tells you that you can read two events, one named n2 and the other named n1. The event named n1 was created without a value. The event named n2 has a value. It was created by using the following command:

cevent -g mygroup -p n2=specified

The cevent -d option specifies a delimiter between the values. You can use delimiters to parse the output from cevent listings.

The default delimiter for -r commands is the : symbol. If you do not specify a delimiter, your output will look like the following:

% cevent -g mygroup -r n1 n2

If you specified the # symbol as a delimiter, your command and output would look like the following:

% cevent -g mygroup -d '#' -r n1 n2

You also can use the -d option to specify a delimiter between the events when you list them by using the -l option. The default delimiter is a space. The following example lists the events in the group mygroup and specifies that the delimiter is a # symbol:

% cevent -g mygroup -d '#' -l
Fri Jan 23 08:42:30 1998#mygroup#n1#<NONE>
Mon Jan 26 10:14:14 1998#mygroup#n2#specified

When you are finished using the events in a group, you can clear them from the database. To clear the events in a group named mygroup, use the following command:

cevent -c -g mygroup

For a summary of cevent options, see the cevent(1) man page.

Job Dependency Example

In the following example, the user wants to submit the request JOBB after the request JOBA successfully completes execution. The example uses ksh syntax.


#set NLB_SERVER (where data is stored) to host pendulum
export NLB_SERVER=pendulum

#set NQS_SERVER (where request is run) to host latte
export NQS_SERVER=latte


# Ensure that no events are outstanding
cevent -c
# Submit request JOBA
cqsub -r JOBA -q nqebatch <<EOF
ls -al

# Wait for completion of JOBA
done_A=`cevent -r -w 10000 EXIT_JOBA`
if (( $? != 0 )); then
echo "No exit event posted for JOBA"
exit 1
echo "JOBA completed with exit status $done_A"

# Submit request JOBB and set its exit status to 23
cqsub -r JOBB -q nqebatch <<EOF
ls -al
exit 23

# Wait for completion of JOBB
done_B=`cevent -r -w 10000 EXIT_JOBB`
if (( $? != 0 )); then
echo "No exit event posted for JOBB"
exit 1
echo "JOBB completed with exit status $done_B"

# List out events
echo "All events posted for group $NQE_GROUP:"
cevent -l

# Clear events
echo "Deleting events..."
cevent -c

exit 0

First the script sets up the user environment and clears all events that might be outstanding.

Next, JOBA is submitted. NQS exports the NQE_GROUP environment variable automatically, so you do not have to use the cqsub -x or qsub -x command.

The next cevent command waits for JOBA to complete. The event name EXIT_JOB1 begins with the prefix EXIT_, which NQS automatically adds, followed by the name you gave the request. The -w option specifies that you want to wait no less than 10,000 seconds for the event to occur.

The next lines accommodate possible errors, such as a client command timing out. When the exit status of the request returns, it is recorded to stdout, and the script continues if the exit status is correct (that is, the exit status is 0, indicating normal completion).

The final lines of the script submit the request JOBB and clear the events from this script.

The output from the script would be as follows:

Request 49.latte submitted to queue: nqebatch.
JOBA completed with exit status 0
Request 50.latte submitted to queue: nqebatch.
JOBB completed with exit status 23
All events posted for group A_THEN_B:
Time:                     Group:         Name:          Value:
Mon Mar 16 15:01:16 1998  A_THEN_B       EXIT_JOBB      23
Mon Mar 16 15:00:44 1998  A_THEN_B       EXIT_JOBA      0
Deleting events...