Monitoring Jobs With qstat

To monitor jobs, type one of the following commands, guided by information that is detailed in the following sections:

qstat
qstat -f
qstat -ext

qstat with no options provides an overview of submitted jobs only. qstat -f includes information about the currently configured queues in addition. qstat -ext contains details such as up-to-date job usage and tickets assigned to a job.

In the first form, a header line indicates the meaning of the columns. The purpose of most of the columns should be self-explanatory. The state column, however, contains single character codes with the following meaning: r for running, s for suspended, q for queued, and w for waiting. See the qstat(1) man page for a detailed explanation of the qstat output format.

The second form is divided into two sections. The first section displays the status of all available queues. The second section, titled PENDING JOBS, shows the status of the sge_qmaster job spool area. The first line of the queue section defines the meaning of the columns with respect to the queues that are listed. The queues are separated by horizontal lines. If jobs run in a queue, the job names appear below the associated queue in the same format as in the qstat command in its first form. The pending jobs in the second output section are also listed as in qstat`s first form.

The columns of the queue description provide the following information:

• qtype – Queue type. Queue type is either B (batch) or I (interactive).

• used/free – Count of used and free job slots in the queue.

• states – State of the queue. See the qstat(1) man page for detailed information about queue states.

The qstat(1) man page contains a more detailed description of the qstat output format.

In the third form, the usage and ticket values assigned to a job are shown in the following columns:

• cpu/mem/io – Currently accumulated CPU, memory, and I/O usage.

• tckts/ovrts/otckt/ftckt/stckt – These values are as follows:

  tckts – Total number of tickets assigned to the job

  ovrts – Override tickets assigned through qalter -ot

  otckt – Tickets assigned through the override policy

  ftckt – Tickets assigned through the functional policy

  stckt – Tickets assigned through the share-based policy

In addition, the deadline initiation time is displayed in the column deadline, if applicable. The share column shows the current resource share that each job has with respect to the usage generated by all jobs in the cluster. See the qstat(1) man page for further details.

Various additional options to the qstat command enhance the functionality. Use the -r option to display the resource requirements of submitted jobs. Furthermore, the output can be restricted to a certain user or to a specific queue. You can use the -l option to specify resource requirements, as described in Defining Resource Requirements, for the qsub command. If resource requirements are used, only those queues, and the jobs that are running in those queues, are displayed that match the resource requirement specified by qstat.

The qstat command has been enhanced so that the administrator and the user may define files that can contain useful options. See the sge_qstat(5) man page. A cluster-wide sge_qstat file may be placed under $xxQS_NAME_Sxx_ROOT/$xxQS_NAME_Sxx_CELL/common/sge_qstat. The user private file is processed under the location $HOME/.sge_qstat. The home directory request file has the highest precedence, then the cluster global file. You can use the command line to override the flags contained in a file.

Example 4–1 and Example 4–2 show examples of output from the qstat and qstat -f commands.

Example 4–1 Example of qstat -f Output

queuename                  qtype  used/free   load_avg   arch         states     

dq                         BIP    0/1         99.99      sun4         au     

durin.q                    BIP    2/2         0.36       sun4          
  231     0    hydra              craig       r          07/13/96     20:27:15    MASTER
  232     0    compile            penny       r          07/13/96     20:30:40    MASTER

dwain.q                    BIP    3/3         0.36       sun4          
  230     0    blackhole          don         r          07/13/96     20:26:10    MASTER
  233     0    mac                elaine      r          07/13/96     20:30:40    MASTER
  234     0    golf               shannon     r          07/13/96     20:31:44    MASTER

fq                         BIP    0/3         0.36       sun4          

################################################################################

- PENDING JOBS - PENDING JOBS - PENDING JOBS - PENDING JOBS - PENDING JOBS -

################################################################################

  236     5    word               elaine      qw         07/13/96     20:32:07

  235     0    andrun             penny       qw         07/13/96     20:31:43

Example 4–2 Example of qstat Output

job-ID   prior   name         user      state   submit/start at     queue      function
231      0       hydra        craig     r       07/13/96            durin.q    MASTER 
                                                20:27:15
232      0       compile      penny     r       07/13/96            durin.q    MASTER 
                                                20:30:40
230      0       blackhole    don       r       07/13/96            dwain.q    MASTER 
                                                20:26:10
233      0       mac          elaine    r       07/13/96            dwain.q    MASTER 
                                                20:30:40
234      0       golf         shannon   r       07/13/96            dwain.q    MASTER 
                                                20:31:44
236      5       word         elaine    qw      07/13/96           
                                                20:32:07
235      0       andrun       penny     qw      07/13/96 20:31:43