How to Use Job Arrays

Job arrays in PBS are an easy way to submit multiple similar jobs. The only difference in them is the array index, which you can use in your PBS script to run each task with a different set of parameters, load different data files, or any other operation that requires a unique index.

To submit a PBS job with 10 elements, use the -t option in your PBS script.

#PBS -t 1-10

If your assigned job number is 5432 the elements in the array will be 5432[1], 5432[2], 5432[3], …, 5432[10]. In each script the environment variable PBS_ARRAYID is set to the numbers 1 through 10.

Note that each array element will appear as a separate job in the queue, and the normal scheduling rules apply to each element.

You can also limit how many of the array elements are running simultaneously by specifying how many job slots you want. So, for example,

#PBS -t 1-100%5

submits an array with 100 jobs in it, but the system will only ever have 5 running at one time. This can be especially useful when you are sharing the processor pool among members of a group.

You can delete individual array elements from the queue by specifying the element number.

qdel 5432[4]

You can also delete the entire array by using the base job number as the argument to the qdel command.

qdel 5432[]

which will delete all remaining array elements.

To view the status of the entire job array, run qstat with the -t option.

qstat -t 5432[]

Suppose you want to run the same program 10 times, each time using a different input file. This is most easily done by naming the input files with a number in it, as in file-1, file-2, etc. A sample PBS script that uses this to run the same executable with 10 different input files that you named with the appropriate names would look like:

#!/bin/bash
#PBS -N yourjobname
#PBS -l nodes=1,walltime=05:00:00
#PBS -S /bin/sh
#PBS -M uniqname@umich.edu
#PBS -t 1-10
#PBS -m abe
#PBS -A example_flux
#PBS -l qos=flux
#PBS -q flux
#PBS -j oe
#PBS -V

cd /path/to/my/program
./myprogram -input=file-${PBS_ARRAYID}

For each value specified with -t, it will run the script with that value substituted for PBS_ARRAYID, in this case that is file-1, file-2, etc.

For more information on job arrays, please see the Cluster Resources web page on them: http://docs.adaptivecomputing.com/torque/4-2-7/help.htm#topics/commands/qsub.htm#-t