Interpreting Torque return codes

When you request in your PBS script that Torque send you mail when a job completes, it will include information about the resources used and what the status was when the job completed. The status reported is the status of the last command run in the PBS script. Here is a table of the most commonly encountered return codes and their interpretation. The codes are listed with positive return codes first, then the negative return codes. Below the table, you will find an example of the information included in the e-mail, with the return code line highlighted to help you find it in your own e-mail.

Return code Interpretation
0 No errors reported
1 The last command the job ran was unsuccessful or job exited with error (SIGHUP)
2 The last command the job ran was unsuccessful or job exited with error (SIGINT)
3 The last command the job ran was unsuccessful or job exited with error (SIGQUIT)
11 Job exited with segmentation fault (SIGSEGV)
126 Command invoked cannot execute
127 Command not found
128 Invalid argument to exit
137 Job stopped by user (SIGKILL) or for exceeding a memory limit (stopped by the kernel)
143 Job stopped by user (SIGTERM)
265 Job stopped by user (SIGKILL)
271 Job stopped by user (SIGTERM)
-1 Job execution failed, before files, no retry
-2 Job execution failed, after files, no retry
-3 Job execution failed, do retry
-4 Job aborted on MOM initialization
-5 Job aborted on MOM init, chkpt, no migrate
-6 Job aborted on MOM init, chkpt, ok migrate
-7 Job restart failed
-8 Exec() of user command failed
-9 Could not create/open stdout stderr files
-10 Job exceeded a memory limit (stopped by the resource manager)
-11 Job exceeded a walltime limit
-12 Job exceeded a CPU time limit

Here is an example of the information sent upon job completion. The line that contains the return code is highlighted in blue below, and is the one that begins with the text Exit_status.

PBS Job Id: 25639679.nyx.arc-ts.umich.edu
Job Name:   pbs_hello
Exec host:  nyx6219/0
Execution terminated
Exit_status=0
resources_used.cput=00:00:00
resources_used.energy_used=0
resources_used.mem=5328kb
resources_used.vmem=0kb
resources_used.walltime=00:00:10
req_information.task_count.0=1
req_information.lprocs.0=1
req_information.memory.0=786432kb
req_information.thread_usage_policy.0=allowthreads
req_information.hostlist.0=nyx6219:ppn=1
req_information.task_usage.0.task.0={"task":{"cpu_list":"0","mem_list":"0","cores":0,"threads":1,"host":"nyx6219"}}
Error_Path: flux-build.arc-ts.umich.edu:/home/bennet/hello/pbs_hello.o25639679
Output_Path: flux-build.arc-ts.umich.edu:/home/bennet/hello/pbs_hello.o25639679