Launcher is a utility developed at the Texas Advanced Computing Center (TACC) to allow running of many serial or multi-threaded applications as a single, large batch submission.

Using Launcher

Launcher is available on Great Lakes as a software module. To enable it, load it as you would any other.

module load launcher

The module will add the installation directory to your $PATH as well as set the following environment variables:

  • LAUNCHER_DIR – installation directory. Only necessary for Launcher’s internal use.
  • LAUNCHER_PLUGIN_DIR – plugin directory for integration with various resource managers. Only necessary for Launcher’s internal use.
  • LAUNCHER_RMI – resource manager integration plugin to use.
  • LAUNCHER_SCHED – scheduling method to use. Can be overridden by the user.

For more information on software modules, see the module documentation here.

Submitting a Launcher Job

*Please note: Launcher is not intended for MPI jobs.* 

We will ultimately be using a job submission script as normal. For information on job submission syntax, see the Slurm User Guide.

First, set up your job file. This should be a plain text file with the commands that you need to execute, one command per line. For example:

python myscript.py input1
python myscript.py input2
python myscript.py input3
...

Assume the name of this file is $HOME/example-job-file. Then, in your job submission script, you would need to set the variable LAUNCHER_JOB_FILE to this value:

export LAUNCHER_JOB_FILE=$HOME/example-job-script

The last line of your submission script would be to call Launcher’s paramrun exectuable:

paramrun

This executable has been added to your path by the module. Tying everything together, your
job submission script for this run would look like this:

#!/bin/bash
#SBATCH --job-name=Parametric
#SBATCH --mail-user=uniqname@umich.edu
#SBATCH --mail-type=BEGIN,END
#SBATCH --cpus-per-task=1
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=2
#SBATCH --mem-per-cpu=4000m 
#SBATCH --time=10:00
#SBATCH --account=example
#SBATCH --partition=standard
#SBATCH --output=/home/%u/%x-%j.log
export LAUNCHER_JOB_FILE=$HOME/example-job-file 

paramrun

A Note About Job Layouts

Launcher tries to match the geometry of your resource request, and having an uneven geometry could leave some cores idle, causing your job to run unnecessarily slow.

For more information on job layouts, see section 1.2 of the Great Lakes user guide.

A Note about running threaded applications with Launcher

All of the processors assigned to a Launcher job are part of the same control group on their computer. If you run threaded applications from inside Launcher, you should be aware that they may contend with each other for processors, as every started program will be able to see all the available processors. It is recommended that Launcher be used with serial programs (one processor) or that threaded programs be set to use only a single thread.

Scheduling Methods

From the TACC documentation:


The launcher has three available behaviors for scheduling jobs, available by setting the
environment variable $LAUNCHER_SCHED: (descriptions below assume k = task, p = num. procs,
n = num. jobs)

  • dynamic (default) – each task k executes the first available unclaimed line
  • interleaved – each task k executes every (k+p)th line
  • block – each task k executes lines [ k(n/p) + 1, (k+1)(n/p) ]

Please note that only the “block” and “interleaved” methods are currently supported by ARC. As such, $LAUNCHER_SCHED is set to “block” by default in the module. Feel free to set this to “interleaved” if you think it would be better for your workflow, but “dynamic” will not behave in the way that you would expect. We plan to support this feature in the future to offer flexibility.

To choose a different scheduling method, you could set $LAUNCHER_SCHED in your job script sometime before calling paramrun:

export LAUNCHER_SCHED="interleaved"

Referencing Launcher

The TACC documentation requests that users cite it in published work.


If you are using Launcher, please remember to make a reference to it when publishing results. The file paper/paper.bib contains the BibTeX-formatted citation list. Please reference entry Wilson:2014:LSF:2616498.2616533 (i.e., in LaTeX: \cite{Wilson:2014:LSF:2616498.2616534}).


To access the citation list from within the ARC module, you can copy it to a directory of your choice ($HOME, for example):

$ cp $LAUNCHER_DIR/paper/paper.bib ~/

Some real text.

Examples

Working examples showing how to use Launcher are available in /sw/examples/launcher/. There are two example directories at that location. To use either, start by loading the Launcher module.

$ module load launcher

Example 1

This is a simple example that writes many “Hello, world!” messages in parallel.

  1. Copy the example directory to your home directory and cd into it.
    $ cp -R /sw/examples/launcher/example1 ~/
    $ cd ~/example1
    
  2. Edit the file helloworldshort.sbat. You only need to add a working Slurm account to the SBATCH script (the --account option).
  3. Submit the job.
    $ sbatch helloworldshort.sbat

This job should finish quickly, and an output file should appear in your current directory with the results printed.

Example 2

This is a slightly more complicated example that perhaps demonstrates more realistic usage.

  1. Copy the example directory to your home directory and cd into it.
    $ cp -R /sw/examples/launcher/example2 ~
    $ cd ~/example2
  2. Run the create-jobfile.sh script. It is important to run this script inside of your copied example directory.
    $ ./create-jobfile.sh
  3. Edit the file process-data.sbat. You only need to add a working Slurm account to the PBS script (the --account option).
  4. Submit the job.
    $ sbatch process-data.sbat

This job should also finish relatively quickly.