Flux in 10 Easy Steps

By February 23, 2016

1. Get Duo

You must use Duo authentication to log on to Flux. For more details see http://its.umich.edu/two-factor-authentication.

2. Get a Flux user account

You need to have a user account to use Flux. Apply for one using this form.

3. Get access to an allocation

If you don’t have one already, an allocation must be purchased before you can use Flux. If you don’t have access, request a Flux allocation by emailing hpc-support@umich.edu, and one of our support staff will help you get set up. For more, see our Allocation pages. (LSA users have access to a public allocation.)

4. Get an SSH client

You need a terminal emulator to log into Flux. This video will help teach you some basic Linux navigation commands if needed.

If you are trying to log in from off campus, or using an unauthenticated wireless network such as MGuest, you have a couple of options:

    • Install VPN software on your computer
    • First ssh to login.itd.umich.edu, then ssh to flux-login.arc-ts.umich.edu from there.

Here’s what a login looks like using a terminal emulator:

Mac using terminal: Open terminal

Type: ssh -l uniqname flux-login.arc-ts.umich.edu [replacing your uniqname in the command]

Windows using PuTTY (http://www.chiark.greenend.org.uk/~sgtatham/putty/)

Launch Putty and enter flux-login.arc-ts.umich.edu as the host name then click open.

putty

For both Mac and Windows:

At the “Enter a passcode or select one of the following options:” prompt, type the number of your preferred choice for Duo authentication.

login-mac

5. Get files

The server you can use to get files onto your home directory on Flux is flux-xfer.arc-ts.umich.edu. Here’s what it looks like on

Cyberduck on a Mac (Video: http://tinyurl.com/flux-cyberduck)

  1. Open Cyberduck and click the Open connection button.

  2. Set the “Server:” to: be flux-xfer.arc-ts.umich.edu

  3. Set your Username: to be your uniqname.

  4. Enter your Kerberos password.

  5. Click connect.

    cyberduck

  1. Drag and drop files between the two systems.  Click the Disconnect button when completed.

An alternative for Windows is WinSCP.exe and you can get this from the U-M Blue Disc site. Here’s what transferring files or directories looks like from the command line (Linux or Mac):

% scp localfile uniqname@flux-xfer.arc-ts.umich.edu:remotefile (copy a file)

or

% scp -r localdir  uniqname@flux-xfer.arc-ts.umich.edu:remotedir    (copy an entire directory)

Globus is a great alternative especially for large data transfers. Learn more at the Globus web site or this how-to video.

6. Get an editor

nano is an easy to use editor available to edit your files on Flux.

7. Get a PBS batch script

The cluster requires some information to wrap your code so that it can submit your job for execution. Wrapping these jobs allows the Flux scheduler to fits jobs to available processing on the system. This means you must make a batch file to submit your code.

  1. Video: PBS Basics.

    Sample batch file, named sample.pbs:

#!/bin/sh
####  PBS preamble

#PBS -N sample_job

# Change "bjensen" to your uniqname:
#PBS -M bjensen@umich.edu
#PBS -m abe

# Change the number of cores (ppn=1), amount of memory, and walltime:
#PBS -l nodes=1:ppn=1,mem=2000mb,walltime=01:00:00
#PBS -j oe
#PBS -V

# Change "example_flux" to the name of your Flux allocation:
#PBS -A example_flux
#PBS -q flux
#PBS -l qos=flux

####  End PBS preamble

#  Show list of CPUs you ran on, if you're running under PBS
if [ -n "$PBS_NODEFILE" ]; then cat $PBS_NODEFILE; fi

#  Change to the directory you submitted from
if [ -n "$PBS_O_WORKDIR" ]; then cd $PBS_O_WORKDIR; fi

#  Put your job commands here:
echo "Hello, world"

8. Get modules

Flux makes software available for use by packaging them in modules.  You must load the modules that you need for your job before you submit it. The commands are:

module avail
module list
module load modulename    (eg module load R)
module unload modulename    (eg module unload R)

9. Get a job

You can submit your job and check its status in the queueing system by using the commands below.

Submitting your Job:

qsub filename.pbs    (eg qsub sample.pbs … outputs jobid on successful submission)

Checking the status of your Job:

For a single job: qstat jobid     OR  
checkjob  jobid   (just the numeric portion)

To see all of your jobs: qstat -u uniqname  OR showq -w user=uniqname

Deleting your job:

qdel  jobid

If a job doesn’t start in within 30 minutes, send an email with a copy of your PBS script and the job number to flux-support@umich.edu.  Do not delete the job.

10. Get output

When your job is completed, you will receive an email similar to this one:

PBS Job Id: 11800755.flux.arc-ts.umich.edu
Job Name:   sample_job
Exec host:  flux5301/0
Exit_status=0
resources_used.cput=00:00:03
resources_used.mem=768124kb
resources_used.vmem=770420kb
resources_used.walltime=00:00:13

Exit_status=0 means “OK.” Anything else indicates an error occurred when running the PBS script — check the PBS output and error files to find out what the problem is.

resources_used.vmem=770420kb means your job used a total of 770 MB of memory. This may be used to revise your memory estimate for future submissions.

resources_used.walltime=00:00:13 means your job took 13 seconds to execute. This may be used to revise your walltime estimate for future submissions.

If you have specified the -j oe option in your PBS script, all script output and error message output will appear in a file named jobname.ojobid in your submission directory after the job completes.  In the above sample, the output generated by your batch script (here “Hello, world”) will be placed into the file sample_job.o11800755 .