Description

SIMPSON is a software package for simulation of multi-spin and multi-dimensional NMR experiments and for specification and visualization of anisotropic spin interactions in biomolecular NMR.

Vendor website: http://nmr.au.dk/software/simpson/

Documentation (published paper): http://bionmr.chem.au.dk/download/simpson/documentation/simpson-paper.pdf

Accessing SIMPSON

SIMPSON is part of the LSA contributed software library. To use it, you must first load the lsa module, additional modules which SIMPSON requires to work (MKL, FFTW, and NFFT), and, finally SIMPSON itself:

$ module load mkl
$ module load fftw
$ module load lsa nfft
$ module load lsa simpson

IMPORTANT NOTE: Do not put the command “module load lsa simpson” into your ~/privatemodules/default file in order to have SIMPSON automatically loaded for you each time you log in to Flux. SIMPSON contains a special version of the Tcl interpreter that causes problems with the “module” command (which also uses Tcl). Due to this conflict, if you run any “module” commands after loading SIMPSON, you will see the warning message

init.c(573):WARN:162: Cannot initialize TCLX modules using extended commands might fail

This warning message should not cause any problems, but you should only load SIMPSON just before you submit jobs that will use SIMPSON, and you should avoid loading any modules after loading SIMPSON, if possible.

Running SIMPSON interactively

As always, please be sure to not run long or intensive computations on the cluster login nodes, as this can cause problems for other users. For anything except for project file setup and quick tests, please run SIMPSON from PBS as described below, or run an interactive job on the Flux compute nodes by using the “qsub -I” command as described in the “How to submit an interactive job” section at http://cac.engin.umich.edu/resources/systems/flux/pbs

To run SIMPSON on the Flux login nodes, use the command:

$ env SIMPSON_NUM_CORES=4 OMP_NUM_THREADS=4 simpson my-simpson-file.in

You should replace “my-simpson-file.in” above with the name of your SIMPSON input file. SIMPSON_NUM_CORES and OMP_NUM_THREADS control how many cores SIMPSON will use on the Flux login nodes; these values should always be the same as each other. Note that while the Flux login nodes each currently have 12 cores, in order to avoid causing problems for other users of the login nodes, you should never use more than a few cores; submit an interactive job (see above) if you require more than a few cores.

Running SIMPSON from PBS

Create the following PBS file, naming the file test.pbs, and changing the following things:

  • change “nodes=8,tpn=4” in the “#PBS -l” line. In “nodes=X,tpn=Y” X should be the total number of cores you want for your entire job (in this context, “nodes” means “cores”), and Y should be the number of cores you want on each of the compute nodes (not to be confused with the different use of the word “node” in “nodes=X”. Y must be evenly divisible into X. In this example, “nodes=8,tpn=4” means “give me a total of 8 cores arranged with exactly 4 cores per node”, that is, two nodes with four cores each.IMPORTANT NOTES: You must use the syntax “nodes=X,tpn=Y” in the “#PBS -l” line to specify the total number of cores you want your job to use (X) and how many cores you want on each compute node (Y). SIMPSON can wind up being very slow or not working at all if you use the more common “procs=N” or “nodes=N:ppn=M” syntaxes (this is because SIMPSON uses both OpenMP in combination with OpenMPI). Also note that “nodes=X” and “tpn=Y” are separated by a comma rather than a colon. Finally, if you want to use only a single node, make X and Y the same number (e.g., “nodes=8,tpn=8” will give the job 8 cores all on a single node) and remove “mpirun” from the last command in the PBS script.
  • change the values of SIMPSON_NUM_CORES=4 OMP_NUM_THREADS=4 to be the same as the the number of cores you requested per compute node (Y, above). Whenever you change the number of cores used by your job, it is very important to update SIMPSON_NUM_CORES and OMP_NUM_THREADS too.
  • change the memory and walltime in the “#PBS -l” if needed.
  • change example_flux to the name of your Flux allocation in the “#PBS -A” line
  • change uniqname@umich.edu to your email address in the “#PBS -M” line
  • change “my-simpson-file.in” to be the name of your SIMPSON input file in the last line.
#!/bin/sh

####  PBS preamble:

#PBS -N simpson_test
#PBS -M uniqname@umich.edu
#PBS -m abe 

# "nodes=8,tpn=4" means "8 cores total, with exactly 4 cores per node"
#PBS -l nodes=8,tpn=4,pmem=3800mb,walltime=1:00:00
#PBS -j oe 
#PBS -V

#PBS -A example_flux
#PBS -l qos=flux
#PBS -q flux

####  End PBS preamble

#  Show list of CPUs you ran on, if you're running under PBS
if [ -n "$PBS_NODEFILE" ]; then cat $PBS_NODEFILE; fi

#  Change to the directory you submitted from
if [ -n "$PBS_O_WORKDIR" ]; then cd $PBS_O_WORKDIR; fi

#  Put your job commands after this line

#    IMPORTANT! Make sure the values for SIMPSON_NUM_CORES and
#    OMP_NUM_THREADS are the same as you specified for tpn= above

env SIMPSON_NUM_CORES=4 OMP_NUM_THREADS=4 mpirun simpson my-simpson-file.in

To submit the job, run

$ qsub test.pbs

Additional information

For any Flux-specific assistance running SIMPSON, contact flux-support@umich.edu.