Description

SPINEVOLUTION is a highly efficient computer program for the numerical simulation of NMR experiments and spin dynamics in general.
Software website: http://spinevolution.com/
Author website: http://web.mit.edu/fbml/cmr/griffin-group/SPINEVOLUTION/

Availability

The SPINEVOLUTION software is non-standard in a number of respects, and hence it is not suitable for being centrally installed on Flux. However, you can easily install the software in your home directory on Flux simply by following the instructions that come with the software. The following section summarizes the commands you need to run in order to install SPINEVOLUTION.

Installing

cd $HOME
wget http://web.mit.edu/fbml/cmr/griffin-group/SPINEVOLUTION/spinev/spinev-3.4.5-Linux-x86_64.tar.gz
tar zxf spinev-3.4.5-Linux-x86_64.tar.gz

In order to use SPINEVOLUTION, you must run the following commands the first time you setup the installation
export SPINEV=${HOME}/spinev-3.4.5-Linux-x86_64
export PATH=${SPINEV}:${PATH}
spinev $SPINEV/examples/csa/csa

That is all you need to do to install SPINEVOLUTION.

Running SPINEVOLUTION

You do not need to load any Flux software modules before submitting a job, since SPINEVOLUTION will run from your
home directory and does not depend upon any other software.

In your PBS script, after the PBS preamble, but before you invoke “spinev”, include the commands

$ export SPINEV=${HOME}/spinev-3.4.5-Linux-x86_64
$ export PATH=${SPINEV}:${PATH}

Include the appropriate command line options for spinev according to the instructions in the SPINEVOLUTION
documentation. In particular, note that our cluster uses PBS and a cluster configuration file (PBS_NODEFILE) is
supplied automatically by Flux when your job runs — you must not supply one. You also must not try to set up RSA
authentication (this is handled automatically by Flux) and you will not need to use ssh.

SPINEVOLUTION actually has three modes of operation: seria model, cluster mode
(multiple cores on multiple nodes), and split mode (multiple cores on a single node). Cluster mode (multiple cores on
multiple nodes) is the way we used SPINEVOLUTION in the example above — it is the most flexible and capable of handling
the largest sized computations. Split mode (multiple cores on a single node), however, is a little faster than cluster
mode, if the size of your computation will fit onto a single node (on Standard Flux, this means “16 cores or less and 64
GB or less” or “12 cores or less and 48 GB or less”; keeping the job under 12 cores and 48 GB will result in the
shortest wait time for your job to start).

Running SPINEVOLUTION in serial mode

#PBS -l nodes=1:ppn=1,pmem=4000mb,walltime=01:00:00
spinenv

Running SPINEVOLUTION in split mode

If you want to use the split mode of operation with SPINEVOLUTION on Flux, you will need to make two changes:

1. Change the resources line in the PBS preamble to be

#PBS -l nodes=1:ppn=4,pmem=4000mb,walltime=01:00:00

This requests 4 cores (ppn=4) all on a single node (nodes=1).

2. Change the spinev command to be

spinev  -split[number-cores-requested-in-ppn-in-PBS]

You will need to make adjustments to both lines above as appropriate for the job you are actually running, including
changing the actual number of cores (ppn=X), amount of memory, and walltime; and changing the name of the SPINEVOLUTION
input file and getting rid of the “-macro’$n’=3” option (unless you are running the r2 example).

ALSO NOTE: when running in split mode, the number immediately after “-split” must be the number of cores that you are
running on and should be the same as the number of cores you requested in the PBS resource line.

Running SPINEVOLUTION in cluster mode

#PBS -l nodes=:ppn=,pmem=4000mb,walltime=01:00:00
spinev  -cluster -clust0

The command runs SPINEVOLUTION with the input file in cluster mode (“-cluster”, which instructs spinev to use multiple cores across multiple nodes). The “-clust0” option forces what SPENVOLUTION calls “regular cluster mode” which is faster than “special mode” (“special mode” is useful if you are running on a collection of workstations some of which might get turned off or rebooted during the middle of the computation — special mode will prevent all of the work from being lost when this happens, but results in slower computation if no node fails).

Additional information

Note that this runs SPINEVOLUTION interactively on the Flux login node. Refer to the documentation on the SPINEVOLUTION website http://web.mit.edu/fbml/cmr/griffin-group/SPINEVOLUTION/ for what command line options you need to give to the “spinev” program, how to set up the SPINEVOLUTION input file, and other details. You should not do any significant computation on the Flux login node, as this can interfere with other users’ use of Flux (in fact, if you run “spinev” on the login nodes, the program will be automatically killed after a few minutes for this reason). To do any significant computation (anything more than making sure “spinev” starts correctly) you will need to submit a job that runs “spinev”.

IMPORTANT NOTE: SPINEVOLUTION is closed-source software that includes Digital Restrictions Management functionality in order to limit your ability to run it past an expiration date. The expiration date of the currently available version is January 1st, which means that the version of SPINEVOLUTION that you download an install today will only work for the next three weeks. According to the SPINEVOLUTION web site, the author of SPINEVOLUTION will make a new download available at some point in the next three weeks (this may or may not actually have a version number greater than 3.4.5).

You will need to download and install this new release of SPINEVOLUTION once it become available in order to keep using SPINEVOLUTION past January 1st, and you will need to do this two or three times each year since it looks like each release of SPINEVOLUTION will only run for four to six months from the time it is released before expiring. For those releases that increment the SPINEVOLUTION version number, you will need to change the version number in all relevant scripts and commands.

export SPINEV=${HOME}/spinev-3.4.5-Linux-x86_64