Using Open OnDemand

Open OnDemand is a way for users to access to their HPC resources via a web interface for the Great Lakes and Lighthouse clusters. This includes:

  • Easy file management
  • Command-line shell access
  • Job management and monitoring
  • Graphical desktop environments and desktop applications such as RStudio, MATLAB, and Jupyter Notebook

   TO SEARCH THIS USER GUIDE, USE THE COMMAND + F (MAC) OR CTRL + F (WIN) KEYBOARD SHORTCUTS.

Getting Started

1. Get Duo

Duo is required for all services at the University of Michigan. Information about Duo can be found here.

2. Get a user login and have access to an account

You must establish a user login on Great Lakes or Lighthouse by filling out this form.
You must have access to an account. An account is paid access to resources on the clusters. If you believe you have access to an account on either cluster but cannot run a job from this interface please email us for assistance.

3. Connect to Great Lakes OnDemand

You must be on campus or on the VPN to connect to Great Lakes OnDemand. If you are trying to log in from off campus or using an unauthenticated wireless network such as MGuest, you should install VPN software on your computer.
Once you are on the University network, follow these instructions to connect:
Open your web browser (Firefox, Edge, or Chrome in incognito recommended) and navigate to:

Log into UMich weblogin using your uniqname and password:

Once you’ve logged in you’ll be prompted for DUO two-factor authentication. Once that process is complete you will be logged in:

4. Get files

At the top of the page, click “Files” and then “Home Directory”. A new tab will be created that contains the File Explorer:

Here you can navigate your home folder. The buttons do the following:

  • “Go To…”: Navigate to a specified folder
  • “Open in Terminal”: Opens the active folder in a terminal session (new tab)
  • “New File”: Creates a new file in the active folder
  • “New Dir”: Creates a new folder in the active folder
  • “Upload”: Select files from your local machine to upload to the active folder
  • “Show Dotfiles”: Reveals hidden files (usually do not need to be changed)
  • “Show Owner/Mode”: Shows ownership and permission information
  • “View”: Shows file contents inside the current tab
  • “Edit”: Opens a file editor in a new tab
  • “Rename/Move”: Gives a file a new path and/or name
  • “Download”: Downloads the file or folder to your local machine
  • “Copy”: Copies selected files to the clipboard
  • “Paste”: Pastes files from the clipboard
  • “(Un)Select All”: Select or unselect all files/folders
  • “Delete”: Deletes selected files/folders

5. Submit a job

At the top of the home page, click “Jobs” and then “Job Composer”. A new tab will be created that contains the Job Composer. Upon your first visit to this page, you’ll go through a helpful tutorial:

The buttons do the following:

  • “New Job”: Creates a new job…
    • “From Default Template”: Uses system defaults for a bare bones “Hello World” job on the Great Lakes cluster. Please note that you will still need to specify your account.
    • “From Specified Path”: Creates a job from a specified job script. See the Slurm User Guide for Great Lakes for information on writing this script. Some attributes (name, account) can be set here if not set in the script.
    • “From Selected Job”: Creates a new job that is a copy of the selected job.
  • “Edit Files”: Opens a the project folder in a new File Explorer tab, allowing you to edit the files within (see “Get Files” above for File Explorer instructions).
  • “Job Options”: Allows for editing the Name, Cluster, Job Script, and Account fields.
  • “Open Terminal”: Opens a terminal session in a new tab, starting in the project folder.
  • “Submit”: Submits the selected job to the cluster.
  • “Stop”: Stops the selected job if it has been submitted.
  • “Delete”: Delete the selected job.

To view active job information, click “Jobs” and then “Active Jobs” from the home page.
This is a simple guide to get your jobs up and running. For more advanced Slurm features and job scripting information, see the Slurm User Guide for Great Lakes. If you are familiar with using the resource manager Torque, you may find the migrating from Torque to Slurm guide useful.

Back To Top

Interactive Apps

Open OnDemand offers several interactive applications for a user to use: RStudio, Jupyter Lab and Notebook, MATLAB, Spark and the ability to set up various Linux remote desktops on Great Lakes or Lighthouse.

 

For each of the interactive apps you’ll need to select the following:

  • Account
  • Number of hours
  • Partition
  • Number of cores
  • Memory per node (GB)
  • Number of GPUs if the gpu partitoon is chosen
  • Number of nodes for some applications

 

Additional steps may be needed to launch each app. More detailed information about launching apps can be found below.

Back To Top

RStudio

Please note: gpus are not available with RStudio at this time.

After choosing RStudio from the Interactive Apps menu you’ll need to specify your desired version, account, hours, cores, partition (standard or largemem), and memory (2GB minimum). You also need to choose the version of RStudio version you want to use from a drop down list of choices.

Upon selecting “Launch”, your job will be queued on one of your nodes and shown on the “My Interactive Sessions” screen. As soon as the job’s status is “Running”, you can change remote desktop settings. For slower internet connections, you can try a higher compression and lower image quality using the sliders. Conversely, if you have a fast connection you can lower compression and raise image quality. You can also directly access your node’s terminal by clicking on the hostname (blue button). Once you’re ready to use the application, click on “Launch RStudio”.

A remote desktop session running RStudio will then be opened in a new tab for the requested amount of time. You may also use the terminal and other basic applications. If you finish early, return to the “My Interactive Sessions” tab and delete the job.

Back To Top

Jupyter Lab and Notebook

 

After choosing Jupyter Lab or Jupyter Notebook from the Interactive Apps menu you’ll need to specify your desired version, account, hours, cores, partition (standard or largemem), and memory (2GB minimum). You’ll also need to specify the Anaconda Python module you want to use:

 

Upon selecting “Launch”, your job will be queued on one of your nodes and shown on the “My Interactive Sessions” screen. As soon as the job’s status is “Running”, you can click on “Connect to Jupyter”.

For instructions on using Jupyter Notebook, see the official documentation.

Back To Top

Spark and PySpark

Spark and Pyspark are available via the Jupyter + Spark Basic and Jupyter + Spark Advanced interactive applications on Open on Demand for Armis2, Great Lakes, and Lighthouse. 

 

The Basic application provides a starter Spark cluster of 16 cpu cores and 90 GB of RAM with a one day (24 hour) walltime limit designed for beginner Spark users.   This is especially useful for python novices who have not spent time customizing their environment, as well as for newcomers to the Spark ecosystem.  

 

The Advanced application provides a more comprehensive and configurable interface for advanced Spark applications. Here, a user can ask for more cores and memory than the default, but understanding that Spark has certain requirements for its software executors (namely 3 CPU cores and 15 GB or RAM per executor).  Users can also load module files or source a setup file where shell variables can be defined.  Note that modules must be compatible with Python 3.10 because the Spark application uses this version of python.  

 

Note that closing the browser tab of the Jupyter Server or Jupyter Notebook DOES NOT stop the Spark cluster that runs in the background. Your account will continue to accrue charges until you explicitly stop the job or the wall time expires. To stop the job, click the ‘Quit’ button in the upper right of the Jupyter Server web page.

Back To Top

MATLAB

After choosing MATLAB from the Interactive Apps menu you’ll need to specify your desired version, account, hours, cores, partition (standard or largemem), and memory (4GB minimum). You also need to choose the version of MATLAB you wish to use:

Upon selecting “Launch”, your job will be queued on one of your nodes and shown on the “My Interactive Sessions” screen. As soon as the job’s status is “Running”, you can change remote desktop settings. For slower internet connections, you can try a higher compression and lower image quality using the sliders. Conversely, if you have a fast connection you can lower compression and raise image quality.  You can also directly access your node’s terminal by clicking on the hostname (blue button). Once you’re ready to use the application, click on “Launch MATLAB”.

A remote desktop session running MATLAB will then be opened in a new tab for the requested amount of time. You may also use the terminal and other basic applications. If you finish early, return to the “My Interactive Sessions” tab and delete the job.

Back To Top

Remote Desktops

We offer three types of Linux rempote desktops, depending on the type of work you will be doing.

  • Basic Desktop intended for single node jobs
  • Multinode/MPI Desktop for jobs requiring more than one node using the MPI protocol
  • Advanced Desktop for very advanced users who hybrid MPI/OpenMP code for multithreaded multitask jobs.

After choosing your desired type of Remote Desktop from the Interactive Apps menu you’ll need to specify your desired version, account, hours, cores, partition (standard, gpu, or largemem), and memory (2GB minimum). You also have the option of choosing gpus if you need to use them:

 

Please keep in mind that if you need gpus you must choose the gpu partition.

Upon selecting “Launch”, your job will be queued on one of your nodes and shown on the “My Interactive Sessions” screen. As soon as the job’s status is “Running”, you can change remote desktop settings. For slower internet connections, you can try a higher compression and lower image quality using the sliders. Conversely, if you have a fast connection you can lower compression and raise image quality. You can also directly access your node’s terminal by clicking on the hostname (blue button). Once you’re ready to use the desktop, click on “Launch Greatlakes Remote Desktop”.

A remote desktop session will then be opened in a new tab for the requested amount of time. If you finish early, return to the “My Interactive Sessions” tab and delete the job.

Back To Top

Troubleshooting

Known Issues

  • Session vanishes even though the job still shows up in the job list. This is due to a socket issue on the cluster. If you experience this please email us.

  • Open OnDemand works best with a default Linux environment. Customizing your environment may produce irregular/unexpected issues or behaviors in Open OnDemand.

Back To Top