Planning a Flux Allocation

If you are unsure of the Flux allocation size for your project, we recommend you start with an initial 4-8 core allocation for the first month. This will give you time to determine your work-flow and validate your software on Flux. Flux Sizing has more information on how to size an allocation.

Creating a Flux Allocation

A Flux allocation is described in How Flux Works. We will create a Flux project for you the first time you request an allocation. The default name of the project will follow this format: uniqname _flux. If you have multiple projects, we will add a number after the uniqname (e.g., powell1_flux).

Flux allocations are made in increments of one core for one month, up to a maximum of 2,500 cores and 60 months. Allocations can be ended or shrunk at any month boundary with no penalty, or can be increased at any time.

Most Flux allocations are created within one business day. To create a Flux allocation, please send email to hpc-support@umich.edu with the following information:

  • the number of cores needed
  • the start date and number of months for the allocation
  • the shortcode for the funding source
  • the list of people who should have access to the allocation
  • the list of people who can change the user list and augment or end the allocations

Establishing a Flux User Account

To request a Flux user account, complete this form. Faculty members should fill in their names in both the requestor and advisor fields. Students should specify the name of their advisor or other associated faculty member in the advisor fields.

Using a Flux Allocation

Flux is used via a batch system called Torque PBS. The Flux-specific additions and changes to the examples at that URL are the queue, the account, and the qos. In a PBS script these should look like:

#PBS -A example_flux

#PBS -l qos=flux

#PBS -q flux

For more information on using Flux, including information on training, application-specific information and more, visit the Using Flux section of this site.

When you submit compute jobs to Flux, the job scheduler will fit them into your allocation. When your allocation is full, or your next job is too large to fit into the remaining room in the allocation, the job will be queued and executed as soon as the resources are available.

If you request special features such as large amounts of memory, processors in a specific physical configuration, GPUs, etc. your job may be queued while the scheduler waits for those resources to become free.

During times of high system utilization jobs may remain in the queue before starting even though your allocation may not be full.

Questions and other support requests from the College of Engineering, Medical School, and College of Literature, Science, and the Arts may be referred to staff located in those units.

Monitoring a Flux Allocation in Real Time

To monitor your allocations in real time, go to the Flux login hosts and type the command:

showq -w acct=the name of your Flux project

You will see the the jobs currently running, idle and blocked.

  • Idle jobs are jobs that could run if resources (cores, memory, software licenses, GPUs, etc.) were available.
  • Blocked jobs are jobs that violate some constraint and are not considered for scheduling onto a core. Most typically this is because the jobs would cause you to exceed the number of cores in your allocation.

Monitoring the Historical Use of a Flux Allocation

Historical usage data for Flux allocations is available via MReports to those who are responsible for an allocation. Complete instructions are available at arc-ts.umich.edu/check-my-flux-allocation.

Changing the Use of a Flux Allocation

The only change you may make to a Flux allocation is to end it at the next month boundary. However, you may create new allocations and overlap them with an existing allocation.

For example, you have one allocation with 24 cores and their associated RAM that runs from April 14 until October 14. During the summer months, you are able to spend more time on the project and require more cores. Starting on May 3 you add another 48 cores to the same Flux project in a new allocation, which you schedule to end on October 3. You may submit jobs to these 72 cores as though they were in a single allocation.

In July, you decide to take a vacation in August, so you end that allocation on August 3 and do not pay for August, September, or the three days in October. Pictorially, this looks like:

Screen-Shot-2012-04-02-at-3.50.21-PM

 

All of these changes are made via email to hpc-support@umich.edu or your local Flux support staff.

Paying for a Flux Allocation

Flux charges are billed monthly through ITS, and must be charged to a U-M shortcode.

The billing period runs from the 13th to the 12th of the month. In the example above, the May, June, and July bills will be for 72 cores and the August, September, and October bills will be for 24 cores.

Funding can come from a variety of sources:

  • federally funded research projects
  • general funds
  • departmental instructional funds
  • faculty discretionary and research incentive accounts
  • cost sharing
  • faculty start-up package funds.

For information on including funding for a Flux allocation in a grant application, see Flux for Grant Writers and Research Administrators.

If you have questions, please send email to hpc-support@umich.edu.

Preparing for the End of a Flux Allocation

Approximately two weeks before the end of an allocation, someone from the Flux operations group will email you a reminder about the end of an allocation to give you the chance to renew the allocation.

If you choose not renew your allocation, your directory on the /scratch filesystem will become inaccessible two weeks after the expiration of your last allocation. After eight weeks without an active allocation, all data in your directory /scratch/example_flux will be permanently deleted.

If you have questions, please send email to hpc-support@umich.edu.