The size of your production Flux allocation depends significantly the program(s) you will be running. One big advantage of Flux is that if you estimate wrong on the low side, you can add capacity at any time, and if you estimate wrong on the high side, you can change your allocation at the next nearest 30 day boundary.
We recommend establishing a small initial Flux allocation — four to eight cores — to work out code compilation, job submission, data transfer, and other work-flow related tasks before requesting the full allocation you expect to need for your production work.
The general approach to determining the size for your Flux allocation is to characterize:
- the type and quantity of Flux compute jobs, for example:
- are the jobs parallel or serial?
- what are the memory requirements of the job?
- how many jobs do you expect to run at a time?
- how many jobs do you expect to run total?
- the users who should be associated with the Flux project
- the number of months for which you want the allocation
- the applications you expect to run using this allocation
With this information, you can request an allocation by emailing firstname.lastname@example.org.
Example Scenarios for Sizing a Flux Allocation
Running a Parallel Application
If your application runs in parallel, you should determine how it scales on Flux by:
- considering the maximum number of cores the application will run on efficiently (ARC-TS staff can help you determine this).
- determining how many copies you expect to run at once based on licenses, work-flow, the number of students on this project, etc. That will help inform you as to the number of cores to have in your allocation.
Using Licensed Commercial Software
If you are using licensed commercial software, first determine the maximum number of licenses you can reasonably expect to use on Flux (ARC-TS staff can help you determine this). Flux’s job scheduling can take software licenses into account, so you may have queued jobs if there aren’t enough licenses. If you determine that the maximum number of licenses is 24, an allocation that would require more than 24 licenses at a time would be a waste of your resources.
Using Flux in a Course
To choose an allocation for a course, you should consider:
- the number of students in the course, and
- the type of application they will be running: Is it serial or parallel? How long does a typical assignment take to complete? How many processors will one student require per assignment?
There are a wide range of course needs. Classes with as many as 25 students have required as few as 36 cores, while other classes have used over 100 cores for the term.
One example is a course where all 20 students are expected to run a parallel application that scales to 16 cores. The cluster-related work is only in the four homework assignments given over the last six weeks of the class, and each run takes 2-4 hours. Students may not start submitting their jobs until 36 hours before the homework is due, so the class, as a whole, will use 1,280 CPU-hours in a 36-hour period, so the course allocation should be a minimum of 36 cores. Since the jobs won’t all stack up perfectly, having a multiple of 16 cores would lead to the best efficiency. A recommended Flux allocation for the course is 48 or 64 cores for two months, starting October 15.
Using Flux to Emulate a Hardware Purchase
A faculty member would like to have a Flux allocation that emulates the ownership of a 96-core system with up to 4GB of RAM for five years. A Flux allocation of 96 cores with the default 4GB of RAM per core for five years should be requested and budgeted for.
Using Flux for a Flexible Allocation of CPU Time
Working with Flux staff and staff from his or her unit, a faculty member has developed an estimated requirement of 8,000 core months that could be used at any time over five years. A budget allocation for a Flux allocation of 8,000 core months should be made.
The faculty member can then adjust the Flux allocation as their research requires and as the 8,000 core month budget allows.
Using Flux to Meet Application Memory Requirements
If you have an application that requires more than the standard memory per core available on Flux, you will need to get an allocation of more cores than you need in order to meet the memory needs.
For example, if Flux has 4GB of RAM per core and your application requires 10 cores and 6GB RAM per core, your allocation will need to be for 15 cores, which will provide the needed 60GB RAM, even though you will only compute on 10 of the 15 cores.