Armis2 is available for general access

By | Armis2, HPC, Systems and Services

U-M Armis2: Now available

What is Armis2?

The Armis2 service is a HIPAA-aligned, HPC platform for all University of Michigan researchers and is a successor to the current Armis cluster. It is based on the same hardware as the current Armis system, but uses the Slurm resource manager rather than the Torque/Moab environment. 

If your data falls under some non-HIPAA, restricted use agreement, contact hpc-support@umich.edu to discuss whether or not you can run your jobs on Armis2. 

Key features of Armis2 

  • 24 standard nodes using the Intel Haswell-processor, each with 24 cores. More capacity will be added in the coming weeks
  • Slurm provides the resource manager and scheduler 
  • The scratch storage system will provide high-performance temporary storage for compute. See the User Guide for quotas and the file purge policy
  • The EDR InfiniBand network is 100Gb/s to each node
  • Large Memory Nodes have 1.5TB memory per node
  • GPU Nodes with NVidia K40GPUs (4GPUs per node)

ARC-TS will be adding more standard, large memory, and GPU nodes in the coming weeks during the transition from Armis to Armis2, as well as migrating hardware from Flux to Armis2. For more information on the technical aspects of Armis2, see the Armis2 configuration page.

When will Armis2 be available?

Armis2 is available now; you can log in at: armis2.arc-ts.umich.edu.  

Using Armis2

Armis2 has a simplified accounting structure. On Armis2, you can use the same account and simply request the resources you need, including standard, GPU, and large memory nodes.

Active accounts have been migrated from Armis to Armis2. To see which accounts you have access to, type my_accounts to view the list of accounts. See the User Guide for more information on accounts and partitions.

Armis2 rates

Armis2 is a “pay only for what you use” model. We will be sharing rates shortly. Send us an email at hpc-support@umich.edu if you have any questions. 

Previously, Armis was in tech-preview mode and you were not billed for the service. Use of Armis2 is currently free, but beginning on December 2, 2019, all jobs run on Armis2 will be subject to applicable rates.

View the Armis2 rates page for more information. 

How does this change impact me?

All migrations from Armis to Armis2 must be completed by November 25, 2019, as Armis will not run any jobs beyond that date. View the Armis2 HPC timeline.

The primary difference between Armis2 and Armis is the resource manager. You will have to update your job submission scripts to work with Slurm; see the Armis2 User Guide for details on how to do this. Additionally, you’ll need to migrate any data and software from Armis to Armis2.

How do I learn how to use Armis2?

To learn how to use Slurm and for a list of policies, view the documentation on the Armis2 website

Additionally, ARC-TS and academic unit support teams will be offering training sessions around campus. As information becomes available, there will be a schedule on the ARC-TS website as well as Twitter and email.

 

Great Lakes Update: August 2019

By | General Interest, Great Lakes, Happenings, HPC, News

Great Lakes cluster is available for general access

What is the current status of the Great Lakes cluster

Now that we have completed Early User testing, the Great Lakes cluster is available for general access to the University community. Until the migration from Flux is complete on November 25, 2019, there will be no charge for using the Great Lakes cluster.

Noteworthy Features

  • The Great Lakes cluster compute nodes use the new Intel Skylake processor. In particular, the Skylake CPUs on the standard and large memory compute nodes will provide researchers more consistent performance, regardless of how many other jobs are on the machine. 
  • The Great Lakes cluster has 20 GPU nodes, each of which contains two NVidia V100 GPUs which are significantly faster than the K20 and K40 GPUs on Flux.
  • The HDR100 InfiniBand network will provide consistent 100Gb/s performance across all nodes. On Flux, this ranged from 40-100Gb/s, depending on the node your job used.
  • The high performance GPFS /scratch system, with a capacity of approximately two petabytes, is significantly faster than /scratch on Flux. 
  • The Torque-based batch job submission environment has been replaced with the Slurm resource manager. We expect this system to be significantly more responsive and quicker at starting jobs than was the case on Flux.
  • For web-based job submission, the Open OnDemand system will replace the ARC Connect environment for providing web based file access, job submission, remote desktop, graphical Matlab, Jupyter Notebooks, and more. For more information, see the web-based access section in our user guide. 

How do I get access?

Every Flux user has a login on the Great Lakes cluster; you should be able to log in via ssh to greatlakes.arc-ts.umich.edu. We have created Slurm accounts for each PI or project based on the current Flux accounts. You can see what Slurm accounts you have access to by running the command `my_accounts.`  

Additionally, you can access the Great Lakes cluster via the web through our Open OnDemand portal. Here you can submit jobs, see submitted jobs, create Jupyter Notebooks and more. Please see the Great Lakes Cluster User Guide for more information.

Where do I read more about the Great Lakes cluster and how to use it?

The current documentation for the Great Lakes cluster, including configuration, user guides, and known issues can be found at https://arc-ts.umich.edu/greatlakes.

There is a schedule for upcoming training sessions on the CSCAR website, and we will communicate new sessions through Twitter and email.

Software

Almost all of the software packages available on Flux have been recompiled on the Great Lakes cluster for improved performance anticipated from the Intel SkyLake architecture. In most cases, the latest software version available is being provided. If you need older versions or need additional packages, let us know via email at hpc-support@umich.edu

We have also reorganized the software module structure to make it easier to find packages you want to load as well as automatically loading prerequisites. To search for packages, use the “module spider” command along with the name of the package or keywords. In many cases we combined similar packages into “Collections” such as Chemistry and BioInformatics. The command “module load Chemistry” will make any Chemistry package available to you and packages in the Chemistry collection will then be discoverable via the “module available” command. After loading a specific collection, you must then load any individual packages within that collection that you would like to use.

What are the rates? 

We are working with ITS and UM Finance for approved service rates. Current plans are to have proposed1 rates identified by end of August. As soon as this information is more concrete, we will provide an update on the Great Lakes cluster website and in our email communication. We understand that this information is necessary for planning purposes and apologize for any impacts this has had on your budget planning. 

What can be shared at this time is the new approach to billing that will be used for the Great Lakes cluster. Unlike Flux, there are no monthly allocations with fixed fees regardless of whether they are used or not. On the Great Lakes cluster, the monthly charge for an account will be calculated based on the resources used by jobs each month. The cost calculation for each job will be based on the amount and type of resources the job reserves and how long the job runs. This should be a significantly more flexible system and won’t require updating allocations as your computing needs change over time.

1 Rates are not considered final until they have been formally approved by OFA.

Flux to the Great Lakes cluster transition efforts

If you have not already, you should be developing a plan to migrate your work from Flux to the Great Lakes cluster.  If you need help in developing a plan, please contact us and we can provide assistance during this migration period. 

  • ARC-TS and academic unit support teams will be offering training sessions around campus. We will have a training sessions schedule on the ARC-TS website. We also communicate new sessions through Twitter and email.
  • To assist your transition, if you have any Turbo or MiStorage NFS mounts on Flux, those mounts will also be available on the Great Lakes cluster.  If you would prefer to not have those volumes mounted on the Great Lakes cluster, email us at hpc-support@umich.edu.

Ensure that your migration from Flux to the Great Lakes cluster is completed by November 25, 2019. No jobs on Flux will run after November 25, 2019.

Additional Information

We will be adding new capabilities in the coming weeks and months and will continue to communicate these capabilities by email as they become available. If you have any questions, email us at hpc-support@umich.edu.

Modular Data Center Electrical Work

By | Flux, Systems and Services, Uncategorized
[Update 2019-05-17 ] The MDC electrical work was completed successfully and Flux has been returned to full production.

 

The Modular Data Center (MDC), which houses Flux, Flux Hadoop, and other HPC resources, has an electrical issue which requires us to bring the power usage below 50% for the some racks in order to resolve the problem.  In order to do this, we have put reservations on some of the nodes to reduce the power draw so the issue can be fixed by ITS Data Centers.  Once we hit the target power level and the issue is resolved, we will remove the reservations and return Flux and Flux Hadoop back into full production level.