Explore ARCExplore ARC

HPC outage due to power maintenance, Saturday, April 2

By | Events

The Modular Data Center, which houses the Flux HPC cluster, will be without power starting from 6 a.m. to approximately 6 p.m. on Saturday, April 2, for preventative maintenance of electrical infrastructure on North Campus.  Additional networking maintenance for campus storage systems is going to start at 11pm Saturday night at the Administrative Services Building machine room which will also impact the HPC clusters.

Therefore, we expect Flux, Armis, /scratch and transfer hosts to be out of service from 6 a.m. until at least midnight.

During the outage, annual preventative maintenance on the MDC will also take place. ARC-TS will also take advantage of the outage to install firmware updates to our new InfiniBand switch.

We will update the outage schedule as needed on Twitter at @ARCTS_UM.

Student groups can access Flux at no charge under Flux For Undergraduates program

By | Educational

Undergraduate groups can now access Flux, U-M’s shared computing cluster, at no cost under the new Flux For Undergraduates program from Advanced Research Computing (ARC).

Flux For Undergraduates aims to provide undergraduates with experience in high performance computing and access to computational resources for their projects; it is not meant for faculty-led research. Jobs submitted underFlux For Undergraduates will run only when unused cycles are available.

Student groups can also purchase Flux allocations for jobs that are higher priority or time constrained; those allocations can also work in conjunction with the free Flux For Undergraduates jobs.

Undergraduate groups must have a faculty sponsor to be eligible for Flux For Undergraduates. For more information, please email hpc-support@umich.edu. To request time under Flux For Undergraduates, fill out the application form.

HPC outage due to power maintenance, Saturday, April 2

By | | No Comments

The Modular Data Center, which houses the Flux HPC cluster, will be without power starting from 6 a.m. to approximately 6 p.m. on Saturday, April 2, for preventative maintenance of electrical infrastructure on North Campus.  Additional networking maintenance for campus storage systems is going to start at 11pm Saturday night at the Administrative Services Building machine room which will also impact the HPC clusters.

Therefore, we expect Flux, Armis, /scratch and transfer hosts to be out of service from 6 a.m. until at least midnight. During the outage, annual preventative maintenance on the MDC will also take place. ARC-TS will also take advantage of the outage to install firmware updates to our new InfiniBand switch. We will update the outage schedule as needed on Twitter at @ARCTS_UM.

New ARC Connect service provides desktop graphical interface for HPC resources

By | Educational, Flux, General Interest, News

Users of ARC-TS computing resources can now use desktop versions of popular software packages like Matlab and R while accessing the Flux shared computing cluster. The new service, called ARC Connect, provides an easily accessible graphical user interface that simplifies doing interactive, graphical work backed by the high performance and large memory capabilities of the Flux cluster.

Using ARC Connect may benefit you if you need to:

  • Very easily interactively use graphical software on HPC clusters (Flux, Armis).
  • Do high performance, interactive visualizations.
  • Share and collaborate with colleagues on HPC-driven research.
  • Use HPC in teaching.
  • Access ARC HPC clusters from off-campus without using the U-M VPN.

Features:

  • Remote desktop sessions (VNC) for using Flux graphically and interactively.
  • Jupyter notebooks for Python and R (other languages coming soon).
  • RStudio interactive development environment for R.

Users can run desktop applications such as MATLAB or RStudio as if running on a laptop, but with all the power of Flux, as opposed to using them in batch mode or via text-only interactive sessions. Users can also use notebooks which require more processing power or memory than are available on their local computer or tablet (currently, Python and R notebooks are available).

ARC Connect is an enhanced version of the TACC / XSEDE Visualization Portal, and has been made possible at the University of Michigan through a collaboration between ARC Technical Services and the Texas Advanced Computing Center at the University of Texas.

For information on how to use ARC Connect, visit arc-ts.umich.edu/arc-connect. If you need further help, contact hpc-support@umich.edu.

Video, slides available: “Advanced Research Computing at Michigan, An Overview,” Brock Palen, ARC-TS

By | General Interest, News

Video (http://myumi.ch/aAG7x) and slides (http://myumi.ch/aV7kz) are now available from Advanced Research Computing – Technology Services (ARC-TS) Associate Director Brock Palen’s presentation “Advanced Research Computing at Michigan, An Overview.”

Palen gave the talk on June 27, 2016, outlining the resources and services available from ARC-TS as well as from off-campus resource providers.

Cluster upgrades completed

By | Flux, General Interest, News

Several key systems were updated and improved during the ARC-TS HPC summer maintenance from July 16 – 23, 2016.

Among other improvements, the updates provide access to more current versions of popular software and libraries, allow new features and more consistent runtimes for job scheduling, and migrate two-factor authentication for the login servers to a new system.

The improvements included:

  • Upgrades to the operating OS and supporting software for the cluster. This was a major update to the previously installed RedHat version (RHEL 6.6) up to CentOS 7.1. This provides newer versions of commonly used software and libraries, and will help us deliver more user-facing features in the coming months.
  • Cluster management software updates and reconfiguration. This includes Torque 6, which has a new set of resource options. The new Torque version will give better language for defining tasks, more consistent runtimes, and a platform for new  features.
  • The Flux Hadoop environment upgrade to Cloudera 5.7, which now includes Hive-On-Spark (the Hadoop cluster will return to service later this week).
  • /scratch on Flux updates.
  • Transition of the software modules environment to a system called Lmod. For more information, see our Lmod transition page. The complete Lmod User Guide can be found here: https://www.tacc.utexas.edu/research-development/tacc-projects/lmod/user-guide.

An HPC 2016 Upgrade Frequently Asked Questions page is available documenting a variety of issues related to the upgrades. Please direct any questions to hpc-support@umich.edu.

Research highlights: Running climate models in the cloud

By | General Interest, News, Research

Xianglei Huang

Can cloud computing systems help make climate models easier to run? Assistant research scientist Xiuhong Chen and MICDE affiliated faculty Xianglei Huang, from Climate and Space Sciences and Engineering (CLASP), provide some answers to this question in an upcoming issue of Computers & Geoscience (Vol. 98, Jan. 2017, online publication link: http://dx.doi.org/10.1016/j.cageo.2016.09.014).

Teaming up with co-authors Dr. Chaoyi Jiao and Prof. Mark Flanner, also in CLASP, as well as Brock Palen and Todd Raeker from U-M’s Advanced Research Computing – Technology Services (ARC-TS), they compared the reliability and efficiency of Amazon’s Web Service – Elastic Compute 2 (AWS EC2) with U-M’s Flux high performance computing (HPC) cluster in running the Community Earth System Model (CESM), a flagship climate model in the U.S. developed by the National Center for Atmospheric Research.

The team was able to run the CESM in parallel on an AWS EC2 virtual cluster with minimal packaging and code compiling effort, finding that the AWS EC2 can render a parallelization efficiency comparable to Flux, the U-M HPC cluster, when using up to 64 cores. When using more than 64 cores, the communication time between virtual EC2 exceeded the distributed computing time.

Until now, climate and earth systems simulations had relied on numerical model suites that run on thousands of dedicated HPC cores for hours, days or weeks, depending on the size and scale of each model. Although these HPC resources have the advantage of being supported and maintained by trained IT support staff, making them easier to use them, they are expensive and not readily available to every investigator that needs them.

Furthermore, the systems within reach are sometimes not large enough to run simulations at the desired scales. Commercial cloud systems, on the other hand, are cheaper and accessible to everyone, and have grown significantly in the last few years. One potential drawback of cloud systems is that the user needs to provide and install all the software and the IT expertise needed to run the simulations’ packages.

Chen and Huang’s work represents an important firstxiangleihuangpost2016 step in the use of cloud computing in large-scale climate simulations. Now, cloud computing systems can be considered a viable alternate option to traditional HPC clusters for computational research, potentially allowing researchers to leverage the computational power offered by a cloud environment.

This study was sponsored by the Amazon Climate Initiative through a grant awarded to Prof. Huang. The local simulation in U-M was made possible by a DoE grant awarded to Prof. Huang.

Top image: http://www.cesm.ucar.edu/

HPC User Meetup

By |

Users of high performance computing resources are invited to meet ARC-TS HPC operators and support staff in person.

There is not a set agenda; come at anytime and stay as long as you please. You can come and talk about your use of any sort of computational resource, Flux, Armis, Hadoop, XSEDE, Amazon, or other.

Ask any questions you may have. The ARC-TS staff will work with you on your specific projects, or just show you new things that can help you optimize your research.

This is also a good time to meet other researchers doing similar work.

This is open to anyone interested; it is not limited to Flux users.

Examples of potential topics:

  • What ARC-TS services are there, and how to access them?
  • I want to do X, do you have software capable of it?
  • What is special about GPU/Xeon Phi/Accelerators?
  • Are there resources for people without budgets?
  • I want to apply for grant X, but it has certain limitations. What support can ARC-TS provide?
  • I want to learn more about the compiler and debugging?
  • I want to learn more about performance tuning, can you look at my code with me?
  • Etc.

HPC User Meetup

By |

Users of high performance computing resources are invited to meet ARC-TS HPC operators and support staff in person.

There is not a set agenda; come at anytime and stay as long as you please. You can come and talk about your use of any sort of computational resource, Flux, Armis, Hadoop, XSEDE, Amazon, or other.

Ask any questions you may have. The ARC-TS staff will work with you on your specific projects, or just show you new things that can help you optimize your research.

This is also a good time to meet other researchers doing similar work.

This is open to anyone interested; it is not limited to Flux users.

Examples of potential topics:

  • What ARC-TS services are there, and how to access them?
  • I want to do X, do you have software capable of it?
  • What is special about GPU/Xeon Phi/Accelerators?
  • Are there resources for people without budgets?
  • I want to apply for grant X, but it has certain limitations. What support can ARC-TS provide?
  • I want to learn more about the compiler and debugging?
  • I want to learn more about performance tuning, can you look at my code with me?
  • Etc.