Explore ARCExplore ARC

ARC-TS HPC Maintenance

By |

Flux, Armis, Flux Hadoop, and their storage systems (/home, /scratch, and HDFS on Flux Hadoop) will be unavailable starting at Saturday, July 16 at 2 p.m., with a return to service targeted for mid-day July 22nd. During this time, ARC-TS will update several key systems. Among other improvements, the updates will provide access to more current versions of popular software and libraries, allow new features and more consistent runtimes for job scheduling, and migrate two-factor authentication for the login servers to a new system.

NOTE: With the University migrating to Duo from RSA for multifactor authentication in July, ARC-TS will switch to Duo for access to our login nodes during this maintenance period. (Units will be leading the switch to Duo with their faculty, staff and students who currently use MTokens. Questions about this change should be directed to IT or administrative leaders in units. More information can be found here:  http://www.itcs.umich.edu/identity/2factor/)

The updates will consist of:

  • OS and supporting software updates for the cluster. This will be a major update to the currently installed RedHat version (RHEL 6.6) moving to CentOS 7.1. This will provide newer versions of commonly used software and libraries, as well as helps us deliver more user-facing features in the coming months.
  • Cluster management software will be updated and reconfigured. This will include Torque 6, which has a new set of resource options. The new Torque version will give better language for defining tasks, more consistent runtimes, and a platform for new  features.
  • The Flux Hadoop environment will be updated to Cloudera 5.7, which now includes Hive-On-Spark.
  • /scratch on Flux will be updated and serviced.
  • The modules environment will transition from the current Environment Modules to a system called Lmod. The Lmod User Guide can be found here: https://www.tacc.utexas.edu/research-development/tacc-projects/lmod/user-guide.
    Many commands are the same, and we will document any significant differences.
  • The paths in which many software packages are installed will also change; e.g., folders like /home/software and /usr/cac will have new locations. This will also be documented.
  • Many default software versions will be changed and some older software packages and/or versions will be retired. In particular, OpenMPI and the compilers will all get updated to new versions.

Status updates will be posted on the ARC-TS Twitter feed  https://twitter.com/arcts_um.

Advanced Research Computing at Michigan — An Overview

By |

Brock Palen, Associate Director of Advanced Research Computing – Technology Services, will provide an overview of the resources available to researchers engaged in computationally intensive science on the University of Michigan campus.

The talk is open to researchers from any department at U-M.

The session will address:

  • high performance computing services
  • data science services such as Hadoop and Spark
  • research storage
  • cloud services
  • database hosting
  • networking services
  • grant consultation and collaboration
  • access to off-campus resources.

There will be time for questions and answers after the presentation.

RSVP requested.

A live video stream of the event will be available:

Video, slides available: “Advanced Research Computing at Michigan, An Overview,” Brock Palen, ARC-TS

By | General Interest, News

Video (http://myumi.ch/aAG7x) and slides (http://myumi.ch/aV7kz) are now available from Advanced Research Computing – Technology Services (ARC-TS) Associate Director Brock Palen’s presentation “Advanced Research Computing at Michigan, An Overview.”

Palen gave the talk on June 27, 2016, outlining the resources and services available from ARC-TS as well as from off-campus resource providers.

Data Science Skills Series week 1

By |

Fast data processing with Go

We will demonstrate basic Go using several practical examples of data manipulation.  Go (golang.org) is an open source programming language that has many syntax features of high level languages like Python, but is statically typed, compiled, and has native concurrency support.  For these reasons, Go can be extremely efficient at many complex data processing tasks.  

No registration is necessary.

Data Science Skills Series Week 2

By |

Data processing and visualization in R

This workshop will delve into common data processing and exploration techniques, especially as a prelude to visualization.  The main focus will be the dplyr and data.table packages, which enhance and facilitate the sorts of operations that typically arise when dealing with data, including faster I/O and grouped operations.  For visualization, the focus will be on using ggplot2 and other packages that allow for interactivity.

No registration is necessary.

Data Science Skills Series Web Scraping with Python

By |

Web Scraping with Python

We will provide an overview of how to scrape data from html pages and website APIs using Python. For demonstration purposes, we will scrape sports and weather data. This will mostly be accomplished using the Python Requests and Beautiful Soup modules.

No registration is necessary.

Cluster upgrades completed

By | Flux, General Interest, News

Several key systems were updated and improved during the ARC-TS HPC summer maintenance from July 16 – 23, 2016.

Among other improvements, the updates provide access to more current versions of popular software and libraries, allow new features and more consistent runtimes for job scheduling, and migrate two-factor authentication for the login servers to a new system.

The improvements included:

  • Upgrades to the operating OS and supporting software for the cluster. This was a major update to the previously installed RedHat version (RHEL 6.6) up to CentOS 7.1. This provides newer versions of commonly used software and libraries, and will help us deliver more user-facing features in the coming months.
  • Cluster management software updates and reconfiguration. This includes Torque 6, which has a new set of resource options. The new Torque version will give better language for defining tasks, more consistent runtimes, and a platform for new  features.
  • The Flux Hadoop environment upgrade to Cloudera 5.7, which now includes Hive-On-Spark (the Hadoop cluster will return to service later this week).
  • /scratch on Flux updates.
  • Transition of the software modules environment to a system called Lmod. For more information, see our Lmod transition page. The complete Lmod User Guide can be found here: https://www.tacc.utexas.edu/research-development/tacc-projects/lmod/user-guide.

An HPC 2016 Upgrade Frequently Asked Questions page is available documenting a variety of issues related to the upgrades. Please direct any questions to hpc-support@umich.edu.

U-M team uses Flux HPC cluster for pre-surgery simulations

By | Flux, General Interest, News

Last summer, Alberto Figueroa’s BME lab at the University of Michigan achieved an important “first” – using computer-generated blood flow simulations to plan a complex cardiovascular procedure.

“I believe this is the first time that virtual surgical planning was done for real and not as a retrospective theoretical exercise ,” says Figueroa.

Using a patient’s medical and imaging data, Figueroa was able to create a model of her unique vasculature and blood flow, then use it to guide U-M pediatric cardiologists Aimee Armstrong, Martin Bocks, and Adam Dorfman in placing a graft in her inferior vena cava to help alleviate complications from pulmonary arteriovenous malformations (PAVMs). The simulations were done using the Flux HPC cluster.

Read more…

MICDE Seminar: Anthony Wachs, University of British Columbia

By |

cropped-Anthony_Wachs_photo2Bio: Anthony Wachs is an assistant professor with a joint appointment in the departments of Mathematics and of Chemical and Biological Engineering at the University of British Columbia, Vancouver, Canada.

He received his B. Sc. and M. Sc. from the University Louis Pasteur of Strasbourg and his PhD from the Institut National Polytechnique of Grenoble in 2000. Right after, he was hired in 2001 as a Fluid Mechanics research engineer at IFP Energies nouvelles (IFPEN, at that time Institut Français du Pétrole) in Paris.

In 2009, he spent a one-year sabbatical at the nuclear research center of Cadarache in the south of France, where he worked for IRSN (the french national safety administration for nuclear energy). In 2010, he got his HDR (French Habilitation to Supervise Research) and was later promoted Scientific Advisor at IFPEN in Multiphase Flows and Scientific Computing. He then moved to IFPEN-Lyon where he supervised a group of researchers (including PhD and post-doc students) on the numerical simulation of reactive particulate flows (www.peligriff.com).

His main research interests are non-Newtonian Flows, Multiphase Flows and High Performance Computing. He collaborates extensively with academic groups in Canada, Brazil, France and Germany.

On-campus telecast of XSEDE workshop on MPI

By |

U-M is hosting a telecast of a workshop on MPI (message passing interface) presented by XSEDE and the Pittsburgh Supercomputing Center.

This workshop is intended to give C and Fortran programmers a hands-on introduction to MPI programming. Attendees will leave with a working knowledge of how to write scalable codes using MPI – the standard programming tool of scalable parallel computing.

 

Contact: Simon Adorf (csadorf@umich.edu)