Using AWS at U-M

Establishing Service

U-M has an enterprise agreement with AWS that provides additional features, including a waiver of normal data retrieval (or “egress”) charges. To participate in the enterprise agreement, U-M users must take these steps:

  • Prepare for Registration
    • Obtain/create an MCommunity group e-mail address. This should be set to “public”. See http://mcommunity.umich.edu
    • Obtain a Shortcode from your department.
  • Step 1: Register for an AWS account
  • Step 2: Register for the M Cloud Billing Service
    • Go to: http://services.it.umich.edu/m-cloud
    • Click on the Request MCloud–AWS button
    • Fill out the on-line form and submit it.  If you want to enroll in consolidated billing, select the level of service that adds this and also provide shortcode in the form.
  • Step 3: Link your AWS Account to M Cloud Billing Services
    • AWS will send you an email within two business days to request confirmation to link your AWS account to the M-Cloud Payer account.
    • Confirm the AWS email by clicking on the link.
    • Note: Delaying your reply for more than 24 hours could result in AWS negating the request.
  • Process is Complete
    • You are now registered for M Cloud shortcode billing services and your AWS Account ID is also registered under the U-M Enterprise Agreement with AWS. You will see Amazon charges on your monthly statement of activity.

For those with AWS credits and not using the M-Cloud Shortcode Billing Service register your AWS account as above but only enroll in “Enterprise Agreement Only” level of service. You then apply your credits under the “Services” menu of AWS console, choose “Billing” and finally Credits.

For more information visit the M-Cloud website.

Using AWS

AWS has extensive documentation covering all aspects of deploy and using their services.  Below is a set of quick tips for getting started using various AWS resources.  These are just general items to keep in mind. U-M also has documentation on how to get started.

AWS Account

Create an AWS account for each research project that has its own unique shortcode or AWS credit. This will be what is called a “root” account by which all resources and users with the project can be managed. Do not proceed with “Consolidated billing” as per the U-M instructions so that you can utilize free tier, reserved and spot instances. Do enter into the Enterprise agreement as per instructions. You will be able to enter any AWS credits you have once the account is created and activated.

Identity and Asset Management

Create an “administrator” group that has full access to all resources and management of account.

Create a “user” group that has access to resources you define. Add policy templates as per your need. Be sure to include Access to IAM so users can change their own password and setup other IAM service tools.

Create account for administrative purposes and assign to the “administrator” group.  Use this account for most all your management activities. There is a special sign-in url assigned to your project “id#”.???.aws.amazon.com that your users defined below will use to login.

Now create user accounts that are assigned to the “user” group.

Each user then should login via that username and create their own resources. They should also create their own secret ID and Key pair under IAM. This allows them remote access to various resources via command line applications and APIs.

Data Storage Management

Since each EC2 instance has its own EBS volume, users should create a dedicated EBS volume to hold their data and applications.  This volume can then be attached to any one instance for portability. This will be especially helpful when getting spot instances.

Cost and time savings tips

Create a small instance to configure the OS and associated applications, libraries or other tools to do your work. Save this as a snapshot and then an Amazon Machine Image (AMI) in your own library. Use this AMI when deploying compute instances so you do not have to spend time reconfiguring.

Learn about deploying, using and managing Spot instances.  These have the potential to be ⅕ the cost of on demand instances.

You are charged for a node running regardless of whether your job is finished. If you are not going to actively monitor your jobs for completion we suggest running jobs via a script that includes an email statement to you that your job is finished and then also perhaps poweroff the node.  This is one reason to have a data EBS volume to write data on instead of the instance volume.  When spot instances are stopped, they are destroyed, including data on that volume.

Consider adding billing monitor alerts so that if total usage exceeds a certain dollar amount, you will be alerted via email.

Security tips

Since EC2 is not a U-M IT managed service you are on your own for maintaining security of your instances.  Please follow these minimum suggested tips to keep your Instance safe.

AWS makes use of ssh security keys for remote ssh login.  Be sure to keep these safe.

To protect your instances from possible attack we suggest creating a “Security Group” with appropriate firewall rules for specific IP addresses and networks.

For U-M wired networks, add the following:

  • 141.211.0.0/16
  • 141.212.0.0/16
  • 141.213.0.0/16

For wireless:

  • 35.1.0.0/16
  • 35.2.0.0/16

For off-campus U-M locations, contact your local IT group for your particular network.

If you intend to work from your home you should add that IP address as well.  A sample would look like below for a home router IP address 100.101.102.103

  • 100.101.102.103/32

Be advised that this address may not stay fixed, depending on your service provider policies.  If you suddenly lose ability to remotely connect your IP address has likely changed.

Order Service

To take advantage of U-M’s enterprise agreement with AWS, which provides additional features including the waiving of data retrieval or “egress” fees, see these directions.

For information on and to apply for AWS credits in increments of $500 available from ARC-TS, please fill out this form.

For more information:

Email hpc-support@umich.edu

Related Events

December 9 @ 3:00 pm - 4:00 pm

MICDE Seminar: Ann Almgren, Lawrence Berkeley National Lab

Bio: Ann S. Almgren is an applied mathematician who works as a staff scientist and acting group leader of the Center for Computational Sciences and Engineering at the Lawrence Berkeley National…

December 12 @ 1:00 pm - 5:00 pm

HPC User Meetup

Users of high performance computing resources are invited to meet ARC-TS HPC operators and support staff in person. There is not a set agenda; come at anytime and stay as long as…