Using Globus

Globus is a robust, cloud-based, file transfer service designed to move many large files, ranging from 10s of GBs to 10s of TBs. ARC is a Globus Subscription Provider for the U-M community, which allows U-M resources to serve as endpoints or collections for file transfers.

   TO SEARCH THIS USER GUIDE, USE THE COMMAND + F (MAC) OR CTRL + F (WIN) KEYBOARD SHORTCUTS.

About Globus

Globus allows the transfer of data between different storage systems, lab computers, and personal desktops/laptops.

The standard features of Globus include:

  • Transfers faster than SCP/SFTP (usually by a factor of two)
  • Automatic restarts or continuation when transfers are disrupted
  • Background transfers so users need not remain logged in to a system
  • Transfers of large files between your laptop/desktop and servers via Global Connect Personal
  • Transfer of small number of small files between your laptop/desktop via a browser (note: not all server endpoints offer this feature)

U-M’s Globus Provider status adds the following features:

  • Sharing of server directories/folders with non-U-M collaborators who are also Globus users (for transfer/copy purposes, not shared use of server)
  • U-M signed BAA for High Assurance feature for sensitive data
  • Sharing of directories/folders from laptops/desktops via Globus Connect Personal clients and a Globus Plus account upgrade.

For more information, visit the Globus website.

Back To Top

First Time Users Getting Started with Globus

If you are a first time user of Globus, you will need to create an Identity Account. At minimum you will need to setup your identity using the University of Michigan organizational login in order to access U-M systems but there are others that you must add if accessing non-UM systems.

  • Go to https://globus.org and choose “settings” on the left side panel
  • Choose the “Subscription” Tab
  • Search for University of Michigan in the “Find a Subscription” link on the right
  • Choose continue and you will be forward to U-M Weblogin

Step-by-step guide to getting started for researchers

Back To Top

Transfer Data To and From Desktops/Laptops

There are two ways to transfer to and from desktops or laptops. Both can be used, but each is better suited to a range of numbers and sizes of files to transfer. Note that a browser is always involved via globus.org but you might have options as described below with rough recommended guidelines.

Small number of files or small sizes downloaded or uploaded:

Uploads – Once you have navigated to the remote storage Collection directory structure and it supports a direct browser upload you see an “Upload” button link in the center panel.

Downloads – Navigate to the remote storage and directy where the files are located. Highlight and choose download.

Many or large files:

Use Globus Connect Personal (GCP) which allows faster and more reliable transfers. GCP supports simple installation for Windows, Mac, and Linux computers.

If you have a need to share data from your desktop or laptop with research collaborators add “University of Michigan” as your Globus Plus Sponsor in your Globus Account Settings. Note: If possible use ARC provided servers to share data for performance and stability.

Please be sure you have your @umich.edu set as your contact email address in your profile. Note this is not required for normal Collections/Endpoints provided by ARC.

    1. Click on username menu top right
    2. Choose Globus Plus
    3. Choose get Globus Plus or if already a Plus member with another institution choose add another provider
    4. Find and click radio box for “University of Michigan” in the list. Advanced research computing will be notified of your request after clicking Continue

Once your request is approved, Globus Plus will allow you to create shared links to your own Globus Connect Personal client. These are also known as guest collections. Be advised, if you allow write access your local hard drive can be inadvertently filled.

If you intend to handle protected sensitive data please be sure to read that section at the bottom if this document. You also need to have your personal connect endpoint enrolled into our UMich HA (High Availability Sensitive Data) Subscription. At installation time choose the HA option and once endpoint is configured email us at arcts-support@umich.edu and include:

    1. Ask for your Personal endpoint be part of UMich HA subscription
    2. Provide Display Name and UUID of your endpoint

Back To Top

ARC Collections and Endpoints

ARC Collections and Endpoints

If your storage volume is not listed on the services Globus collection below open a request to arcts-support@umich.edu with your service and volume name to have it added.

 

ARC Service Globus Name
Great Lakes cluster umich#greatlakes
Lighthouse cluster umich#lighthouse
Armis2 cluster umich#armis2 v2
Sensitive Data Turbo Volumes UMich ARC Sensitive Turbo Volume Collection
Non-Sensitive Data Turbo Volumes UMich ARC Non-Sensitive Turbo Volume Collection
Non-Sensitive Data Locker Volumes UMich ARC Non-Sensitive Locker Volume Collection
Non-Sensitive Data Den UMich ARC Non-Sensitive Data Den Volume Collection
Sensitive Data Den UMich ARC Sensitive Data Den Volume Collection

Search for the collections when you wish to transfer files among endpoints, including your own Personal Connect Endpoint. You can bookmark these for latter reuse without having to search.

Notice: Starting in December 2021, the University of Michigan will no longer be offering Box at U-M as a storage service. Please visit the Box at U-M Retirement Project site for the latest updates and information.

ARC makes the Box connector available to assist migration of data to other U-M resources. Most users should wait for communications from the Retirement Project to automate their migration. The Globus Box connector is of most interest to those migrating to non-default options provided by the retirement project. In most cases under consultation with their Box Retirement Ambassador.

Contact: Box-Retirement-Project@umich.edu

Back To Top

Sensitive Data with Globus

Users are responsible for security and compliance related to sensitive code and/or data. Security and compliance are shared responsibilities. If you process or store sensitive university data, software, or libraries on your storage volume, you are responsible for understanding and adhering to any relevant legal, regulatory or contractual requirements.

SENSITIVE DATA TURBO COLLECTION TERMS OF USAGE

  1. This service is for sensitive data only. Be advised that you should not move sensitive data off of this system, unless it is to another service or machine that has been approved for hosting the same types of sensitive data.
  2. It is your responsibility, not ARC’s, to be aware of and comply with all applicable laws, regulations, and universities policies (e.g., ITAR, EAR, HIPAA) as part of any research activity that may raise compliance issues under those laws. For assistance with export controlled research, contact the U-M Export Control Officer at exportcontrols@umich.edu. For assistance with HIPAA-related computational research, contact the ARC liaison to the Medical School.
  3. Please review on a regular basis the export settings for sensitive data volumes.
  4. Please review on a regular basis any shares (Guest Collections) you may have made to ensure those collaborators should still have access.

SENSITIVE DATA GLOBUS CONNECT PERSONAL

Users using Globus Connect Personal (GCP) with sensitive data are required to enable that endpoint as Globus High Assurance (HA). This will require registering it with the campus Globus agreement.

  1. Select High Assurance when installing GCP
  2. Visit globus.org -> Collections -> Administered by You -> Select endpoint -> Edit Attributes
  3. Set Visible To Public (This is required until you hear back from ARC)
  4. Email arc-support@umich.edu your Endpoint UUID from the endpoint overview page

Back To Top

Globus and MiStorage

ITS Mistorage customers can request their storage volume be available via Globus by making a request to ITS via an email 4help@umich.edu or call 4-HELP (734-764-4357). Note that currently ony NFS volumes are supported with Globus.

Include:

  • Volume name
  • List of all users who need access via Globus
  • Indicate if Sharing is to be enabled

The MiStorage collection name is: umich#ITS-Storage

Back To Top

Globus for Admins

ARC is a licensed subscriber for Globus that allows the entire University community to take advantage of enhanced features which include enabling Sharing or Guest collections on Personal and Server endpoints.

To enable sharing of an existing endpoint, ARC can add the endpoint to our management subscription. The system administrator or owner must email arcts-support@umich.edu to ask that the endpoint be put under the U-M subscription to enable sharing. Note that to be joined into the subscription the endpoint must be made public. After being added into the subscription, it can be set back to private if desired.

The request must include:

endpoint name = xxxx#yyyy , displayname or uid

Globus offers two distinct versions of the Connect Server v4 and v5, the procedures to install and configure each are considerable different.

GCS v4 is simpler and all file systems are behind a single endpoint:

To enable “Sharing” the system administrator should:

edit the configuration file at /etc/globus-connect-server.conf to set the following option:

[GridFTP]
Sharing = True

review the configuration file for other relevant parameters (for example, consider setting “SharingRestrictPaths’ to restrict what users can share.

run GCS setup command again to read new configuration: # globus-connect-server-setup GCS v5 has enhanced management and control features that System Admins may want to take advantage of. These include but not limited to finer degree of separation of filesystems via what are called Gateways and Collections.  For example it may be beneficial to make available local file systems as a separate offering from network file systems. Configuration of GCS v5 is considerable more complicated and it is highly recommended that Systems Admin read and thoroughly understand all documentation before proceeding.

Back To Top

.