Explore ARCExplore ARC

File transfers with Globus – GridFTP

By February 24, 2016

Globus GridFTP is a reliable high performance parallel file transfer service provided by many HPC sites around the world. A GridFTP server is available for Flux.

How to use GridFTP

Globus Online is a web front end to GridFTP, this is the recommended way to interact with GridFTP on campus. Globus Online is a web based project hosted off campus. Globus Online accounts are free and your username need not match your campus uniquename.

Globus Connect

Globus Online also allows for simple installation of the GridFTP endpoint for Windows, Mac(OSX), and Linux. These installations while simple are only for a single user on a machine at a time. If you want your cluster or shared data repo to support all users on your systems as a public endpoint like Flux or Nyx your admin needs to install Globus Connect Server (former known as Global Connect Multi-User) — see below for details.

Batch File Copies

A non-standard use of Globus Online is that you can use it to copy files form one location to another on the same cluster. To do this use the same endpoint (umich#flux as an example) for both the sending and receiving machines. Setup the transfer and Globus will make sure the rest happens. The service will email you when the copy is finished.

Command Line GridFTP

There are Command line tools for GridFTP. If you wish to use these contact the FLUX support group. Their use is discouraged.

Globus Connect Server (GCMU)

Globus Connect Server, formerly Globus Connect Multi-User (GCMU), is a full featured GridFTP endpoint that will enable any user on your system to use GridFTP to transfer files from your system to any other GridFTP endpoint. Installation is more complicated than Globus Connect for single users and only support Linux at this time.

This package is required to have your system appear as a public endpoint in Globus Online. Authentication is handled by the campus wide CiLogin server against the Michigan Kerberos password database. It also does not require the cluster to procure its own signed certificates.

  1. Install the globus-connect-server package for your platform as instructed from Globus.
  2. Use this globus-connect-server.conf
  3. Follow the remaining instructions from 1.

Groups on campus who wish to install Globus Connect Server and add their machines to the umich# name space in Globus Online should contact hpc-support@umich.edu.