Several levels of data storage are provided for the HPC clusters, varying by capacity, I/O rate, and longevity of storage. Nothing is backed up by default, except AFS. Turbo can have duplication enabled upon request. Contact with questions.

Storage type / location Description Best Used For Access and Policy Details
Local directory unique to each node. Not shared. High-speed read and writes for small files (less than 10GB) System will automatically delete data here 30 days after date of creation
Shared across the entire cluster. Only for use with currently running jobs. Quota of 80GB per user on the Flux and Beta clusters; 10GB on Armis Currently running jobs. To check your quota on the Flux cluster, use flux-userquota -sOn the other clusters, run du -sh .  to see your usage
Lustre-based parallel file system shared across all Flux nodes. /scratch is temporary storage only, and is purged periodically by ARC-TS admins. Large reads and writes of very large data files. Checkpoint/restart files and large data sets that are frequently read from or written to are common examples. Also, code that uses MPI. ARC-TS /scratch page
AFS is a filesystem maintained and backed up by ITS. It is the only storage option available for Flux that is regularly backed up, and is therefore the most secure choice. It is only available on Flux login nodes and can provide up to 10GB of backed-up storage. Storing important files. NOT available for running jobs on compute nodes. ARC-TS AFS page
Turbo Research Storage is a high-speed storage service providing NFSv3 and NFSv4 access. It is available only for research data. Data stored in Turbo can be easily shared with collaborators when used in combination with the Globus file transfer service. Storing research data. ARC-TS Turbo page
Long-term storage
Users who need long-term storage can purchase it from ITS MiStorage. Once established, it can be mounted on the Flux login and compute nodes. Long-term storage. ITS MiStorage page