SuperCloud Disk Space Limits
The SuperCloud has imposed guardrails for disk space and number of files on both home and group directories for these reasons:
- To make sure no one unintentionally uses up a lot of storage
- So we can have a conversation on what your data needs are when your requirements start to grow
- Large numbers of files have a significant impact on the overall performance of the filesystem
The current guardrails are listed when you run the command to list your disk usage (see below). If you need to have these guardrails relaxed temporarily, email us at supercloud@mit.edu so we can discuss your needs.
It is recommended that you not use your SuperCloud home directory as primary storage. You should transfer data from your local workstation to your SuperCloud home directory before processing, and transfer the results back for analysis.
Checking your Disk Usage
You can check your disk usage by going to the Profile page on the Web Portal. The bottom of this page lists the disk usage in your home directory and in any group directories you may be a part of.
You can also use this command to check the disk usage in your home directory:
We recognize that during the prototyping and data analysis phases, a lot of intermediate results may be generated and which can accumulate. We ask you to be judicial in your storage use and to periodically do data "spring cleaning"!s
Please also see our tips for Improving File System Performance.
We do not backup files on SuperCloud. We strongly recommend that you regularly move files back to your local workstation for backup.