Storage¶
Several different locations are available for file storage for each cluster. Each filesystem has unique performance and backup characteristics and is optimized for certain kinds of use.
Locations¶
Location |
Filesystem Type |
Quota |
Backup |
Use Case |
|---|---|---|---|---|
|
Standard NFS |
8 GiB |
Yes |
Config files, small scripts |
|
Lustre |
1.3 PiB (center-wide) |
No |
MSU user and project files, job I/O |
|
Lustre |
Per project |
No |
NOAA project files, job I/O |
|
Lustre |
Per project |
No |
NOAA project files, job I/O |
|
Lustre |
None |
Purged |
Temporary files |
NOAA projects are provided space under either /work or /work2 as determined by RDHPCS. To request quota increases, the project PI or manager should contact RDHPCS at rdhpcs.orion.help@noaa.gov.
MSU users may request space on /work. To request space, email help@hpc.msstate.edu with the cluster name, the user or project name, a justification, and the storage allocation size you need. MSU users are provided space under /scratch/orion/users automatically, however note that this space is subject to regular purges. Shared project scratch directories can be created upon request and with justification.
MSU user and project files follow the structure outline in this table:
Directory Type |
Path |
|---|---|
MSU Project |
|
MSU User |
|
MSU Project Scratch |
|
MSU User Scratch |
|
Location |
Filesystem Type |
Quota |
Backup |
Use Case |
|---|---|---|---|---|
|
Standard NFS |
8 GiB |
Yes |
Config files, small scripts |
|
Lustre |
2 PiB (center-wide) |
No |
MSU user and project files, job I/O |
|
Lustre |
Per project |
No |
NOAA project files, job I/O |
|
Lustre |
Per project |
No |
NOAA project files, job I/O |
|
Lustre |
None |
Purged |
Temporary files |
NOAA projects are provided space under either /work or /work2 as determined by RDHPCS. To request quota increases, the project PI or manager should contact RDHPCS at rdhpcs.hercules.help@noaa.gov.
MSU users may request space on /work. To request space, email help@hpc.msstate.edu with the cluster name, the user or project name, a justification, and the storage allocation size you need. MSU users are provided space under /scratch/hercules/users automatically, however note that this space is subject to regular purges. Shared project scratch directories can be created upon request and with justification.
MSU user and project files follow the structure outline in this table:
Directory Type |
Path |
|---|---|
MSU Project |
|
MSU User |
|
MSU Project Scratch |
|
MSU User Scratch |
|
Location |
Filesystem Type |
Quota |
Backup |
Use Case |
|---|---|---|---|---|
|
GPFS |
8 GiB |
No |
Config files, small scripts |
|
GPFS |
Per Project or User |
No |
User and project files, job I/O |
|
GPFS |
None |
Purged |
Temporary files |
|
GPFS |
None |
No |
Reference datasets for classes and research groups |
Users may request space on /work. To request space, email help@hpc.msstate.edu with the cluster name, the user or project name, a justification, and the storage allocation size you need. Users are provided space under /scratch/ptolemy/users automatically, however note that this space is subject to regular purges. Shared project scratch directories and reference directories can be created upon request and with justification.
User, project, and reference files follow the structure outline in this table:
Directory Type |
Path |
|---|---|
Project |
|
User |
|
Project Scratch |
|
User Scratch |
|
Class Reference |
`/reference/ptolemy/class/ |
Examples |
`/reference/ptolemy/examples/ |
Other Reference |
`/reference/ptolemy/ |
Location |
Filesystem Type |
Quota |
Backup |
Use Case |
|---|---|---|---|---|
|
VastFS NFS |
30 GB |
No |
Config files, small scripts |
|
VastFS NFS |
Per project |
No |
Project files, job I/O |
|
VastFS NFS |
None |
Purged |
Temporary files |
|
VastFS NFS |
None |
No |
Reference datasets |
Project are automatically populated under the /project and /90daydata directories. To request a quota increase for an existing SCINet project, see the SCINet Request Resources page.
The /reference directory is a nightly sync from ceres.scinet.usda.gov:/reference/data.
Location |
Filesystem Type |
Quota |
Backup |
Use Case |
|---|---|---|---|---|
|
VastFS NFS |
12 GB |
Yes |
Config files, small scripts |
|
VastFS NFS |
Per project |
No |
User and project files, job I/O |
|
VastFS NFS |
Node |
Purged |
Temporary files |
Users may request space on /work. To request space, email help@hpc.msstate.edu with the cluster name, the user or project name, a justification, and the storage allocation size you need. Users are provided space under /scratch/morrill/users automatically, however note that this space is subject to regular purges. Shared project scratch directories and dataset directories can be created upon request and with justification.
User, project, and dataset files follow the structure outline in this table:
Directory Type |
Path |
|---|---|
Project |
|
User |
|
Datasets |
|
Project Scratch |
|
User Scratch |
|
Important
DO NOT run jobs in /home due to the small quotas and often poor performance for parallel I/O.
Important
Long term storage of data should not be on cluster storage as most directories are not backed up and some directories are subject to regular purges. Data should be moved off of the cluster when not in use.
Warning
DO NOT touch files to prevent purging of files. Attempting to circumvent purging is a violation of ARC policies.
For convenience, three environment variables are set on each cluster (except for Atlas):
Variable |
Location |
|---|---|
|
|
|
|
|
|
Quota Management¶
Important
Avoid filesystem traversal commands (find, du -s *, etc.) on large directories.
These commands can degrade performance for all users by consuming shared I/O resources.
Some storage locations have user-available mechanisms to check quotas.
Location |
Command |
|---|---|
|
|
|
|
|
|
For all locations not listed, there is not a user-available mechanism to determine quotas at this time. For all locations, if you need to identify files contributing to your quota or locate large files:
NOAA Users: contact the RDHPCS team at rdhpcs.orion.help@noaa.gov.
MSU Users: contact the helpdesk at help@hpc.msstate.edu.
The support team can run these operations efficiently without impacting cluster performance.
Location |
Command |
|---|---|
|
|
|
|
|
|
For all locations not listed, there is not a user-available mechanism to determine quotas at this time. For all locations, if you need to identify files contributing to your quota or locate large files:
NOAA Users: contact the RDHPCS team at rdhpcs.hercules.help@noaa.gov.
MSU Users: contact the helpdesk at help@hpc.msstate.edu.
The support team can run these operations efficiently without impacting cluster performance.
Ptolemy does not have a user-available mechanism at this time. If you need to identify files contributing to your quota or locate large files, contact the helpdesk at help@hpc.msstate.edu. The support team can run these operations efficiently without impacting cluster performance.
Location |
Command |
|---|---|
|
|
For all locations not listed, there is not a user-available mechanism to determine quotas at this time. For all locations, if you need to identify files contributing to your quota or locate large files, contact VSRC at help-usda@hpc.msstate.edu. The support team can run these operations efficiently without impacting cluster performance.
Morrill does not have a user-available mechanism at this time. If you need to identify files contributing to your quota or locate large files, contact the helpdesk at help@hpc.msstate.edu. The support team can run these operations efficiently without impacting cluster performance.
Further Reading¶
Topic |
Description |
|---|---|
Moving data to and from the cluster |
|
High-performance storage details |