/scratch

Description of File System:

/scratch is mounted on all worker nodes. The file system is intended to provide extremely high I/O throughput to or from disk. Users are granted a default quota of 100GB in /scratch and are requested to apply for additional storage by e-mailing the HPC administrators. Quotas are implemented for file system administration. This is the preferred place to put any intermediate files required while a job is executing. Users are expected to move their data from /scratch to long term storage as part of their workflow. /scratch is purposely built as a computational work space area and is not intended for long term storage. This file system is not backed up.

General Usage:

This file system is designed to provide high throughput to disk. We encourage users who have large datasets to upload them directly to /scratch, not /home. The output of all computation should also be directed to /scratch. Data that you wish to keep must be removed from /scratch. Please note that you may only store 20GB of data on /home.

File System Backup:

The /scratch file system is not backed up. Any data which is lost cannot be recovered. Ensure that your data is moved off the file system when jobs complete.  Code, scripts, libraries and other data that cannot easily be replicated should not be stored in /scratch.

File System Quota Policy:

The /scratch file system is shared space and is intended for temporary processing, not long term storage. Should the /scratch file system run out of space all users are affected. It is your responsibility to move processed data off the cluster. There are no backups of /scratch

QuotaLifespan (days)
100GB (default)indefinite
500GBindefinite
1TB12 months by agreement
5TBeResearchdata or by agreement
Over 5TBeResearchdata

Quota:  Storage allocation is applied to a user on the /scratch file system. These are not directory quotas, they are user quotas applied to the entire file system.

Inode Limits:  We have implemented a default chunk file (inode) quotas. This means you can effectively create 5,000 files/folders in /home and 10,000,000 files/folders in /scratch. If you need to create more files please let us know.

Long term storage:  Users that require reliable long term central storage should contact eresearch@uct.ac.za

File System Quota Check:

Users are encouraged to check their quotas by executing the following command:

myquota

This will produce two results, your quota in /home and /scratch for example:

=====================================================================================
QUOTA IN /home/jsmith
Quota information for storage pool Default (ID: 1):

        user/group   ||           size          ||    chunk files
         name |  id  ||       used | hard       ||    used | hard
--------------|------||------------|------------||---------|---------
        jsmith| 1000 ||  5.82 GiB  |  20.00 GiB ||    74685| 1000000

=====================================================================================

QUOTA IN /scratch/jsmith
Quota information for storage pool Default (ID: 1):

         user/group  ||           size          ||    chunk files
         name |   id ||       used | hard       ||    used | hard
--------------|------||------------|------------||---------|---------
        jsmith| 1000 || 29.69 GiB  | 100.00 TiB ||   22647 | 10000000

=====================================================================================

You should pay attention to the chunk files (inodes) parameter. This indicates the number of files that you may create, this is also a limit. If you anticipate creating very large numbers of files please inform the system administrator.

Menu