Quotas
On 03/25/2011 quotas have been enabled on cluster greentail. Because group quota numbers are weird, it is like counting duplicates or following links or looping, not sure, we have to rely on individual quotas. Just as well.
Here is what was put in place, and will mature/change over time. But first, it is important to understand:
- A quota is not disk space allocated for you, as in reserved. It is simply a threshold (soft/low) that can be adjusted, or not.
- Reason for quotas: to avoid /home from filling up by a runaway process, and constrain large users from growing beyond arbitrary thresholds.
The quotas fall into two tiers: the “sliding” tier and the “custom” tier.
- In the sliding tier, when you reach your soft limit, notify HPC admins and your quota will slide to the next strata (< 90/100G, < 225/250G, < 450/500G). Once you exceed the last strata, and you still need more disk space, you enter the custom tier.
- In the custom tier, we discuss setting a quota after agreeing on size and availability of disk space, plus 100G. Then you are on your own. (Users that currently fall into this group have been grandfathered in and notified).
We urge all users to consider archiving data and files off the cluster disk array. Any quota can ofcourse shrink, please let us know if you move files off the array and free up disk space for others.
How to find my quota? You can find your quota levels and current usage as follows. Units are in kilobytes. So the blocks soft limit (quota) below is 90 GB, the hard limit (limit) is 100 GB.
- The hard block limit is the absolute maximum amount of disk space that a user can use. Once this limit is reached, no additional disk space can be used. Writes to disks will fail.
- The soft block limit defines the maximum amount of disk space that can be used. Unlike the hard limit, the soft limit can be exceeded for a certain amount of time. That time is known as the grace period. Warnings will appear in your terminal/shell.
- The grace period is globally set for 7 days.
- We do not enforce inode limits.
[hmeij@greentail ~]$ quota -v hmeij
Disk quotas for user hmeij (uid 8216):
Filesystem blocks quota limit grace files quota limit grace
/dev/mapper/vghome-lvhome
1227284 90000000 100000000 1020 0 0
Notes
Implementation
- add usrquota,grpquota options to fstab, remount
- log everybody out or reboot (quota error on boot, not initialized yet)
- quotaoff
- quotacheck -cvug /home (file system goes read only! 10TB of whcih 6TB used, took 3 hours)
- set default user quota and propagate to all users
- set custom quotas for other users
- wrap adduser
- quotaon
- group quota numbers do not add up !?
Array
- 48 one TB disk minus 4 global spares leave 44 disks
- after formatting 36 TB available (81%)
- /home 12 disks, /snapshots 12 disks, /sanscratch 24 disks (presents a 5 TB /sanscratch volume)
- leaves roughly 11 disks free in /sanscratch, needed for /snapshot growth
- snapshots holds base plus deltas hence needs to be somewhere between 1-2x base
- so if we wanted to increase home, we delete sanscratch volume (is anybody using it?)
- if we do not backup, then … questions for later
Links
- quotacheck invocation, file system goes read only
