This is an old revision of the document!
The HPCC cluster's file server
sharptail.wesleyan.edu serves out all home directories to all nodes at location /home. It is 10 TB in size and it currently takes the nightly process of backup 1-2 hours to churn through. Making it larger would thus generate more traffic on /home. So we've, for now while it works for us, come up with this policy:
At this point users need to off load static content to other locations. Contents like old analyses, results of published papers, etc. Users typically one local option available:
Users whom are considered inactive have their home directories relocated to /archives/inactive
The remote option, if your storage needs cannot be supported by /archives, is off-cluster storage. Rstore is our latest storage solution for groups and labs which such needs.
Well, move stuff around? Try to avoid programs such as cp, sftp/scp for large contents migrations. The better bet is
man rsync for the manual page.
With rsync you can:
rsync -vac –dry-run
* and much more … So to put it all together, move my directory in my hom edirectory named stuff elsewherersync –vac –delete –bwlimit=10000 –dry-run /home/username/stuff rstore0:/data/2/somelabgroup/mydirecotory/''
Note the lack of source trailing slash but present destination trailing slash; meaning put source inside destination location. If both had a trailing slash it would mean; update source and target at these locations.