DokuWiki

User Tools

Site Tools

Trace:
cluster:103

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
cluster:103 [2011/12/21 20:31] – [Best Practices] hmeijcluster:103 [2011/12/22 19:34] (current) – [Submit 2] hmeij
Line 133: Line 133:
  
 On the back end compute nodes, unless specified, the job runs inside your home directory.  That job competes with other activities inside /home.  Compute nodes have two other areas where the jobs could be submitted: /localscratch and /sanscratch.  The former is a local filesystem for each node and should be used if file locking is essential. The later is a filesystem from greentails diskarray (5 TB) served vi IPoIB (that is NFS traffic over fast interconnects switches, the performance should be much better than gigabit ethernet switches).  It is comprised of disks and spindles that are not impacted by what happens on /home.  So we're going to use that. On the back end compute nodes, unless specified, the job runs inside your home directory.  That job competes with other activities inside /home.  Compute nodes have two other areas where the jobs could be submitted: /localscratch and /sanscratch.  The former is a local filesystem for each node and should be used if file locking is essential. The later is a filesystem from greentails diskarray (5 TB) served vi IPoIB (that is NFS traffic over fast interconnects switches, the performance should be much better than gigabit ethernet switches).  It is comprised of disks and spindles that are not impacted by what happens on /home.  So we're going to use that.
 +
 +In the SAS program we add the following lines
 +
 +<code>
 +%let jobpid = %sysget(LSB_JOBID);
 +libname here "/sanscratch/&jobpid";
 +</code>
 +
 +And change this line to use local disks for storage
 +
 +<code>
 +data here.one;
 +</code>
 +
 +In the submission script we change the following
  
   * new submission file with edits   * new submission file with edits
Line 139: Line 154:
   * scheduler creates unique dirs in scratch by JOBPID for you, so we'll stage the job there   * scheduler creates unique dirs in scratch by JOBPID for you, so we'll stage the job there
   * but now we must copy relevant files //to// scratch dir and results back //to// home dir   * but now we must copy relevant files //to// scratch dir and results back //to// home dir
 +
  
 <code> <code>
cluster/103.1324499475.txt.gz · Last modified: by hmeij