DokuWiki

User Tools

Site Tools

Trace:
cluster:93

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
cluster:93 [2011/01/11 16:43]
hmeij 		cluster:93 [2011/01/11 20:55] (current)
hmeij
Line 112: Line 112:
 However, since our fantastic crash of June 2008 ([[cluster:67|The catastrophic crash of June 08]] page) /sanscratch was simply a directory inside /home and thus competes for disk space and IO. However, since our fantastic crash of June 2008 ([[cluster:67|The catastrophic crash of June 08]] page) /sanscratch was simply a directory inside /home and thus competes for disk space and IO.
  
-On greentail, /sanscratch will be a separate logical volume of 5 TB using a different disk set.  So I urge those that have very large files, or generate lots of IO, to stage their files in /sanscratch when running their jobs for best performance.  The scheduler will always create (and delete!) two directories for you.  The JOBPID of your job is used to create /localscratch/jobpid and /sanscratch/jobpid.+On greentail, /sanscratch will be a separate logical volume of 5 TB using a different disk set.  So I urge those that have very large files, or generate lots of IO, to stage their files in /sanscratch/jobid when running their jobs for best performance.  The scheduler will always create (and delete!) two directories for you.  The JOBPID of your job is used to create /localscratch/jobpid and /sanscratch/jobpid.
  
 ===== MPI ===== ===== MPI =====
Line 118: Line 118:
 For those of you running MPI or MPI enabled applications, you will need to make some changes to your scripts.  The ''wrapper'' program to use with greentail's Lava scheduler is the same as for cluster sharptail. It can be found here:  /share/apps/bin/lava.openmpi.mpirun.   If other flavors are desired, you can inform me or look look at the example scripts lava.//mpi_flavor//.mpi[run|exec]. For those of you running MPI or MPI enabled applications, you will need to make some changes to your scripts.  The ''wrapper'' program to use with greentail's Lava scheduler is the same as for cluster sharptail. It can be found here:  /share/apps/bin/lava.openmpi.mpirun.   If other flavors are desired, you can inform me or look look at the example scripts lava.//mpi_flavor//.mpi[run|exec].
  
-Sometime ago I wrote some code to detect if a node is infiniband enabled or not, and based on the result, add command line arguments to the mpirun invocation.  If you use that code, you will need to change:  the path to obtain the port status (/usr/bin/ibv_devinfo) and in the block specify the interface change eth1 to ib0.+Sometime ago I wrote some code to detect if a node is infiniband enabled or not, and based on the result, add command line arguments to the mpirun invocation.  If you use that code, you will need to change:  the path to obtain the port status (/usr/bin/ibv_devinfo) and in the block specifing the interface change eth1 to ib0. 
 + 
 +===== Software ===== 
 + 
 +The same /share/apps software directory build when we were working petaltail as a new administrative host has been copied to greentail.  Petaltail is redhat linux 5.1 while greentail is redhat linux 5.5.  I anticipate everything to work but we may encounter missing libraries (major/minor versions).
  
 ===== ... ===== ===== ... =====
cluster/93.1294764237.txt.gz · Last modified: 2011/01/11 16:43 by hmeij

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
CC Attribution-Share Alike 4.0 International Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki