DokuWiki

User Tools

Site Tools

Trace:
cluster:126

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision Both sides next revision
cluster:126 [2017/06/01 19:46]
hmeij07 [Our Queues] 		cluster:126 [2017/07/12 14:32]
hmeij07
Line 17: Line 17:
   * (old node) ''whitetail'' (Angstrom Blade 1U), Hadoop Cloudera test server   * (old node) ''whitetail'' (Angstrom Blade 1U), Hadoop Cloudera test server
   * (not to be used as) login node ''sharptail'' (Supermicro 4U), /home primary NFS server   * (not to be used as) login node ''sharptail'' (Supermicro 4U), /home primary NFS server
-  * (to be installed summer 2017) ''homedr'' (Supermicro 2U), disaster recovery for /home, off site+  * (to be populated summer 2017) ''sharptail2'' (Supermicro 2U), disaster recovery for /home, off site
   * Storage servers ''rstore0'' and ''rstore2'' (Supermicro 4U), NFS mounts and Samba shares   * Storage servers ''rstore0'' and ''rstore2'' (Supermicro 4U), NFS mounts and Samba shares
  
 Several types of compute nodes are available via the OpenLava scheduler, http://www.openlava.org:  Several types of compute nodes are available via the OpenLava scheduler, http://www.openlava.org: 
  
-  * All are running CentOS6.8, x86_64, Intel Xeon chips (except the Angstrom blades which are AMD Opteron)+  * All are running CentOS6.[4-9], x86_64, Intel Xeon chips (except the Angstrom blades which are AMD Opteron)
   * All are on private networks (no internet)   * All are on private networks (no internet)
   * All mount /home (10TB, to be expanded to 25TB fall 2017) and /sanscratch (33TB)    * All mount /home (10TB, to be expanded to 25TB fall 2017) and /sanscratch (33TB) 
Line 40: Line 40:
   * 14 nodes with dual ten core chips (Xeon E5-2550 v3, 2.3 Ghz) in Supermicro 1U rack servers with a memory footprint of 32 GB each (256 GB). This cluster has a compute capacity of 12 teraflops (estimated). Known as the Microway tinymem cluster, or n46-n59, queue tinymem, 448 job slots.   * 14 nodes with dual ten core chips (Xeon E5-2550 v3, 2.3 Ghz) in Supermicro 1U rack servers with a memory footprint of 32 GB each (256 GB). This cluster has a compute capacity of 12 teraflops (estimated). Known as the Microway tinymem cluster, or n46-n59, queue tinymem, 448 job slots.
  
-All queues are available for job submissions via all login nodes. All nodes on Infiniband switches for parallel computational jobs (excludes bss24 and tinymem queues).  Our total job slot count is roughly 1,040, our physical core count 744. Our total teraflops compute capacity is about 22 cpu side, 23 gpu side. Our total memory footprint is about 100 GB gpu side,  4,976 GB cpu side (excludes queue bss24).+  * 18 nodes with dual twelve core chips (Xeon E5-2650 v4, 2.2 Ghz) in Supermicro 1U rack servers with a memory footprint of 128 GB each (2,304 GB). This cluster has a compute capacity of 14.3 teraflops (estimated). Known as the Microway "Carlos" CPU cluster, or nodes n60-n77, queue mw128, 648 job slots. 
 + 
 +All queues are available for job submissions via all login nodes. All nodes on Infiniband switches for parallel computational jobs (excludes bss24, tinymem and mw128 queues).  Our total job slot count is roughly 1,688, our physical core count 1,176. Our total teraflops compute capacity is about 36 cpu side, 23 gpu side. Our total memory footprint is about 100 GB gpu side,  7,280 GB cpu side (excludes queue bss24).
  
 Home directory file system are provided (via NFS or IPoIB) by the node ''sharptail'' (our file server) from a direct attached disk array. In total, 10 TB of /home disk space is accessible to the users. Node ''greentail'' makes available 33 TB of scratch space at /sanscratch via NFS.  In addition all nodes provide local scratch space at /localscratch (excludes queue tinymem). The Openlava scheduler automatically makes directories in both these scratch areas for each job (named after JOBPID). Backup services for /home are provided via disk-to-disk snapshots from node ''sharptail'' to node ''cottontail'' disk arrays. (daily, weekly, monthly snapshots are mounted read only on ''cottontail'' for self-serve content retrievals) Home directory file system are provided (via NFS or IPoIB) by the node ''sharptail'' (our file server) from a direct attached disk array. In total, 10 TB of /home disk space is accessible to the users. Node ''greentail'' makes available 33 TB of scratch space at /sanscratch via NFS.  In addition all nodes provide local scratch space at /localscratch (excludes queue tinymem). The Openlava scheduler automatically makes directories in both these scratch areas for each job (named after JOBPID). Backup services for /home are provided via disk-to-disk snapshots from node ''sharptail'' to node ''cottontail'' disk arrays. (daily, weekly, monthly snapshots are mounted read only on ''cottontail'' for self-serve content retrievals)
Line 66: Line 68:
 |  mw256fd  |   8  |  256  |  256  | QDR infiniband  | n38-n45 |  CPU  | |  mw256fd  |   8  |  256  |  256  | QDR infiniband  | n38-n45 |  CPU  |
 |  tinymem  |   14  |  32  |  448  | gigabit ethernet  | n39-n59 |  CPU  | |  tinymem  |   14  |  32  |  448  | gigabit ethernet  | n39-n59 |  CPU  |
 +|  mw128  |   18  |  129  |  648  | gigabit ethernet  | n60-n77 |  CPU  |
  
 Some guidelines for appropriate queue usage with detailed page links: Some guidelines for appropriate queue usage with detailed page links:
Line 89: Line 92:
     * nodes have a sataDOM (non spinning 16G USB device on motherboard) for operating system     * nodes have a sataDOM (non spinning 16G USB device on motherboard) for operating system
     * do not use /localscratch on these nodes, no no     * do not use /localscratch on these nodes, no no
 +  * mw128 (bought with faculty startup funds) tailored for Gaussian jobs
 +    * About 2TB /localscratch (Raid 10) on each node
 +    * Priority access for Carlos' group till summer 2020
   * test (swallowtail, petaltail, cottontail2)   * test (swallowtail, petaltail, cottontail2)
     * wall time of 8 hours of CPU usage     * wall time of 8 hours of CPU usage
cluster/126.txt · Last modified: 2023/10/23 19:37 by hmeij07

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
CC Attribution-Share Alike 4.0 International Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki