cluster:126
Warning: Undefined array key -1 in /usr/share/dokuwiki/inc/html.php on line 1458
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
cluster:126 [2020/02/27 09:40] hmeij07 |
cluster:126 [2023/10/23 15:37] (current) hmeij07 |
||
|---|---|---|---|
| Line 8: | Line 8: | ||
| ===== Description ===== | ===== Description ===== | ||
| - | The High Performance Compute Cluster (HPCC) is comprised of several login nodes (all are on our domain | + | The High Performance Compute Cluster (HPCC) is comprised of several login nodes (all are on our internal network (vlan 52) |
| - | * primary login node '' | + | * server |
| - | * secondary | + | * primary |
| - | * secondary login node '' | + | * zenoss monitoring and alerting server '' |
| - | * sandbox '' | + | * secondary login server '' |
| - | * sandbox '' | + | * server '' |
| - | * zenoss monitoring and alerting server '' | + | * server |
| - | * NFS server '' | + | * storage servers '' |
| - | * (only log in when moving conternt) file server | + | * storage servers '' |
| - | * DR node '' | + | * storage servers '' |
| - | * storage servers '' | + | * storage server '' |
| - | * storage servers '' | + | |
| - | * mindstore | + | |
| Several types of compute nodes are available via the scheduler: | Several types of compute nodes are available via the scheduler: | ||
| - | * All are running | + | * All are running |
| * All are x86_64, Intel Xeon chips from 2006 onwards | * All are x86_64, Intel Xeon chips from 2006 onwards | ||
| - | * All are on private networks (192.168.x.x or 10.10.x.x, no internet) | + | * All are on private networks (192.168.x.x |
| - | * All mount /home (10TB, to be replaced by FreeNAS/ZFS 190T appliance 2020) and /sanscratch (xfs,55TB) | + | * All mount /zfshomes |
| * All have local disks providing varying amounts of / | * All have local disks providing varying amounts of / | ||
| * Hyperthreading is on but only 50% of logical cores allocated via scheduler | * Hyperthreading is on but only 50% of logical cores allocated via scheduler | ||
| Line 48: | Line 46: | ||
| * 12 nodes with dual twelve core chips (Xeon Silver 4214, 2.20 Ghz) in ASUS ESC4000G4 2U rack servers with a memory footprint of 96 GB ( 1,152 GB, about 20 teraflops | * 12 nodes with dual twelve core chips (Xeon Silver 4214, 2.20 Ghz) in ASUS ESC4000G4 2U rack servers with a memory footprint of 96 GB ( 1,152 GB, about 20 teraflops | ||
| - | All queues are available for job submissions via all login nodes. Some nodes on Infiniband switches for parallel computational jobs (queues: me256fd, hp12). | + | * 2 nodes with dual twelve core chips (Xeon 4214R “Cascade Lake Refresh” |
| - | Home directory file system are provided | + | * 6 nodes with dual 28 core chips (Xeon Gold 'Ice Lake-SP' |
| - | Two (old) Rstore storage servers each provide | + | All queues are available for job submissions via all login nodes. Some nodes on Infiniband switches for parallel computational jobs (queues: mw256fd, hp12, mw256). Our total job slot count is roughly 2,144 with our physical core count 1,480. Our total teraflops compute capacity is about 58 cpu side, 25 gpu side (double precision floating point) and 702 gpu side (mixed mode). Our total memory footprint is about 560 GB gpu side, 10,452 GB cpu side. |
| + | |||
| + | Home directory file system are provided (via NFS or IPoIB) by the node '' | ||
| ===== Our Queues ===== | ===== Our Queues ===== | ||
| - | Commercial software has their own queue limited by available licenses. | + | There are no scheduler |
| ^Queue^Nr Of Nodes^Total GB Mem Per Node^Total Cores In Queue^Switch^Hosts^Notes^ | ^Queue^Nr Of Nodes^Total GB Mem Per Node^Total Cores In Queue^Switch^Hosts^Notes^ | ||
| Line 66: | Line 66: | ||
| ^Queue^Nr Of Nodes^Total GB Mem Per Node^Job SLots In Queue^Switch^Hosts^Notes^ | ^Queue^Nr Of Nodes^Total GB Mem Per Node^Job SLots In Queue^Switch^Hosts^Notes^ | ||
| - | | hp12 | | + | | hp12 | |
| | mwgpu | | | mwgpu | | ||
| | mw256fd | | mw256fd | ||
| Line 73: | Line 73: | ||
| | amber128 | | amber128 | ||
| | exx96 | | | exx96 | | ||
| + | | test | | ||
| + | | mw256 | | ||
| Some guidelines for appropriate queue usage with detailed page links: | Some guidelines for appropriate queue usage with detailed page links: | ||
| Line 78: | Line 80: | ||
| * hp12 is the default queue | * hp12 is the default queue | ||
| * for processing lots of small to medium memory footprint jobs | * for processing lots of small to medium memory footprint jobs | ||
| - | * mwgpu is for GPU enabled software primarily (Amber, Lammps, NAMD, Gromacs, Matlab, Mathematica) | + | * mwgpu is for GPU (K20) enabled software primarily (Amber, Lammps, NAMD, Gromacs, Matlab, Mathematica) |
| * be sure to reserve one or more job slot for each GPU used [[cluster: | * be sure to reserve one or more job slot for each GPU used [[cluster: | ||
| * be sure to use the correct wrapper script to set up mpirun from mvapich2, mpich3 or openmpi | * be sure to use the correct wrapper script to set up mpirun from mvapich2, mpich3 or openmpi | ||
| Line 97: | Line 99: | ||
| * Be sure to use mpich3 for Amber | * Be sure to use mpich3 for Amber | ||
| * Priority access for Amber jobs till 10/01/2020 | * Priority access for Amber jobs till 10/01/2020 | ||
| - | * test (swallowtail, petaltail, cottontail2) | + | |
| - | * wall time of 8 hours of CPU usage | + | * same setup as mwgpu queue |
| + | | ||
| + | * can be used for production runs | ||
| + | * beware of preemptive events, checkpoint! | ||
| + | * mw128, NFSoRDMA, bought with faculty startup monies | ||
| + | * beware | ||
| + | * 6 compute nodes | ||
| + | * Priority access for Sarah' | ||
| - | **There are no wall time limits in our HPCC environment except for queue '' | + | **NOTE**: we are migrating from Openlava to Slurm during summer 2022. All queues except '' |
| + | |||
| + | * [[cluster: | ||
| + | * [[cluster: | ||
| + | |||
| + | **There are no wall time limits in our HPCC environment except for queue '' | ||
| ===== Other Stuff ===== | ===== Other Stuff ===== | ||
| Line 109: | Line 123: | ||
| For a list of software installed consult [[cluster: | For a list of software installed consult [[cluster: | ||
| + | |||
| + | For a list of OpenHPC software installed consult [[cluster: | ||
| Details on all scratch spaces consult [[cluster: | Details on all scratch spaces consult [[cluster: | ||
| Line 114: | Line 130: | ||
| For HPCC acknowledgements consult [[cluster: | For HPCC acknowledgements consult [[cluster: | ||
| - | Sample scripts for job submissions (serial, array, parallel, forked and gpu) can be found at ''/ | + | Sample scripts for job submissions (serial, array, parallel, forked and gpu) can be found at ''/ |
| - | From off-campus you need to VPN in first at [[http:// | + | From off-campus you need to VPN in first, download GlobalProtect client |
cluster/126.1582814426.txt.gz · Last modified: 2020/02/27 09:40 by hmeij07
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
