This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
cluster:175 [2018/09/22 12:50] hmeij07 [Gromacs] |
cluster:175 [2018/11/29 18:00] (current) hmeij07 |
||
---|---|---|---|
Line 3: | Line 3: | ||
**[[cluster: | **[[cluster: | ||
- | ==== GTX vs P100 & K20 ==== | + | As of |
+ | --- // | ||
+ | The P100 with 12 GB is end of life, replaced by the P100 16 GB or V100 and \\ | ||
+ | The GTX 1080Ti will be replaced by the GTX 2080 (no specs yet and not certified for Amber18, yet)\\ | ||
+ | As of | ||
+ | --- // | ||
+ | New GROMACS performance benchmarks featuring 2x and 4x NVIDIA RTX 2080 GPUs are now available (GTX too). The RTX 2080 graphics card utilizes the new NVIDIA Turing GPU architecture and provides up to 6x the performance of the previous generation. (Exxact newsletter) | ||
- | Comparing these GPUs yields the following | + | |
+ | ==== P100 vs GTX & K20 ==== | ||
+ | |||
+ | ^ ^ P100 ^ GTX ^ K20 ^ ^ | ||
+ | | cores | 3,584 | 3,584 | 2,496 | count | | ||
+ | | mem | 12/16 | 11 | 5 | gb | | ||
+ | | ghz | 2.6 | 1.6 | 0.7 | speed | | ||
+ | | flops | 4.7/ | ||
+ | |||
+ | Comparing these GPUs yields the following | ||
Credits: This work was made possible, in part, through HPC time donated by Microway, Inc. We gratefully acknowledge Microway for providing access to their GPU-accelerated compute cluster. | Credits: This work was made possible, in part, through HPC time donated by Microway, Inc. We gratefully acknowledge Microway for providing access to their GPU-accelerated compute cluster. | ||
Line 30: | Line 45: | ||
</ | </ | ||
+ | |||
+ | Look at these gpu temperatures, | ||
==== Lammps ==== | ==== Lammps ==== | ||
Line 61: | Line 78: | ||
</ | </ | ||
+ | |||
+ | ==== Lammps (PMMA) ==== | ||
+ | |||
+ | Using a material called PMMA (https:// | ||
+ | |||
+ | ^ gpu ^ cpus ^ ns/ | ||
+ | | 1 P100 | 4 | 89 | x4 | 356 | | ||
+ | | 1 GTX | 6 | 90 | x4 | 360 | | ||
+ | | 1 K20 | 6 | 47 | x4 | 188 | | ||
+ | |||
+ | That means the P100 works as well as the GTX. The K20 works at 50% the performance level of the others which is impressive for this old gpu. | ||
+ | |||
+ | |||
+ | |||
==== Gromacs ==== | ==== Gromacs ==== | ||
Line 66: | Line 97: | ||
Gromacs has shown vastly improved performance between versions. v5 delivered about 20 ns/day per K20 server and 350 ns/day on GTX server. v2018 delivered 75 ns/day per K20 server and 900 ns/day on GTX server. A roughly 3x improvement. | Gromacs has shown vastly improved performance between versions. v5 delivered about 20 ns/day per K20 server and 350 ns/day on GTX server. v2018 delivered 75 ns/day per K20 server and 900 ns/day on GTX server. A roughly 3x improvement. | ||
- | On the P100 test node, I could not invoke the multidir option of gromacs (have run it on GTX, weird). The utilization of the gpu drops as more and more gpus are deployed. | + | On the P100 test node, I could not invoke the multidir option of gromacs (have run it on GTX, weird). The utilization of the gpu drops as more and more gpus are deployed. |
< | < | ||
Line 75: | Line 106: | ||
localhost, | localhost, | ||
localhost, | localhost, | ||
- | gmx_mpi mdrun -gpu_id 0123 -ntmpi 0 \ | + | gmx_mpi mdrun -gpu_id 0123 -ntmpi |
-s topol.tpr -ntomp 4 -npme 1 -nsteps 20000 -pin on -nb gpu | -s topol.tpr -ntomp 4 -npme 1 -nsteps 20000 -pin on -nb gpu | ||
Line 83: | Line 114: | ||
| | ||
- | gpu=4 mpi=25 ntomp=4 -npme 1 | + | gpu=4 mpi=25 |
Performance: | Performance: | ||
gpu=3 (same) | gpu=3 (same) | ||
Line 96: | Line 127: | ||
</ | </ | ||
+ | |||
+ | ==== Gromacs 2018.3 ==== | ||
+ | |||
+ | The multidir not running in Gromacs 2018 is a bug in the code clashing with the call MPI_Barrier (communication timing error). | ||
+ | |||
+ | < | ||
+ | |||
+ | # multidir -gpu_id 0123 with four simultaneous gromacs processes | ||
+ | |||
+ | -np 8 -ntomp | ||
+ | 01/ | ||
+ | 02/ | ||
+ | 03/ | ||
+ | 04/ | ||
+ | |||
+ | -np 16 -ntomp | ||
+ | 01/ | ||
+ | 02/ | ||
+ | 03/ | ||
+ | 04/ | ||
+ | |||
+ | multidir -gpu_id 00112233 with eight simultaneous gromacs processes | ||
+ | sharing the gpus, 2 processes per gpu | ||
+ | |||
+ | -np 8 -ntomp | ||
+ | Error in user input: | ||
+ | The string of available GPU device IDs ' | ||
+ | device IDs | ||
+ | |||
+ | </ | ||
+ | |||
+ | That last error when loading multiple processes per gpu is *not* according to their documentation. So the multidir performance is similar to previous single dir performance but still lags GTX performance by quite a bit. Albeit, there is room in utilization rate of the gpus. | ||
+ | |||
+ | |||
+ | ==== What to Buy ==== | ||
+ | |||
+ | * Amber folks: does not matter | ||
+ | * Lammps folks: P100 nodes please, although in hybrid runs does not matter | ||
+ | * Gromacs folks: GTX nodes please | ||
+ | |||
+ | Remember that GTX gpus placed in Data Center voids their warranty. | ||
+ | |||
\\ | \\ |