DokuWiki

User Tools

Site Tools

Trace:
cluster:182

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
Next revision Both sides next revision
cluster:182 [2019/08/12 17:20]
hmeij07 [Lammps] 		cluster:182 [2019/08/12 17:50]
hmeij07 [Gromacs]
Line 88: Line 88:
 ==== Gromacs ==== ==== Gromacs ====
  
 +Gromacs was build on each of the nodes locally letting it select the optimal CPU (AVX, SSE) and GPU accelerators. The ''cmake'' flag -DGMX_BUILD_OWN_FFTW=ON yields a mixed precision compilation which is recommended. The we ran multidir options 01-04 on single GPU, and 01-08 and 01-16 on all 4 GPUs when possible.
  
  
 +^  ns/day  ^  P100[1]  ^  P100[4]  ^  RTX[1]  ^  T4[1]  ^  T4[4]  ^  Notes  ^
 +|  Mixed  |  |  |  254|  |  |  gpu=1, 01-04  |
 +|  Mixed  |  |  551|  |  |  546|  gpu=4, 01-04  |
 +|  Mixed  |  |  |  |  |  650|  gpu=4, 01-08  |
 +|  Mixed  |  |  |  |  |  733|  gpu=4, 01-16  |
  
-^  tau/day  ^  P100[1]  ^  P100[4]  ^  RTX[1]  ^  T4[1]  ^  T4[4]  ^  Notes  ^ +The T4 is P100's equal in mixed precision performance. Add the wattage factor and you have a favorite.
-|  Mixed  |  981,897|  ?|  916,225|  881,247|  2,294,344|  |+
  
 ==== Scripts ==== ==== Scripts ====
Line 156: Line 161:
  
 </code> </code>
 +
 +  * Gromacs
 +
 +<code>
 +
 +#!/bin/bash
 +#SBATCH --nodes=1
 +#SBATCH --nodelist=node9
 +#SBATCH --job-name="T4 dd"
 +#SBATCH --ntasks-per-node=32
 +#SBATCH --gres=gpu:4
 +#SBATCH --exclusive
 +
 +export PATH=$HOME/gromacs-2018/bin:$PATH
 +export LD_LIBRARY_PATH=$HOME/gromacs-2018/lib:$LD_LIBRARY_PATH
 +. $HOME/gromacs-2018/bin/GMXRC.bash
 +rm -f gpu/??/c* gpu/??/e* gpu/??/s* gpu/??/traj* gpu/??/#* gpu/??/m*
 +cd gpu
 +
 +# T4
 +#export CUDA_VISIBLE_DEVICES=0123
 +
 +mpirun -np 8 gmx_mpi mdrun -maxh 1 -gpu_id 0123 \
 +-nsteps 1000000 -multidir 05 06 07 08 \
 +-ntmpi 0 -npme 0 -s topol.tpr -ntomp 0 -pin on -nb gpu
 +
 +</code>
 +
 +And GPU utilization was outstanding.
 +
 +[heme@login1 gromacs-2018]$ ssh node9 ./gpu-info\\
 +id,name,temp.gpu,mem.used,mem.free,util.gpu,util.mem\\
 +0, Tesla T4, 66, 866 MiB, 14213 MiB, 98 %, 9 %\\
 +1, Tesla T4, 67, 866 MiB, 14213 MiB, 98 %, 9 %\\
 +2, Tesla T4, 66, 866 MiB, 14213 MiB, 99 %, 9 %\\
 +3, Tesla T4, 64, 866 MiB, 14213 MiB, 97 %, 9 %\\
 +
 \\ \\
 **[[cluster:0|Back]]** **[[cluster:0|Back]]**
cluster/182.txt · Last modified: 2019/12/13 13:33 by hmeij07

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
CC Attribution-Share Alike 4.0 International Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki