This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
cluster:214 [2022/06/23 18:25] hmeij07 [Amber20] |
cluster:214 [2022/08/03 14:05] hmeij07 |
||
---|---|---|---|
Line 460: | Line 460: | ||
Amber cmake download fails with READLINE error ... package readline-devel needs to be installed to get past that which pulls in | Amber cmake download fails with READLINE error ... package readline-devel needs to be installed to get past that which pulls in | ||
- | ** Example script run.rocky for cpu or gpu run** (not for queues | + | ** Example script run.rocky for cpu or gpu run** (for queues |
< | < | ||
Line 531: | Line 531: | ||
</ | </ | ||
- | The script amber.sh was converted to a module like so | + | ** Example script run.centos for cpus or gpu run** (queues mwgpu, exx96) |
+ | |||
+ | < | ||
+ | # | ||
+ | # [found at XStream] | ||
+ | # Slurm will IGNORE all lines after the FIRST BLANK LINE, | ||
+ | # even the ones containing #SBATCH. | ||
+ | # Always put your SBATCH parameters at the top of your batch script. | ||
+ | # Took me days to find ... really silly behavior -Henk | ||
+ | # | ||
+ | # GENERAL | ||
+ | #SBATCH --job-name=" | ||
+ | #SBATCH --output=out | ||
+ | #SBATCH --error=err | ||
+ | ##SBATCH --mail-type=END | ||
+ | ##SBATCH --mail-user=hmeij@wesleyan.edu | ||
+ | # | ||
+ | # NODE control | ||
+ | #SBATCH -N 1 # default, nodes | ||
+ | # | ||
+ | # CPU control | ||
+ | #SBATCH -n 1 # tasks=S*C*T | ||
+ | #SBATCH -B 1:1:1 # S: | ||
+ | ###SBATCH -B 2:4:1 # S: | ||
+ | # | ||
+ | # GPU control | ||
+ | ###SBATCH --gres=gpu: | ||
+ | #SBATCH --gres=gpu: | ||
+ | #SBATCH --cpus-per-gpu=1 | ||
+ | #SBATCH --mem-per-gpu=7168 | ||
+ | # | ||
+ | # Node control | ||
+ | #SBATCH --partition=exx96 | ||
+ | #SBATCH --nodelist=n88 | ||
+ | |||
+ | |||
+ | # unique job scratch dirs | ||
+ | MYSANSCRATCH=/ | ||
+ | MYLOCALSCRATCH=/ | ||
+ | export MYSANSCRATCH MYLOCALSCRATCH | ||
+ | cd $MYLOCALSCRATCH | ||
+ | |||
+ | # amber20/ | ||
+ | export PATH=/ | ||
+ | export LD_LIBRARY_PATH=/ | ||
+ | export CUDA_HOME=/ | ||
+ | export PATH=/ | ||
+ | export LD_LIBRARY_PATH=/ | ||
+ | export LD_LIBRARY_PATH="/ | ||
+ | export PATH=/ | ||
+ | export LD_LIBRARY_PATH=/ | ||
+ | which nvcc mpirun python | ||
+ | |||
+ | |||
+ | source / | ||
+ | # stage the data | ||
+ | cp -r ~/ | ||
+ | |||
+ | ###export CUDA_VISIBLE_DEVICES=`shuf -i 0-3 -n 1` | ||
+ | ###export CUDA_VISIBLE_DEVICES=0 | ||
+ | |||
+ | |||
+ | # for amber20 on n[33-37] gpus, select gpu model | ||
+ | mpirun -x LD_LIBRARY_PATH -machinefile ~/ | ||
+ | -np 1 \ | ||
+ | pmemd.cuda \ | ||
+ | -O -o mdout.$SLURM_JOB_ID -inf mdinfo.1K10 -x mdcrd.1K10 -r restrt.1K10 -ref inpcrd | ||
+ | |||
+ | # for amber20 on n59/n100 cpus, select partition | ||
+ | #mpirun -x LD_LIBRARY_PATH -machinefile ~/ | ||
+ | #-np 8 \ | ||
+ | #pmemd.MPI \ | ||
+ | #-O -o mdout.$SLURM_JOB_ID -inf mdinfo.1K10 -x mdcrd.1K10 -r restrt.1K10 -ref inpcrd | ||
+ | |||
+ | scp mdout.$SLURM_JOB_ID ~/tmp/ | ||
+ | </ | ||
+ | |||
+ | |||
+ | **The script amber.sh was converted to a module like so** | ||
< | < | ||
Line 561: | Line 639: | ||
</ | </ | ||
+ | |||
+ | ==== Amber22 ==== | ||
+ | |||
+ | Amber22 is somehow incompatible with CentOS/ | ||
+ | |||
+ | https:// | ||
+ | " | ||
+ | |||
+ | < | ||
+ | |||
+ | [hmeij@n79 src]$ echo $AMBERHOME | ||
+ | / | ||
+ | |||
+ | [hmeij@n79 src]$ which mpirun mpicc | ||
+ | / | ||
+ | / | ||
+ | |||
+ | </ | ||
+ | |||
+ | First establish a successful run with the **run.rocky** script for Amber20 (listed above). Then change the module in your script. (for queues amber128 [n78] and test [n100-n101] for gpus and mw128 and tinymem for cpus) | ||
+ | |||
+ | < | ||
+ | |||
+ | module load amber/22 | ||
+ | |||
+ | # if the module does not show up in the output of your console | ||
+ | |||
+ | module avail | ||
+ | |||
+ | # treat your module cache as out of date | ||
+ | |||
+ | module --ignore_cache avail | ||
+ | |||
+ | </ | ||
+ | |||
+ | First establish a success full run with the **run.centos** script for Amber20 (listed above, for cpus or gpus on queues mwgpu and exx96). | ||
+ | |||
+ | Then edit the script and apply these edits. We had to use a specific compatible '' | ||
+ | |||
+ | < | ||
+ | |||
+ | # comment out the 2 export lines pointing to openmpi | ||
+ | ##export PATH=/ | ||
+ | ##export LD_LIBRARY_PATH=/ | ||
+ | |||
+ | # additional gcc 6.5.0 | ||
+ | export PATH=/ | ||
+ | export LD_LIBRARY_PATH=/ | ||
+ | |||
+ | # edit or add correct source line, which and ldd lines just for debugging | ||
+ | ###source / | ||
+ | ###source / | ||
+ | source / | ||
+ | which nvcc mpirun python | ||
+ | ldd `which pmemd.cuda_SPFP` | ||
+ | |||
+ | </ | ||
+ | |||
\\ | \\ | ||
**[[cluster: | **[[cluster: |