Differences

This shows you the differences between two versions of the page.

--- cluster:73 [2020/01/30 13:06]
hmeij07
+++ cluster:73 [2020/06/18 15:27]
hmeij07 [Deep Learning Software Stack]
@@ Line 2: / Line 2: @@
 **[[cluster:0|Back]]**
-===== Software =====
+====== Software ======
 The listings below will be updated as software is installed, location is /share/apps/ or /home/apps/ (same thing).
@@ Line 14: / Line 14: @@
 on either cottontail2 or node n33 located at /share/apps/CENTOS6 \\
 Some compilations require 7.x and are in /share/apps/CENTOS7 \\
+====== Miniconda2 ======
+<code>
+Installing collected packages: MarkupSafe, jinja2, markdown, click, python-engineio, python-socketio, Werkzeug, itsdangerous, flask, flask-socketio, greenlet, dnspython, monotonic, eventlet, future, msgpack-python, bowtie
+Successfully installed MarkupSafe-1.1.1 Werkzeug-1.0.1 bowtie-0.8.1 click-7.1.2 dnspython-1.16.0 eventlet-0.25.2 flask-1.1.2 flask-socketio-4.3.0 future-0.18.2 greenlet-0.4.16 itsdangerous-1.1.0 jinja2-2.11.2 markdown-3.1.1 monotonic-1.5 msgpack-python-0.5.6 python-engineio-3.13.0 python-socketio-4.6.0
+</code>
+====== Deep Learning Software Stack ======
+  * queue ''exx96'' provides access to docker containers and
+    * **digits, tensorflow, pytorch, caffe, rapidsai**
+    * cuda (several versions)
+    * jupyterlab
+    * portainer
+  * consult [[cluster:193|Docker Containers Usage]]
 ====== Paleogenomics Tools ======
@@ Line 132: / Line 148: @@
 ====== Python3 ======
+Added to both versions\\
+pycuda\\
+ --- //[[hmeij@wesleyan.edu|Henk]] 2020/02/18 08:45//
+Added to both 3.5.2 versions\\
+Successfully installed\\
+boto-2.49.0 boto3-1.12.0 botocore-1.15.0 gensim-3.8.1 jmespath-0.9.4 s3transfer-0.3.3 smart-open-1.9.0\\
+joblib, scikit-learn, sklearn\\
+\\
+Requirement already satisfied:\\
+keras keras-preprocessing keras-applications\\
+tensorflow (requires centos7)\\
+ --- //[[hmeij@wesleyan.edu|Henk]] 2020/02/17 13:02//
 Added to both 3.5.2 versions\\
@@ Line 1131: / Line 1162: @@
 ====== LAMMPS CPU/GPU ======
+** CPU ONLY **
+  * /share/apps/CENTOS6/lammps/3Mar2020
+    * lmp_mpi, lmp_serial
+  * consult this file for packages installed and compilations
+    * /share/apps/CENTOS6/lammps/03Mar2020.install
+    * should work on all queues
+  * sample script in ~hmeij/k20redo/run.cpu.lammps
+** CPU or GPU **
+  * /share/apps/CENTOS7/lammps/3Mar2020
+    * lmp_mpi-[double-double|single-double|single-single]
+  * consult this file for packages installed and compilations
+    * /share/apps/CENTOS7/lammps/03Mar2020.install
+    * CUDA_ARCH = sm_75 (probably only works on queue exx96)
+  * sample script in ~hmeij/k20redo/run.rtx.lammps
+<code>
+# simple colloid example
+CPU run -n1
+Performance: 493311 tau/day, 1141 timesteps/s
+GPU run gpu4=1 double-double: 1.1x
+Performance: 571354 tau/day, 1322 timesteps/s
+GPU run gpu4=1 single-double: 1.2x
+Performance: 608498 tau/day, 1408 timesteps/s
+GPU run gpu4=1 single-single: 1.3x
+Performance: 623405 tau/day, 1443 timesteps/s
+</code>
     * /share/apps/CENTOS6/lammps/22Aug18
       * consult [[cluster:161|lammps-11Aug17]] page
@@ Line 1875: / Line 1937: @@
 ====== R & R/parallel ======
+  * https://edwards.sdsu.edu/research/c11-on-centos-6/
+    * C11++ on CentOS6, two version of gcc
+    * read the R-3.6.1_configure file in ''/share/apps/src/cottontail2''
+<code>
+install.packages('lme4', dependencies = TRUE)
+install.packages('tidyverse', dependencies = TRUE)
+install.packages('mice', dependencies = TRUE)
+install.packages('rpart', dependencies = TRUE)
+install.packages('party', dependencies = TRUE)
+install.packages('caret', dependencies = TRUE)
+install.packages('nnet', dependencies = TRUE)
+install.packages('e1071', dependencies = TRUE)
+install.packages('xgboost', dependencies = TRUE)
+install.packages('kernlab', dependencies = TRUE)
+install.packages('randomForest', dependencies = TRUE)
+</code>
   * R 3.6.1 (running against "packages", had to comment out libcurl version check)
@@ Line 2015: / Line 2096: @@
 ====== Amber ======
+** AMBER 18 **
+  * **/share/apps/CENTOS7/amber/amber18**
+  * compiled on ''n33'' with a ''/usr/local/n37-cuda-9.2'' soft link
+  * view compile steps at /share/apps/CENTOS7/amber/amber18.install
+  * when submitting on ''exx96'' we're mixing
+    * 10.2 cuda drivers
+    * 9.2 toolkits
+  * some speed results below K20 vs RTX2080s, Amber 16 vs 18
+  * sample jobs at ~hmeij/k20redo/run.amber[16|18]
+<code>
+# compiled on n33
+[hmeij@n33 amber18]$ ls -l bin/pmemd*
+-rwxr-xr-x 1 hmeij its  3549803 Mar 30 10:19 bin/pmemd
+lrwxrwxrwx 1 hmeij its       15 Mar 30 11:57 bin/pmemd.cuda -> pmemd.cuda_SPFP
+-rwxr-xr-x 1 hmeij its 93895208 Mar 30 12:11 bin/pmemd.cuda_DPFP
+-rwxr-xr-x 1 hmeij its 94734272 Mar 30 13:07 bin/pmemd.cuda_DPFP.MPI
+lrwxrwxrwx 1 hmeij its       19 Mar 30 12:51 bin/pmemd.cuda.MPI -> pmemd.cuda_SPFP.MPI
+-rwxr-xr-x 1 hmeij its 75579744 Mar 30 11:57 bin/pmemd.cuda_SPFP
+-rwxr-xr-x 1 hmeij its 76418760 Mar 30 12:51 bin/pmemd.cuda_SPFP.MPI
+-rwxr-xr-x 1 hmeij its  4304392 Mar 30 11:23 bin/pmemd.MPI
+# some quick results
+|     Average timings for all steps:
+~hmeij/k20redo/run.amber18 - mwgpu - K20
+|         ns/day =       2.84   seconds/ns =   30385.24
+~hmeij/k20redo/run.amber18 - exx96 - RTX2080S
+|         ns/day =      11.66   seconds/ns =    7408.08
+~hmeij/k20redo/run.amber16 - exx96 - RTX2080S
+|         ns/day =       7.09   seconds/ns =   12185.91
+</code>
 **AMBER 16**
@@ Line 2029: / Line 2148: @@
   * docs: doc dir inside install_dir, please read Amber17.pdf
   * www: http://amber.scripps.edu
-  * sample script: /home/hmeij/amber/cd/run.cpu
+  * sample script: ~hmeij/amber/cd/run.cpu
   * no gpu capabilitities, requires cuda 7.5+
@@ Line 2609: / Line 2728: @@
 For all these installations you will need to set $PATH and $LD_LIBRARY_PATH, look for samples on this page ...
+  * program: GCC v 9.2.0
+  * /share/apps/CENTOS6/9.2.0
+  * note: libgomp disabled for nanopolish compile attempt
+<code>
+  wget ftp://ftp.nluug.nl/mirror/languages/gcc/releases/gcc-9.2.0/gcc-9.2.0.tar.gz
+  tar zxvf gcc-9.2.0.tar.gz
+  cd gcc-9.2.0
+  ./contrib/download_prerequisites
+  ./configure --prefix=/share/apps/CENTOS6/gcc/9.2.0 --disable-libgomp --disable-multilib
+  make
+  make install
+</code>
   * program: GCC v 4.8.4
@@ Line 2723: / Line 2859: @@
   * program: Matlab, link to binary is 'matlab' in ''/share/apps/bin''
-    * R2018a, campus wide license so no more restrictions on licenses
+    * R2020a, campus wide license so no more restrictions on licenses
-    * No Distributed Computing Engine (for now)
+    * No Distributed Computing Engine or Matlab Parallel (for now)
-  * install dir: /share/apps/CENTOS7/matlab/R2018a
+  * install dir: /share/apps/CENTOS7/matlab/R2020a
   * docs & www: http://mathworks.com
@@ Line 2736: / Line 2872: @@
     * http://www.mathworks.com/help/matlab/mapreduce.html
-Details about the Matlab configuration/installation and __how to run Matlab jobs__ on the cluster can be found at this **[[cluster:39|Link]]**. Please read that page, the distributed and parallel job submissions are quite different from what we are used to on our  matlab host "rintintin".
 ====== Stata ======

DokuWiki

User Tools

Site Tools

Differences

Page Tools