Differences

This shows you the differences between two versions of the page.

--- cluster:73 [2020/02/17 18:02]
hmeij07 [Python3]
+++ cluster:73 [2020/06/18 19:25]
hmeij07
@@ Line 2: / Line 2: @@
 **[[cluster:0|Back]]**
-===== Software =====
+====== Software ======
 The listings below will be updated as software is installed, location is /share/apps/ or /home/apps/ (same thing).
@@ Line 14: / Line 14: @@
 on either cottontail2 or node n33 located at /share/apps/CENTOS6 \\
 Some compilations require 7.x and are in /share/apps/CENTOS7 \\
+====== Sequencing Tools #2 ======
+  * macs python package (macs2 see below)
+  * http://bowtie-bio.sourceforge.net/bowtie2/index.shtml
+  * https://deweylab.github.io/RSEM/
+  * https://broadinstitute.github.io/picard/
+  * https://bedtools.readthedocs.io/en/latest/content/installation.html
+  * padillalab
+<code>
+export PATH="/share/apps/CENTOS6/miniconda2/bin:$PATH"
+export LD_LIBRARY_PATH="/share/apps/CENTOS6/miniconda2/lib:$LD_LIBRARY_PATH"
+Successfully installed macs-1.4.3  # note: macs2 Python version must >= 3.6!
+# or change python version to
+# ls /share/apps/CENTOS6/python/2.7.9/bin/macs2
+/share/apps/CENTOS6/python/2.7.9/bin/macs2
+export PATH="/share/apps/CENTOS6/sequencingtools/bowtie2-2.4.1-linux-x86_64:$PATH"
+export PATH=/share/apps/CENTOS6/gcc/4.8.4/bin:$PATH
+export LD_LIBRARY_PATH=/share/apps/CENTOS6/gcc/4.8.4/lib64:/share/apps/CENTOS6/gcc/4.8.4/lib:$LD_LIBRARY_PATH
+which gcc: /share/apps/CENTOS6/gcc/4.8.4/bin/gcc
+which python: /share/apps/CENTOS6/miniconda2/bin/python
+which perl: /share/apps/CENTOS6/miniconda2/bin/perl
+export PATH=/share/apps/CENTOS6/R/3.6.1b/bin:$PATH
+export LD_LIBRARY_PATH=/share/apps/CENTOS6/R/3.6.1b/lib64:$LD_LIBRARY_PATH
+export PATH=/home/apps/CENTOS6/R/packages/bin:$PATH
+export LD_LIBRARY_PATH=/home/apps/CENTOS6/R/packages/lib:$LD_LIBRARY_PATH
+which R: /share/apps/CENTOS6/R/3.6.1b/bin/R
+ldd /home/apps/CENTOS6/R/3.6.1b/lib64/R/bin/exec/R
+java --version: java 12.0.2 2019-07-16 # /share/apps/java/latest -> jdk-12.0.2
+java -jar /share/apps/CENTOS6/sequencingtools/picard-2.23.0/picard.jar -h
+USAGE: PicardCommandLine <program name> [-h]
+export PATH=/share/apps/CENTOS6/sequencingtools/bedtools-2.29.2/bin:$PATH
+bedtools --version: bedtools v2.29.2
+</code>
+====== Deep Learning Software Stack ======
+  * queue ''exx96'' provides access to docker containers and
+    * **digits, tensorflow, pytorch, caffe, rapidsai**
+    * cuda (several versions)
+    * jupyterlab
+    * portainer
+  * consult [[cluster:193|Docker Containers Usage]]
+  * QAC
 ====== Paleogenomics Tools ======
@@ Line 23: / Line 78: @@
   * BCFtools:Reading/writing BCF2/VCF/gVCF files and calling/filtering/summarising SNP and short indel sequence variants
   * HTSlib: A C library for reading/writing high-throughput sequencing data
+  * Archeology (kbrunson)
 <code>
@@ Line 70: / Line 126: @@
   * https://openstructure.org/
   * recipe at /share/apps/CENTOS6/openstructure/1.9-install.txt
+  * Starrlab
 OpenStructure relies on these applications and Python 2.7.15 (see python2 section).
@@ Line 124: / Line 181: @@
   * add the bin/ directory to your PATH (see examples on this page)
   * http://abacus.gene.ucl.ac.uk/software/paml.html
+  * fcolanlab (wmei, ndelgaudio)
 ====== Spyder ======
@@ Line 132: / Line 190: @@
 ====== Python3 ======
+Added to both versions\\
+pycuda\\
+ --- //[[hmeij@wesleyan.edu|Henk]] 2020/02/18 08:45//
 Added to both 3.5.2 versions\\
@@ Line 330: / Line 392: @@
   * location: /share/apps/FastQC/0.11.8
   * https://www.bioinformatics.babraham.ac.uk/projects/fastqc/
+  * biolab (jcoolon, tearley)
@@ Line 450: / Line 513: @@
-====== Sequencing Tools ======
+====== Sequencing Tools #1 ======
 All of these are located in /share/apps/CENTOS6/python/3.5.2/
@@ Line 459: / Line 522: @@
   * deepbinner --help
   * https://github.com/rrwick/Deepbinner
+  * fcohanlab (ndelgaudio)
 **Canu**: a single molecule sequence assembler for genomes large and small.
@@ Line 567: / Line 631: @@
   * gpu version local to n78
   * sample programs /home/hmeij/freesurfer
+  * mindlab (ploui et al)
 ====== FSL ======
@@ Line 576: / Line 641: @@
   * gpu version local on n78
   * sample programs /home/hmeij/fsl
+  * mindlab (ploui et al)
 ====== Campari ======
@@ Line 645: / Line 711: @@
   * www: https://github.com/hpparvi/PyTransit
+  * astrolab
@@ Line 749: / Line 816: @@
   * program: see below, binary download, no requirement checks.
   * www: http://www.atnf.csiro.au/computing/software/miriad/INSTALL.html
+  * astrolab (kflaherty)
 <code>
@@ Line 795: / Line 863: @@
   * program: see below
   * www: http://dan.iel.fm/emcee
+  * astrolab (kflaherty)
 <code>
@@ Line 1142: / Line 1211: @@
 ====== LAMMPS CPU/GPU ======
+** CPU ONLY **
+  * /share/apps/CENTOS6/lammps/3Mar2020
+    * lmp_mpi, lmp_serial
+  * consult this file for packages installed and compilations
+    * /share/apps/CENTOS6/lammps/03Mar2020.install
+    * should work on all queues
+  * sample script in ~hmeij/k20redo/run.cpu.lammps
+** CPU or GPU **
+  * /share/apps/CENTOS7/lammps/3Mar2020
+    * lmp_mpi-[double-double|single-double|single-single]
+  * consult this file for packages installed and compilations
+    * /share/apps/CENTOS7/lammps/03Mar2020.install
+    * CUDA_ARCH = sm_75 (probably only works on queue exx96)
+  * sample script in ~hmeij/k20redo/run.rtx.lammps
+<code>
+# simple colloid example
+CPU run -n1
+Performance: 493311 tau/day, 1141 timesteps/s
+GPU run gpu4=1 double-double: 1.1x
+Performance: 571354 tau/day, 1322 timesteps/s
+GPU run gpu4=1 single-double: 1.2x
+Performance: 608498 tau/day, 1408 timesteps/s
+GPU run gpu4=1 single-single: 1.3x
+Performance: 623405 tau/day, 1443 timesteps/s
+</code>
     * /share/apps/CENTOS6/lammps/22Aug18
       * consult [[cluster:161|lammps-11Aug17]] page
@@ Line 1886: / Line 1986: @@
 ====== R & R/parallel ======
+  * https://edwards.sdsu.edu/research/c11-on-centos-6/
+    * C11++ on CentOS6, two version of gcc
+    * read the R-3.6.1_configure file in ''/share/apps/src/cottontail2''
+<code>
+install.packages('lme4', dependencies = TRUE)
+install.packages('tidyverse', dependencies = TRUE)
+install.packages('mice', dependencies = TRUE)
+install.packages('rpart', dependencies = TRUE)
+install.packages('party', dependencies = TRUE)
+install.packages('caret', dependencies = TRUE)
+install.packages('nnet', dependencies = TRUE)
+install.packages('e1071', dependencies = TRUE)
+install.packages('xgboost', dependencies = TRUE)
+install.packages('kernlab', dependencies = TRUE)
+install.packages('randomForest', dependencies = TRUE)
+</code>
   * R 3.6.1 (running against "packages", had to comment out libcurl version check)
@@ Line 2026: / Line 2145: @@
 ====== Amber ======
+** AMBER 18 **
+  * **/share/apps/CENTOS7/amber/amber18**
+  * compiled on ''n33'' with a ''/usr/local/n37-cuda-9.2'' soft link
+  * view compile steps at /share/apps/CENTOS7/amber/amber18.install
+  * when submitting on ''exx96'' we're mixing
+    * 10.2 cuda drivers
+    * 9.2 toolkits
+  * some speed results below K20 vs RTX2080s, Amber 16 vs 18
+  * sample jobs at ~hmeij/k20redo/run.amber[16|18]
+<code>
+# compiled on n33
+[hmeij@n33 amber18]$ ls -l bin/pmemd*
+-rwxr-xr-x 1 hmeij its  3549803 Mar 30 10:19 bin/pmemd
+lrwxrwxrwx 1 hmeij its       15 Mar 30 11:57 bin/pmemd.cuda -> pmemd.cuda_SPFP
+-rwxr-xr-x 1 hmeij its 93895208 Mar 30 12:11 bin/pmemd.cuda_DPFP
+-rwxr-xr-x 1 hmeij its 94734272 Mar 30 13:07 bin/pmemd.cuda_DPFP.MPI
+lrwxrwxrwx 1 hmeij its       19 Mar 30 12:51 bin/pmemd.cuda.MPI -> pmemd.cuda_SPFP.MPI
+-rwxr-xr-x 1 hmeij its 75579744 Mar 30 11:57 bin/pmemd.cuda_SPFP
+-rwxr-xr-x 1 hmeij its 76418760 Mar 30 12:51 bin/pmemd.cuda_SPFP.MPI
+-rwxr-xr-x 1 hmeij its  4304392 Mar 30 11:23 bin/pmemd.MPI
+# some quick results
+|     Average timings for all steps:
+~hmeij/k20redo/run.amber18 - mwgpu - K20
+|         ns/day =       2.84   seconds/ns =   30385.24
+~hmeij/k20redo/run.amber18 - exx96 - RTX2080S
+|         ns/day =      11.66   seconds/ns =    7408.08
+~hmeij/k20redo/run.amber16 - exx96 - RTX2080S
+|         ns/day =       7.09   seconds/ns =   12185.91
+</code>
 **AMBER 16**
@@ Line 2040: / Line 2197: @@
   * docs: doc dir inside install_dir, please read Amber17.pdf
   * www: http://amber.scripps.edu
-  * sample script: /home/hmeij/amber/cd/run.cpu
+  * sample script: ~hmeij/amber/cd/run.cpu
   * no gpu capabilitities, requires cuda 7.5+
@@ Line 2751: / Line 2908: @@
   * program: Matlab, link to binary is 'matlab' in ''/share/apps/bin''
-    * R2018a, campus wide license so no more restrictions on licenses
+    * R2020a, campus wide license so no more restrictions on licenses
-    * No Distributed Computing Engine (for now)
+    * No Distributed Computing Engine or Matlab Parallel (for now)
-  * install dir: /share/apps/CENTOS7/matlab/R2018a
+  * install dir: /share/apps/CENTOS7/matlab/R2020a
   * docs & www: http://mathworks.com
@@ Line 2764: / Line 2921: @@
     * http://www.mathworks.com/help/matlab/mapreduce.html
-Details about the Matlab configuration/installation and __how to run Matlab jobs__ on the cluster can be found at this **[[cluster:39|Link]]**. Please read that page, the distributed and parallel job submissions are quite different from what we are used to on our  matlab host "rintintin".
 ====== Stata ======

DokuWiki

User Tools

Site Tools

Differences

Page Tools