User Tools

Site Tools


cluster:73

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Next revision Both sides next revision
cluster:73 [2020/03/06 12:51]
hmeij07 [Software]
cluster:73 [2020/06/19 17:30]
hmeij07 [Python3]
Line 2: Line 2:
 **[[cluster:0|Back]]** **[[cluster:0|Back]]**
  
-===== Software =====+====== Software ======
  
 The listings below will be updated as software is installed, location is /share/apps/ or /home/apps/ (same thing). The listings below will be updated as software is installed, location is /share/apps/ or /home/apps/ (same thing).
Line 15: Line 15:
 Some compilations require 7.x and are in /share/apps/CENTOS7 \\ Some compilations require 7.x and are in /share/apps/CENTOS7 \\
  
-===== Deep Learning Stack =====+====== Sequencing Tools #2 ====== 
 + 
 +  * macs python package (macs2 see below) 
 +  * http://bowtie-bio.sourceforge.net/bowtie2/index.shtml 
 +  * https://deweylab.github.io/RSEM/ 
 +  * https://broadinstitute.github.io/picard/ 
 +  * https://bedtools.readthedocs.io/en/latest/content/installation.html 
 +  * padillalab  
 + 
 +<code> 
 + 
 +export PATH="/share/apps/CENTOS6/miniconda2/bin:$PATH" 
 +export LD_LIBRARY_PATH="/share/apps/CENTOS6/miniconda2/lib:$LD_LIBRARY_PATH" 
 +Successfully installed macs-1.4.3  # note: macs2 Python version must >= 3.6! 
 + 
 +# or change python version to 
 +# ls /share/apps/CENTOS6/python/2.7.9/bin/macs2 
 +/share/apps/CENTOS6/python/2.7.9/bin/macs2 
 + 
 +# or change python version to 
 +which python: /share/apps/CENTOS7/python/3.8.3/bin/python 
 +pip list: Package         Version 
 +--------------- ------- 
 +bowtie          0.11.0 
 +MACS2           2.2.7.1 
 +numpy           1.18.5 
 +picard          2.3.2 
 +scipy           1.4.1 
 + 
 + 
 +export PATH="/share/apps/CENTOS6/sequencingtools/bowtie2-2.4.1-linux-x86_64:$PATH" 
 + 
 +export PATH=/share/apps/CENTOS6/gcc/4.8.4/bin:$PATH 
 +export LD_LIBRARY_PATH=/share/apps/CENTOS6/gcc/4.8.4/lib64:/share/apps/CENTOS6/gcc/4.8.4/lib:$LD_LIBRARY_PATH 
 + 
 +which gcc: /share/apps/CENTOS6/gcc/4.8.4/bin/gcc 
 +which python: /share/apps/CENTOS6/miniconda2/bin/python 
 +which perl: /share/apps/CENTOS6/miniconda2/bin/perl 
 + 
 +export PATH=/share/apps/CENTOS6/R/3.6.1b/bin:$PATH 
 +export LD_LIBRARY_PATH=/share/apps/CENTOS6/R/3.6.1b/lib64:$LD_LIBRARY_PATH 
 +export PATH=/home/apps/CENTOS6/R/packages/bin:$PATH 
 +export LD_LIBRARY_PATH=/home/apps/CENTOS6/R/packages/lib:$LD_LIBRARY_PATH 
 +which R: /share/apps/CENTOS6/R/3.6.1b/bin/
 +ldd /home/apps/CENTOS6/R/3.6.1b/lib64/R/bin/exec/
 + 
 +java --version: java 12.0.2 2019-07-16 # /share/apps/java/latest -> jdk-12.0.2 
 +java -jar /share/apps/CENTOS6/sequencingtools/picard-2.23.0/picard.jar -h 
 +USAGE: PicardCommandLine <program name> [-h] 
 + 
 +export PATH=/share/apps/CENTOS6/sequencingtools/bedtools-2.29.2/bin:$PATH 
 +bedtools --version: bedtools v2.29.2 
 + 
 +</code> 
 + 
 +====== Deep Learning Software Stack ======
  
   * queue ''exx96'' provides access to docker containers and   * queue ''exx96'' provides access to docker containers and
-    * digits, tensorflow, pytorch, caffe, rapidsai+    * **digits, tensorflow, pytorch, caffe, rapidsai**
     * cuda (several versions)     * cuda (several versions)
     * jupyterlab     * jupyterlab
     * portainer     * portainer
   * consult [[cluster:193|Docker Containers Usage]]   * consult [[cluster:193|Docker Containers Usage]]
 +  * QAC
  
 ====== Paleogenomics Tools ====== ====== Paleogenomics Tools ======
Line 32: Line 88:
   * BCFtools:Reading/writing BCF2/VCF/gVCF files and calling/filtering/summarising SNP and short indel sequence variants   * BCFtools:Reading/writing BCF2/VCF/gVCF files and calling/filtering/summarising SNP and short indel sequence variants
   * HTSlib: A C library for reading/writing high-throughput sequencing data    * HTSlib: A C library for reading/writing high-throughput sequencing data 
 +  * Archeology (kbrunson)
  
 <code> <code>
Line 79: Line 136:
   * https://openstructure.org/   * https://openstructure.org/
   * recipe at /share/apps/CENTOS6/openstructure/1.9-install.txt   * recipe at /share/apps/CENTOS6/openstructure/1.9-install.txt
 +  * Starrlab
  
 OpenStructure relies on these applications and Python 2.7.15 (see python2 section). OpenStructure relies on these applications and Python 2.7.15 (see python2 section).
Line 133: Line 191:
   * add the bin/ directory to your PATH (see examples on this page)   * add the bin/ directory to your PATH (see examples on this page)
   * http://abacus.gene.ucl.ac.uk/software/paml.html   * http://abacus.gene.ucl.ac.uk/software/paml.html
 +  * fcolanlab (wmei, ndelgaudio)
  
 ====== Spyder ====== ====== Spyder ======
Line 141: Line 200:
  
 ====== Python3 ====== ====== Python3 ======
 +
 +  * Python 3.8.3
 +  * /share/apps/CENTOS7/python/3.8.3  (so queues mwgpu, exx96)
 +  * most packages found on this page
 +  * picked up mpirun from miniconda2 apps (that may be a problem)
  
 Added to both versions\\ Added to both versions\\
Line 343: Line 407:
   * location: /share/apps/FastQC/0.11.8   * location: /share/apps/FastQC/0.11.8
   * https://www.bioinformatics.babraham.ac.uk/projects/fastqc/   * https://www.bioinformatics.babraham.ac.uk/projects/fastqc/
 +  * biolab (jcoolon, tearley)
  
  
Line 463: Line 528:
  
  
-====== Sequencing Tools ======+====== Sequencing Tools #1 ======
  
 All of these are located in /share/apps/CENTOS6/python/3.5.2/ All of these are located in /share/apps/CENTOS6/python/3.5.2/
Line 472: Line 537:
   * deepbinner --help   * deepbinner --help
   * https://github.com/rrwick/Deepbinner   * https://github.com/rrwick/Deepbinner
 +  * fcohanlab (ndelgaudio)
  
 **Canu**: a single molecule sequence assembler for genomes large and small.  **Canu**: a single molecule sequence assembler for genomes large and small. 
Line 580: Line 646:
   * gpu version local to n78   * gpu version local to n78
   * sample programs /home/hmeij/freesurfer   * sample programs /home/hmeij/freesurfer
 +  * mindlab (ploui et al)
  
 ====== FSL ======  ====== FSL ====== 
Line 589: Line 656:
   * gpu version local on n78   * gpu version local on n78
   * sample programs /home/hmeij/fsl   * sample programs /home/hmeij/fsl
 +  * mindlab (ploui et al)
  
 ====== Campari ======  ====== Campari ====== 
Line 658: Line 726:
  
   * www: https://github.com/hpparvi/PyTransit   * www: https://github.com/hpparvi/PyTransit
 +  * astrolab
  
  
Line 762: Line 831:
   * program: see below, binary download, no requirement checks.   * program: see below, binary download, no requirement checks.
   * www: http://www.atnf.csiro.au/computing/software/miriad/INSTALL.html   * www: http://www.atnf.csiro.au/computing/software/miriad/INSTALL.html
 +  * astrolab (kflaherty)
  
 <code> <code>
Line 808: Line 878:
   * program: see below   * program: see below
   * www: http://dan.iel.fm/emcee   * www: http://dan.iel.fm/emcee
 +  * astrolab (kflaherty)
  
 <code> <code>
Line 1155: Line 1226:
 ====== LAMMPS CPU/GPU ====== ====== LAMMPS CPU/GPU ======
  
 +** CPU ONLY **
  
 +  * /share/apps/CENTOS6/lammps/3Mar2020
 +    * lmp_mpi, lmp_serial
 +  * consult this file for packages installed and compilations
 +    * /share/apps/CENTOS6/lammps/03Mar2020.install
 +    * should work on all queues
 +  * sample script in ~hmeij/k20redo/run.cpu.lammps
 +
 +
 +** CPU or GPU **
 +
 +  * /share/apps/CENTOS7/lammps/3Mar2020
 +    * lmp_mpi-[double-double|single-double|single-single]
 +  * consult this file for packages installed and compilations
 +    * /share/apps/CENTOS7/lammps/03Mar2020.install
 +    * CUDA_ARCH = sm_75 (probably only works on queue exx96)
 +  * sample script in ~hmeij/k20redo/run.rtx.lammps
 +
 +<code>
 +
 +# simple colloid example
 +CPU run -n1
 +Performance: 493311 tau/day, 1141 timesteps/s
 +GPU run gpu4=1 double-double: 1.1x
 +Performance: 571354 tau/day, 1322 timesteps/s
 +GPU run gpu4=1 single-double: 1.2x
 +Performance: 608498 tau/day, 1408 timesteps/s
 +GPU run gpu4=1 single-single: 1.3x
 +Performance: 623405 tau/day, 1443 timesteps/s
 +
 +</code>
     * /share/apps/CENTOS6/lammps/22Aug18     * /share/apps/CENTOS6/lammps/22Aug18
       * consult [[cluster:161|lammps-11Aug17]] page       * consult [[cluster:161|lammps-11Aug17]] page
Line 2058: Line 2160:
  
 ====== Amber ======  ====== Amber ====== 
 +
 +** AMBER 18 **
 +
 +  * **/share/apps/CENTOS7/amber/amber18**
 +  * compiled on ''n33'' with a ''/usr/local/n37-cuda-9.2'' soft link
 +  * view compile steps at /share/apps/CENTOS7/amber/amber18.install
 +  * when submitting on ''exx96'' we're mixing 
 +    * 10.2 cuda drivers
 +    * 9.2 toolkits
 +  * some speed results below K20 vs RTX2080s, Amber 16 vs 18
 +  * sample jobs at ~hmeij/k20redo/run.amber[16|18]
 +
 +
 +<code>
 +
 +# compiled on n33
 +[hmeij@n33 amber18]$ ls -l bin/pmemd*
 +-rwxr-xr-x 1 hmeij its  3549803 Mar 30 10:19 bin/pmemd
 +lrwxrwxrwx 1 hmeij its       15 Mar 30 11:57 bin/pmemd.cuda -> pmemd.cuda_SPFP
 +-rwxr-xr-x 1 hmeij its 93895208 Mar 30 12:11 bin/pmemd.cuda_DPFP
 +-rwxr-xr-x 1 hmeij its 94734272 Mar 30 13:07 bin/pmemd.cuda_DPFP.MPI
 +lrwxrwxrwx 1 hmeij its       19 Mar 30 12:51 bin/pmemd.cuda.MPI -> pmemd.cuda_SPFP.MPI
 +-rwxr-xr-x 1 hmeij its 75579744 Mar 30 11:57 bin/pmemd.cuda_SPFP
 +-rwxr-xr-x 1 hmeij its 76418760 Mar 30 12:51 bin/pmemd.cuda_SPFP.MPI
 +-rwxr-xr-x 1 hmeij its  4304392 Mar 30 11:23 bin/pmemd.MPI
 +
 +# some quick results
 +|     Average timings for all steps:
 +
 +~hmeij/k20redo/run.amber18 - mwgpu - K20
 +|         ns/day =       2.84   seconds/ns =   30385.24
 +~hmeij/k20redo/run.amber18 - exx96 - RTX2080S
 +|         ns/day =      11.66   seconds/ns =    7408.08
 +~hmeij/k20redo/run.amber16 - exx96 - RTX2080S
 +|         ns/day =       7.09   seconds/ns =   12185.91
 +
 +
 +</code>
  
 **AMBER 16** **AMBER 16**
Line 2072: Line 2212:
   * docs: doc dir inside install_dir, please read Amber17.pdf   * docs: doc dir inside install_dir, please read Amber17.pdf
   * www: http://amber.scripps.edu   * www: http://amber.scripps.edu
-  * sample script: /home/hmeij/amber/cd/run.cpu+  * sample script: ~hmeij/amber/cd/run.cpu
   * no gpu capabilitities, requires cuda 7.5+   * no gpu capabilitities, requires cuda 7.5+
  
Line 2783: Line 2923:
  
   * program: Matlab, link to binary is 'matlab' in ''/share/apps/bin''   * program: Matlab, link to binary is 'matlab' in ''/share/apps/bin''
-    * R2018a, campus wide license so no more restrictions on licenses  +    * R2020a, campus wide license so no more restrictions on licenses  
-    * No Distributed Computing Engine (for now) +    * No Distributed Computing Engine or Matlab Parallel (for now) 
-  * install dir: /share/apps/CENTOS7/matlab/R2018a+  * install dir: /share/apps/CENTOS7/matlab/R2020a
   * docs & www: http://mathworks.com   * docs & www: http://mathworks.com
  
Line 2796: Line 2936:
     * http://www.mathworks.com/help/matlab/mapreduce.html     * http://www.mathworks.com/help/matlab/mapreduce.html
  
-Details about the Matlab configuration/installation and __how to run Matlab jobs__ on the cluster can be found at this **[[cluster:39|Link]]**. Please read that page, the distributed and parallel job submissions are quite different from what we are used to on our  matlab host "rintintin".+
  
 ====== Stata ====== ====== Stata ======
cluster/73.txt · Last modified: 2024/05/15 12:58 by hmeij07