User Tools

Site Tools


cluster:73

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Next revision Both sides next revision
cluster:73 [2020/06/18 18:26]
hmeij07 [Sequencing Tools]
cluster:73 [2021/03/11 19:15]
hmeij07 [Rstudio]
Line 3: Line 3:
  
 ====== Software ====== ====== Software ======
 +
 +IMPORTANT NOTE: **Since moving to the TrueNAS/ZFS appliance all references to** ''/home/apps'' **should be replaced with** ''/share/apps'' **which points to** ''/zfshomes/apps''
 +
 + --- //[[hmeij@wesleyan.edu|Henk]] 2020/07/31 15:08//  
  
 The listings below will be updated as software is installed, location is /share/apps/ or /home/apps/ (same thing). The listings below will be updated as software is installed, location is /share/apps/ or /home/apps/ (same thing).
Line 15: Line 19:
 Some compilations require 7.x and are in /share/apps/CENTOS7 \\ Some compilations require 7.x and are in /share/apps/CENTOS7 \\
  
-====== Sequencing Tools ======+====== StringTie2 ====== 
 + 
 +  * Requires CentOS 7 so queues mwgpu, amber128, exx96 
 +  * /share/apps/CENTOS7/stringtie2/stringtie2 binarie 
 +  * https://github.com/skovaka/stringtie2 
 + 
 +====== Rstudio ====== 
 + 
 +  * http://our-sanscratch-server:8787/ 
 +  * https://support.rstudio.com/hc/en-us/articles/200552316-Configuring-the-Server 
 +  * centos7 R v 3.6.0 
 +  * use your HPC username and credentials 
 + 
 +====== DMTCP ===== 
 + 
 +  * /share/apps/CENTOS7/dmtcp/3.0.0 
 +  * this came from the CRAC source, trying to install gpu checkpoint tool 
 +  * [[cluster:198|GPU checkpoint/restart]] 
 +  * but plugin will not compile, checking with developers 12 Dec 2020 
 + 
 +  * Installed as package ins OS on all nodes 
 +  * v2.5.2 
 +  * dmctp_launch ... etc 
 + 
 +====== EMBOSS ====== 
 + 
 +  * /share/apps/CENTOS6/EMBOSS 
 +  * /share/apps/CENTOS6/EMBOSS/6.6.0.install (read to setup env) 
 +  * https://www.ebi.ac.uk/services 
 +  * jfabry/kthayer 
 + 
 +====== Sequencing Tools #2 ====== 
 + 
 +  * macs python package (macs2 see below) 
 +  * http://bowtie-bio.sourceforge.net/bowtie2/index.shtml 
 +  * https://deweylab.github.io/RSEM/ 
 +  * https://broadinstitute.github.io/picard/ 
 +  * https://bedtools.readthedocs.io/en/latest/content/installation.html 
 +  * padillalab 
  
 <code> <code>
Line 22: Line 64:
 export LD_LIBRARY_PATH="/share/apps/CENTOS6/miniconda2/lib:$LD_LIBRARY_PATH" export LD_LIBRARY_PATH="/share/apps/CENTOS6/miniconda2/lib:$LD_LIBRARY_PATH"
 Successfully installed macs-1.4.3  # note: macs2 Python version must >= 3.6! Successfully installed macs-1.4.3  # note: macs2 Python version must >= 3.6!
 +
 +# or change python version to
 +# ls /share/apps/CENTOS6/python/2.7.9/bin/macs2
 +/share/apps/CENTOS6/python/2.7.9/bin/macs2
 +
 +# or change python version to
 +which python: /share/apps/CENTOS7/python/3.8.3/bin/python
 +pip list: Package         Version
 +--------------- -------
 +bowtie          0.11.0
 +MACS2           2.2.7.1
 +numpy           1.18.5
 +picard          2.3.2
 +scipy           1.4.1
 +
  
 export PATH="/share/apps/CENTOS6/sequencingtools/bowtie2-2.4.1-linux-x86_64:$PATH" export PATH="/share/apps/CENTOS6/sequencingtools/bowtie2-2.4.1-linux-x86_64:$PATH"
Line 43: Line 100:
 USAGE: PicardCommandLine <program name> [-h] USAGE: PicardCommandLine <program name> [-h]
  
 +export PATH=/share/apps/CENTOS6/sequencingtools/bedtools-2.29.2/bin:$PATH
 +bedtools --version: bedtools v2.29.2
  
 </code> </code>
Line 54: Line 113:
     * portainer     * portainer
   * consult [[cluster:193|Docker Containers Usage]]   * consult [[cluster:193|Docker Containers Usage]]
 +  * QAC
  
 ====== Paleogenomics Tools ====== ====== Paleogenomics Tools ======
Line 63: Line 123:
   * BCFtools:Reading/writing BCF2/VCF/gVCF files and calling/filtering/summarising SNP and short indel sequence variants   * BCFtools:Reading/writing BCF2/VCF/gVCF files and calling/filtering/summarising SNP and short indel sequence variants
   * HTSlib: A C library for reading/writing high-throughput sequencing data    * HTSlib: A C library for reading/writing high-throughput sequencing data 
 +  * Archeology (kbrunson)
  
 <code> <code>
Line 110: Line 171:
   * https://openstructure.org/   * https://openstructure.org/
   * recipe at /share/apps/CENTOS6/openstructure/1.9-install.txt   * recipe at /share/apps/CENTOS6/openstructure/1.9-install.txt
 +  * Starrlab
  
 OpenStructure relies on these applications and Python 2.7.15 (see python2 section). OpenStructure relies on these applications and Python 2.7.15 (see python2 section).
Line 164: Line 226:
   * add the bin/ directory to your PATH (see examples on this page)   * add the bin/ directory to your PATH (see examples on this page)
   * http://abacus.gene.ucl.ac.uk/software/paml.html   * http://abacus.gene.ucl.ac.uk/software/paml.html
 +  * fcolanlab (wmei, ndelgaudio)
  
 ====== Spyder ====== ====== Spyder ======
Line 173: Line 236:
 ====== Python3 ====== ====== Python3 ======
  
-Added to both versions\\+  * Python 3.8.3 
 +  * /share/apps/CENTOS7/python/3.8.3  (so queues mwgpu, exx96) 
 +  * most packages found on this page 
 +  * picked up mpirun from miniconda2 apps (that may be a problem) 
 + 
 +  * Added to 3.8.3 "acpipe" 
 +  * https://github.com/alanwilter/acpype 
 + 
 + --- //[[hmeij@wesleyan.edu|Henk]] 2021/02/18 14:41// 
 + 
 +<code> 
 + 
 +# needed for _ctypes to compile properly for https retrievals 
 + yum install openssl openssl-devel 
 + yum install yum install libffi-devel libffi 
 + 
 +  tar -xvf ../Python-3.8.3.tgz 
 +  cd Python-3.8.3/ 
 +  ./configure --prefix=/share/apps/CENTOS7/python/3.8.3 --enable-optimizations 
 +  make 
 +  make install 
 +  pip3 install --upgrade pip 
 +# make symbolic links python3->pyhton, pip3->pip 
 +# drop all the packages in from previous versions 
 +https://dokuwiki.wesleyan.edu/doku.php?id=cluster:73#python3 
 + 
 +</code> 
 + 
 + 
 +Added to both 3.5.2 versions\\
 pycuda\\ pycuda\\
  --- //[[hmeij@wesleyan.edu|Henk]] 2020/02/18 08:45//  --- //[[hmeij@wesleyan.edu|Henk]] 2020/02/18 08:45//
Line 374: Line 466:
   * location: /share/apps/FastQC/0.11.8   * location: /share/apps/FastQC/0.11.8
   * https://www.bioinformatics.babraham.ac.uk/projects/fastqc/   * https://www.bioinformatics.babraham.ac.uk/projects/fastqc/
 +  * biolab (jcoolon, tearley)
  
  
Line 494: Line 587:
  
  
-====== Sequencing Tools ======+====== Sequencing Tools #1 ======
  
 All of these are located in /share/apps/CENTOS6/python/3.5.2/ All of these are located in /share/apps/CENTOS6/python/3.5.2/
Line 503: Line 596:
   * deepbinner --help   * deepbinner --help
   * https://github.com/rrwick/Deepbinner   * https://github.com/rrwick/Deepbinner
 +  * fcohanlab (ndelgaudio)
  
 **Canu**: a single molecule sequence assembler for genomes large and small.  **Canu**: a single molecule sequence assembler for genomes large and small. 
Line 611: Line 705:
   * gpu version local to n78   * gpu version local to n78
   * sample programs /home/hmeij/freesurfer   * sample programs /home/hmeij/freesurfer
 +  * mindlab (ploui et al)
  
 ====== FSL ======  ====== FSL ====== 
Line 620: Line 715:
   * gpu version local on n78   * gpu version local on n78
   * sample programs /home/hmeij/fsl   * sample programs /home/hmeij/fsl
 +  * mindlab (ploui et al)
  
 ====== Campari ======  ====== Campari ====== 
Line 689: Line 785:
  
   * www: https://github.com/hpparvi/PyTransit   * www: https://github.com/hpparvi/PyTransit
 +  * astrolab
  
  
Line 793: Line 890:
   * program: see below, binary download, no requirement checks.   * program: see below, binary download, no requirement checks.
   * www: http://www.atnf.csiro.au/computing/software/miriad/INSTALL.html   * www: http://www.atnf.csiro.au/computing/software/miriad/INSTALL.html
 +  * astrolab (kflaherty)
  
 <code> <code>
Line 839: Line 937:
   * program: see below   * program: see below
   * www: http://dan.iel.fm/emcee   * www: http://dan.iel.fm/emcee
 +  * astrolab (kflaherty)
  
 <code> <code>
Line 884: Line 983:
  
 ====== Gromacs ===== ====== Gromacs =====
 +
 +Gromacs-2020.3 for GPU (RTX2080S)
 +  * ''exx96'' queue
 +  * see /share/apps/CENTOS7/gromacs/2020.3.install for env
 +  * change env in script below
 +
 +Gromacs-2018 for GPU (RTX2080S)
 +  * ''exx96'' queue
 +  * **/zfshomes/hmeij/k20redo/run.rtx sample script**
  
  
Line 991: Line 1099:
 [[https://dokuwiki.wesleyan.edu/doku.php?id=cluster:119|GPU version]] [[https://dokuwiki.wesleyan.edu/doku.php?id=cluster:119|GPU version]]
  
-  * program: mcc, math, mathematica (v 11.0.1+  * program: mcc, math, mathematica (v 12.2
-  * install dir: /share/apps/CENTOS6/mathematica/11.0.1+  * install dir: /share/apps/CENTOS7/mathematica/12.2
   * docs: on the web   * docs: on the web
   * www:  http://www.wolfram.com/mathematica/   * www:  http://www.wolfram.com/mathematica/
 +  * Centos7, use queues mwgpu, amber128 or exx96
 +  * For GUI launch on greentail52
  
   * Parallel kernels   * Parallel kernels
Line 1001: Line 1111:
     * http://reference.wolfram.com/mathematica/guide/ParallelComputing.html     * http://reference.wolfram.com/mathematica/guide/ParallelComputing.html
  
-Job submissions should go to queue //mathematica// which containes licensed nodes (all the hp12 nodes for now).+
  
 ====== Eigen ===== ====== Eigen =====
Line 1185: Line 1295:
  
 ====== LAMMPS CPU/GPU ====== ====== LAMMPS CPU/GPU ======
 +
 +** CPU only ** -Centos6
 +
 +  *  /share/apps/CENTOS6/lammps/29Oct20.install
 +  * read for env and compilation env: lmp_mpi
 +
 +** CPU only OR CPU+GPU**  for ''exx96'' queue - CentOS7
 +
 +  * 3Mar2020 added USER-REAXC
 +  * For KOKKOS MPI aware version  /share/apps/CENTOS7/openmpi/4.0.4-ucx with UCX
 +  * git download of latest did not work (lots of fortran errors?)
 +  * latest tarball is still 3Mar2020 (?)
 +  * added user-eaxc (but had to remove package python)
 +  * "kokkos" version includes GPU accelerator
 +  * please read **/share/apps/CENTOS7/lammps/3Mar2020.install **
 +
 +<code>
 +
 +# CPU
 +/share/apps/CENTOS7/lammps/3Mar2020/lmp_serial+user-omp+user-reaxc
 +/share/apps/CENTOS7/lammps/3Mar2020/lmp_mpi+user-omp+user-reaxc
 +
 +# CPU+GPU
 +/share/apps/CENTOS7/lammps/3Mar2020/lmp_mpi-single-single_kokkos_cuda_mpi+user-omp+user-reaxc
 +
 +# env
 +
 + export PATH=/share/apps/CENTOS7/openmpi/4.0.4/bin:$PATH
 + export LD_LIBRARY_PATH=/share/apps/CENTOS7/openmpi/4.0.4/lib:$LD_LIBRARY_PATH
 + export PATH=/share/apps/CENTOS7/python/3.8.3/bin:$PATH
 + export LD_LIBRARY_PATH=/share/apps/CENTOS7/python/3.8.3/lib:$LD_LIBRARY_PATH
 +
 + export CUDA_HOME=/usr/local/cuda
 + export PATH=/usr/local/cuda/bin:$PATH
 + export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
 + export LD_LIBRARY_PATH=/usr/local/cuda/lib:$LD_LIBRARY_PATH
 +
 +</code>
 +
 +** CPU ONLY v2 CentOS7 **
 +
 +  * add user-opm on next compile ... done ... binaries with string +user-omp
 +  * /share/apps/CENTOS7/lammps/3Mar2020
 +    * lmp_serial, lmp_mpi
 +  * //latest// openmpi version as of today v4.0.4
 +  * consult this file for packages installed and compilations
 +    * /share/apps/CENTOS7/lammps/03Mar2020.install
 +    * should work only on exx96 and mwgpu queues
 +
 +<code>
 + export PATH=/share/apps/CENTOS7/openmpi/4.0.4/bin:$PATH
 + export LD_LIBRARY_PATH=/share/apps/CENTOS7/openmpi/4.0.4/lib:$LD_LIBRARY_PATH
 +</code>
  
 ** CPU ONLY ** ** CPU ONLY **
Line 1197: Line 1360:
  
 ** CPU or GPU ** ** CPU or GPU **
 +
 +  * added lmp_mpi-single-single+user-omp
  
   * /share/apps/CENTOS7/lammps/3Mar2020   * /share/apps/CENTOS7/lammps/3Mar2020
Line 1203: Line 1368:
     * /share/apps/CENTOS7/lammps/03Mar2020.install     * /share/apps/CENTOS7/lammps/03Mar2020.install
     * CUDA_ARCH = sm_75 (probably only works on queue exx96)     * CUDA_ARCH = sm_75 (probably only works on queue exx96)
-  * sample script in ~hmeij/k20redo/run.rtx.lammps+  * **sample script in ~hmeij/k20redo/run.rtx.lammps**
  
 <code> <code>
Line 2120: Line 2285:
  
 ====== Amber ======  ====== Amber ====== 
 +
 +** Amber 20 **
 +
 +  * Two versions compiled, 
 +    * one for cuda 8.0/mpich3 for queue amber128
 +    * one for queues mwgpu and exx96 (cuda 9.2/openmpi).
 +
 +  * Consult the run.amber20 script to set up environment
 +    * /share/apps/CENTOS7/amber/amber20/run.amber20
 +
 +  * Both are installed in /usr/local/amber20
 +
 +
  
 ** AMBER 18 ** ** AMBER 18 **
Line 2883: Line 3061:
  
   * program: Matlab, link to binary is 'matlab' in ''/share/apps/bin''   * program: Matlab, link to binary is 'matlab' in ''/share/apps/bin''
-    * R2020a, campus wide license so no more restrictions on licenses +    * R2020b, campus wide license so no more restrictions on licenses 
     * No Distributed Computing Engine or Matlab Parallel (for now)     * No Distributed Computing Engine or Matlab Parallel (for now)
-  * install dir: /share/apps/CENTOS7/matlab/R2020a+  * install dir: /share/apps/CENTOS7/matlab/R2020bISO
   * docs & www: http://mathworks.com   * docs & www: http://mathworks.com
 +  * CentOS7 required use on queues mwgpu, amber128 and exx96 
 +  * For GUI launch on greentail52
  
 Note: Note:
cluster/73.txt · Last modified: 2024/05/15 12:58 by hmeij07