DokuWiki

User Tools

Site Tools

Trace:
cluster:73

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
Next revision Both sides next revision
cluster:73 [2019/07/23 15:02]
hmeij07 [Python3] 		cluster:73 [2020/04/06 17:03]
hmeij07 [Amber]
Line 2: Line 2:
 **[[cluster:0|Back]]** **[[cluster:0|Back]]**
  
 +====== Software ======
  
 The listings below will be updated as software is installed, location is /share/apps/ or /home/apps/ (same thing). The listings below will be updated as software is installed, location is /share/apps/ or /home/apps/ (same thing).
Line 10: Line 10:
 Most of the compilations have been performed by Intel's icc/ifort compilers. Most of the compilations have been performed by Intel's icc/ifort compilers.
  
-The list of software was primarily compiled under RHEL5.1 but more recent compilation are done on host sharptail under RHEL5.5. Newer compilations are done with CentOS 6.5 (since summer 2013) on either swallowtail or node n33.+Old compilations, mostly located in /share/apps, are RHEL 5.5 \\ 
 +Newer compilations are done with CentOS 6.10 (since summer 2013) \\ 
 +on either cottontail2 or node n33 located at /share/apps/CENTOS6 \\ 
 +Some compilations require 7.x and are in /share/apps/CENTOS7 \\ 
 + 
 +====== Deep Learning Software Stack ====== 
 + 
 +  * queue ''exx96'' provides access to docker containers and 
 +    * **digits, tensorflow, pytorch, caffe, rapidsai** 
 +    * cuda (several versions) 
 +    * jupyterlab 
 +    * portainer 
 +  * consult [[cluster:193|Docker Containers Usage]] 
 + 
 +====== Paleogenomics Tools ====== 
 + 
 +Samtools is a suite of programs for interacting with high-throughput sequencing data. It consists of three separate repositories: 
 + 
 +  * http://www.htslib.org/download/ 
 +  * Samtools: Reading/writing/editing/indexing/viewing SAM/BAM/CRAM format 
 +  * BCFtools:Reading/writing BCF2/VCF/gVCF files and calling/filtering/summarising SNP and short indel sequence variants 
 +  * HTSlib: A C library for reading/writing high-throughput sequencing data  
 + 
 +<code> 
 + 
 +# add to environment 
 +export PATH=/share/apps/CENTOS6/samtools/1.10/bin:$PATH 
 +export PATH=/share/apps/CENTOS6/bcftools/1.10.2/bin:$PATH 
 +export PATH=/share/apps/CENTOS6/htslib/1.10.2/bin:$PATH 
 + 
 +</code> 
 + 
 +   * https://github.com/ANGSD/angsd 
 +   * Program for analysing NGS data 
 + 
 +<code> 
 + 
 +# does not seem to work against htslib version above... 
 +export  HTSSRC=/share/apps/CENTOS6/ANGSD/htslib 
 +export PATH=/share/apps/CENTOS6/ANGSD/angsd:$PATH 
 + 
 +</code> 
 + 
 +  * https://www.beast2.org/ 
 +  * BEAST 2 is a cross-platform program for Bayesian phylogenetic analysis of molecular sequences.  
 + 
 +<code> 
 + 
 +# with local java jre1.8.0_212 
 +export PATH=/share/apps/CENTOS6/beast/2.6.0/bin:$PATH 
 + 
 +</code> 
 + 
 + 
 +====== CD-HIT ====== 
 + 
 +CD-HIT is a widely used program for clustering biological sequences to reduce sequence redundancy and improve the performance of other sequence analyses.  
 + 
 +  * https://github.com/weizhongli/cdhit/wiki 
 +  * /share/apps/CENTOS6/cd-hit-v4.8.1-2019-0228/ 
  
 ====== OpenStructure ====== ====== OpenStructure ======
Line 81: Line 141:
  
 ====== Python3 ====== ====== Python3 ======
 +
 +Added to both versions\\
 +pycuda\\
 + --- //[[hmeij@wesleyan.edu|Henk]] 2020/02/18 08:45//
 +
 +Added to both 3.5.2 versions\\
 +Successfully installed\\
 +boto-2.49.0 boto3-1.12.0 botocore-1.15.0 gensim-3.8.1 jmespath-0.9.4 s3transfer-0.3.3 smart-open-1.9.0\\
 +joblib, scikit-learn, sklearn\\
 +\\
 +Requirement already satisfied:\\
 +keras keras-preprocessing keras-applications\\
 +tensorflow (requires centos7)\\
 + --- //[[hmeij@wesleyan.edu|Henk]] 2020/02/17 13:02//
 +
  
 Added to both 3.5.2 versions\\ Added to both 3.5.2 versions\\
Line 261: Line 336:
   * www: http://www.python.org/   * www: http://www.python.org/
  
-====== Kallisto & Trinity ======+====== Kallisto & Trinity & FastQC====== 
 + 
 +**FastQC** A quality control tool for high throughput sequence data. 
 + 
 +  * ''java -version'' latest JDK installed, in default PATH ''which java'' 
 +  * location: /share/apps/FastQC/0.11.8 
 +  * https://www.bioinformatics.babraham.ac.uk/projects/fastqc/ 
  
 **kallisto** is a program for quantifying abundances of transcripts from bulk and single-cell RNA-Seq data, or more generally of target sequences using high-throughput sequencing reads. **kallisto** is a program for quantifying abundances of transcripts from bulk and single-cell RNA-Seq data, or more generally of target sequences using high-throughput sequencing reads.
Line 377: Line 459:
  
 </code> </code>
 +
 +
  
  
Line 1071: Line 1155:
 ====== LAMMPS CPU/GPU ====== ====== LAMMPS CPU/GPU ======
  
 +  * /share/apps/CENTOS7/lammps/3Mar2020
 +    * lmp_mpi-[double-double|single-double|single-single]
 +  * consult this file for packages installed and compilations
 +    * /share/apps/CENTOS7/lammps/03Mar2020.install
 +    * CUDA_ARCH = sm_75 (probably only works on queue exx96)
 +  * sample script in ~hmeij/k20redo/run.rtx.lammps
  
 +<code>
 +
 +# simple colloid example
 +CPU run -n1
 +Performance: 493311 tau/day, 1141 timesteps/s
 +GPU run gpu4=1 double-double: 1.1x
 +Performance: 571354 tau/day, 1322 timesteps/s
 +GPU run gpu4=1 single-double: 1.2x
 +Performance: 608498 tau/day, 1408 timesteps/s
 +GPU run gpu4=1 single-single: 1.3x
 +Performance: 623405 tau/day, 1443 timesteps/s
 +
 +</code>
     * /share/apps/CENTOS6/lammps/22Aug18     * /share/apps/CENTOS6/lammps/22Aug18
       * consult [[cluster:161|lammps-11Aug17]] page       * consult [[cluster:161|lammps-11Aug17]] page
Line 1816: Line 1919:
 ====== R & R/parallel ====== ====== R & R/parallel ======
  
 +  * https://edwards.sdsu.edu/research/c11-on-centos-6/
 +    * C11++ on CentOS6, two version of gcc
 +    * read the R-3.6.1_configure file in ''/share/apps/src/cottontail2''
 +
 +<code>
 +install.packages('lme4', dependencies = TRUE)
 +install.packages('tidyverse', dependencies = TRUE)
 +install.packages('mice', dependencies = TRUE)
 +install.packages('rpart', dependencies = TRUE)
 +install.packages('party', dependencies = TRUE)
 +install.packages('caret', dependencies = TRUE)
 +install.packages('nnet', dependencies = TRUE)
 +install.packages('e1071', dependencies = TRUE)
 +install.packages('xgboost', dependencies = TRUE)
 +install.packages('kernlab', dependencies = TRUE)
 +install.packages('randomForest', dependencies = TRUE)
 +</code>
 +
 +
 +  * R 3.6.1 (running against "packages", had to comment out libcurl version check)
 +  * New java jdk-12.0.2 (by default in your path)
 +  * packages lme4 and tidyverse installed (dependencies=TRUE)
 +  * https://www.r-bloggers.com/what-are-the-best-machine-learning-packages-in-r/
 +  * packages: mice, rpart, party, caret, randomforest, nnet, e1071, kernLab, xgboost
 +    * note: had to make a link to omp.h from gcc 4.8.4 to OS native gcc...works?
 +
 +<code>
 +
 +  export PATH=/share/apps/CENTOS6/R/packages/bin:$PATH
 +  export LD_LIBRARY_PATH=/share/apps/CENTOS6/R/packages/lib:$LD_LIBRARY_PATH
 +  export PATH=/share/apps/CENTOS6/R/3.6.1b/bin:$PATH
 +  export LD_LIBRARY_PATH=/share/apps/CENTOS6/R/3.6.1b/lib64:$LD_LIBRARY_PATH
 +  export PATH=/share/apps/CENTOS6/gcc/4.8.4/bin:$PATH
 +  export LD_LIBRARY_PATH=/share/apps/CENTOS6/gcc/4.8.4/lib64:/share/apps/CENTOS6/gcc/4.8.4/lib:$LD_LIBRARY_PATH
 +
 +  #export PATH=/share/apps/java/jdk-12.0.2/bin:$PATH
 +  #export LD_LIBRARY_PATH=/share/apps/java/jdk-12.0.2/lib:$LD_LIBRARY_PATH
 +
 +</code>
  
   * R 3.5.1 (new libcurl package)   * R 3.5.1 (new libcurl package)
-  * Uses same old java home 1.7.0_21, should remember to update+  * Uses same old java home 1.7.0_21 (jdk or jre?), should remember to update
   * package lme4 installed   * package lme4 installed
  
Line 1827: Line 1969:
 export PATH=/home/apps/CENTOS6/R/3.5.1/bin:$PATH export PATH=/home/apps/CENTOS6/R/3.5.1/bin:$PATH
 export LD_LIBRARY_PATH=/home/apps/CENTOS6/R/3.5.1/lib64:$LD_LIBRARY_PATH export LD_LIBRARY_PATH=/home/apps/CENTOS6/R/3.5.1/lib64:$LD_LIBRARY_PATH
 +
 +export PATH=/share/apps/java/jdk1.7.0_21/bin:$PATH
 +export LD_LIBRARY_PATH=/share/apps/java/jdk1.7.0_21/lib:$LD_LIBRARY_PATH
  
 </code> </code>
Line 1932: Line 2077:
  
 ====== Amber ======  ====== Amber ====== 
 +
 +** AMBER 18 **
 +
 +  * **/share/apps/CENTOS7/amber/amber18**
 +  * compiled on ''n33'' with a ''/usr/local/n37-cuda-9.2'' soft link
 +  * view compile steps at /share/apps/CENTOS7/amber/amber18.install
 +  * when submitting on ''exx96'' we're mixing 
 +    * 10.2 cuda drivers
 +    * 9.2 toolkits
 +  * some speed results below K20 vs RTX2080s, Amber 16 vs 18
 +  * sample jobs at ~hmeij/k20redo/run.amber[16|18]
 +
 +
 +<code>
 +
 +# compiled on n33
 +[hmeij@n33 amber18]$ ls -l bin/pmemd*
 +-rwxr-xr-x 1 hmeij its  3549803 Mar 30 10:19 bin/pmemd
 +lrwxrwxrwx 1 hmeij its       15 Mar 30 11:57 bin/pmemd.cuda -> pmemd.cuda_SPFP
 +-rwxr-xr-x 1 hmeij its 93895208 Mar 30 12:11 bin/pmemd.cuda_DPFP
 +-rwxr-xr-x 1 hmeij its 94734272 Mar 30 13:07 bin/pmemd.cuda_DPFP.MPI
 +lrwxrwxrwx 1 hmeij its       19 Mar 30 12:51 bin/pmemd.cuda.MPI -> pmemd.cuda_SPFP.MPI
 +-rwxr-xr-x 1 hmeij its 75579744 Mar 30 11:57 bin/pmemd.cuda_SPFP
 +-rwxr-xr-x 1 hmeij its 76418760 Mar 30 12:51 bin/pmemd.cuda_SPFP.MPI
 +-rwxr-xr-x 1 hmeij its  4304392 Mar 30 11:23 bin/pmemd.MPI
 +
 +# some quick results
 +|     Average timings for all steps:
 +
 +~hmeij/k20redo/run.amber18 - mwgpu - K20
 +|         ns/day =       2.84   seconds/ns =   30385.24
 +~hmeij/k20redo/run.amber18 - exx96 - RTX2080S
 +|         ns/day =      11.66   seconds/ns =    7408.08
 +~hmeij/k20redo/run.amber16 - exx96 - RTX2080S
 +|         ns/day =       7.09   seconds/ns =   12185.91
 +
 +
 +</code>
  
 **AMBER 16** **AMBER 16**
Line 1946: Line 2129:
   * docs: doc dir inside install_dir, please read Amber17.pdf   * docs: doc dir inside install_dir, please read Amber17.pdf
   * www: http://amber.scripps.edu   * www: http://amber.scripps.edu
-  * sample script: /home/hmeij/amber/cd/run.cpu+  * sample script: ~hmeij/amber/cd/run.cpu
   * no gpu capabilitities, requires cuda 7.5+   * no gpu capabilitities, requires cuda 7.5+
  
Line 2337: Line 2520:
   * Python 2.7.15   * Python 2.7.15
     * compile with enable-share     * compile with enable-share
-    * numpy and scipy packages+    * numpy and scipy packages, and 
 +    * Successfully installed Send2Trash-1.5.0 backports-abc-0.5 futures-3.3.0 ipaddress-1.0.22  
 +    * ipykernel-4.10.1 jupyter-client-5.3.4 jupyterlab-0.33.12 jupyterlab-launcher-0.11.2  
 +    * notebook-5.7.8 prometheus-client-0.7.1 python-dateutil-2.8.0 singledispatch-3.4.0.3  
 +    * subprocess32-3.5.4 terminado-0.8.2 tornado-5.1.1 
  
 <code> <code>
Line 2521: Line 2709:
  
 For all these installations you will need to set $PATH and $LD_LIBRARY_PATH, look for samples on this page ... For all these installations you will need to set $PATH and $LD_LIBRARY_PATH, look for samples on this page ...
 +
 +  * program: GCC v 9.2.0
 +  * /share/apps/CENTOS6/9.2.0
 +  * note: libgomp disabled for nanopolish compile attempt
 +
 +<code>
 +
 +  wget ftp://ftp.nluug.nl/mirror/languages/gcc/releases/gcc-9.2.0/gcc-9.2.0.tar.gz
 +  tar zxvf gcc-9.2.0.tar.gz
 +  cd gcc-9.2.0
 +  ./contrib/download_prerequisites
 +  ./configure --prefix=/share/apps/CENTOS6/gcc/9.2.0 --disable-libgomp --disable-multilib
 +  make
 +  make install
 +
 +
 +</code>
  
   * program: GCC v 4.8.4   * program: GCC v 4.8.4
cluster/73.txt · Last modified: 2024/05/15 12:58 by hmeij07

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
CC Attribution-Share Alike 4.0 International Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki