Warning: Undefined array key -1 in /usr/share/dokuwiki/inc/html.php on line 1458

Differences

This shows you the differences between two versions of the page.

--- cluster:73 [2022/04/26 13:12]
hmeij07 [Paleogenomics Tools]
+++ cluster:73 [2023/09/21 16:25] (current)
hmeij07 [Sequencing Tools #3]
@@ Line 23: / Line 23: @@
 on either cottontail2 or node n33 located at /share/apps/CENTOS6 \\
 Some compilations require 7.x and are in /share/apps/CENTOS7 \\
+====== Miniconda3-py311 ======
+  * Starting a new branch, the other Miniconda3 section listed on this is quite loaded
+  * Python 3.11
+<code>
+source /share/apps/CENTOS7/miniconda3-py311/etc/profile.d/conda.sh
+or
+export PATH=/share/apps/CENTOS7/miniconda3-py311/bin:$PATH
+export LD_LIBRARY_PATH=/share/apps/CENTOS7/miniconda3-py311/lib:$LD_LIBRARY_PATH
+which mpirun python conda
+conda list
+export CUDA_HOME=/usr/local/cuda
+export PATH=/usr/local/cuda/bin:$PATH
+export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
+export LD_LIBRARY_PATH=/usr/local/cuda/lib:$LD_LIBRARY_PATH
+</code>
+  * conda install (same as what is listed on openhpc [[https://dokuwiki.wesleyan.edu/doku.php?id=cluster:215#miniconda3-py311|miniconda3-py311]]) page
+  * cudatoolkit=10.2 cudnn=7 for exx96 queue
+  * same error, module cudatoolkit not found, but I found Cupy and it does load
+  * https://docs.cupy.dev/en/stable/overview.html
+  * thayerlab
+====== Masurca ======
+  * module: masurca/4.1.0
+  * https://github.com/alekseyzimin/masurca/releases
+  * supports openmp (maybe)
+  * queues mwgpu, exx96 (centos 7)
+<code>
+export PATH=/share/apps/CENTOS7/masurca/4.1.0/bin:$PATH
+export LD_LIBRARY_PATH=/share/apps/CENTOS7/masurca/4.1.0/lib:$LD_LIBRARY_PATH
+[hmeij@cottontail2 ~]$ masurca --version
+version 4.1.0
+</code>
+====== Trimmomatic ======
+  * http://www.usadellab.org/cms/?page=trimmomatic
+  * binary download and stage
+  * java -jar trimmomatic-0.39.jar ...
+<code>
+/share/apps/CENTOS6/Trimmomatic-0.39/
+[root@cottontail Trimmomatic-0.39]# ll
+total 176
+drwxr-xr-x 2 root root      8 Apr  8  2019 adapters
+-rw-r--r-- 1 root root  35147 May 16  2018 LICENSE
+-rw-r--r-- 1 root root 128502 Apr  8  2019 trimmomatic-0.39.jar
+</code>
@@ Line 223: / Line 293: @@
 Suite of tools for high-accuracy basecaling, assembly, polishing, and alignment. Installed in a specific miniconda3 environment (see below).
-  * Requires centos7 so queues mwgpu, amber128, exx96
+  * Requires centos7 so queues mwgpu, <del>amber128</del>, exx96
     * debug server greentail52
   * PREFIX=/share/apps/CENTOS7/miniconda3
@@ Line 233: / Line 303: @@
 <code>
-# environment
+# environment for all modules
 source /share/apps/CENTOS7/miniconda3/etc/profile.d/conda.sh
 export PATH=/share/apps/CENTOS7/miniconda3/bin:$PATH
@@ Line 244: / Line 314: @@
 which conda python pip mpirun gcc cmake
 python --version
+/share/apps/CENTOS7/miniconda3/bin/conda
+/share/apps/CENTOS7/miniconda3/bin/python
+/share/apps/CENTOS7/miniconda3/bin/pip
+/share/apps/CENTOS7/miniconda3/bin/mpirun
+/share/apps/CENTOS7/gcc/9.2.0/bin/gcc
+/share/apps/CENTOS7/cmake/3.12.1/bin/cmake
 </code>
@@ Line 319: / Line 397: @@
 ** Flye **
-Somewhat annoying, no miniconda3 package, so local install, use minconda3 env
+Somewhat annoying, no miniconda3 package, (python 3.9, gcc 9.2). Local building (without installation) on centos7 greentail52.
-  * v2.9
+  * setup env as for all packages in #3, see top of section
+  * v2.9.2 (June 2023)
+  * python bin/flye
+    * change the PATHS accordingly
+<code>
+/share/apps/CENTOS7/flye/2.9.2/bin/flye --version
+.9.2-b1794
+</code>
+  * v2.9.1 (October 2022)
+  * python bin/flye
+    * change the PATHs accordingly
+<code>
+/share/apps/CENTOS7/flye/2.9.1/bin/flye --version
+.9.1-b1780
+</code>
+  * v2.9 (August 2021)
   * python bin/flye
@@ Line 335: / Line 436: @@
 </code>
-  * https://github.com/fenderglass/Flye
-  * miniconda3
-  * De novo assembler for single molecule sequencing reads using repeat graphs
-  * v2.8.3
-<code>
-# To activate this environment, use
-#     $ conda activate flye
-# To deactivate an active environment, use
-#     $ conda deactivate
-</code>
 ** Filtlong **
@@ Line 376: / Line 465: @@
 export LD_LIBRARY_PATH=/share/apps/CENTOS7/bamtools/2.5.2/lib:$LD_LIBRARY_PATH
 ldd `which bamtools`
+</code>
+** BBmap **
+  * location: /share/apps/CENTOS7/bbmap/39.01
+  * java programs
+  * https://sourceforge.net/projects/bbmap/
+** Pilon **
+  * location: /share/apps/CENTOS7/pilon/
+  * just one big jar file
+  * latest java is in $PATH
+<code>
+[hmeij@greentail52 pilon]$ java -jar pilon-1.24.jar --help
+Pilon version 1.24 Thu Jan 28 13:00:45 2021 -0500
+    Usage: pilon --genome genome.fasta [--frags frags.bam] [--jumps jumps.bam] [--unpaired unpaired.bam]
+                 [...other options...]
+           pilon --help for option details
+<snip>
+</code>
+** haslr **
+  * miniconda3 env
+  * https://github.com/vpc-ccg/haslr
+  * conda install -c bioconda haslr
+<code>
+The following NEW packages will be INSTALLED:
+  _openmp_mutex      conda-forge/linux-64::_openmp_mutex-4.5-2_gnu
+  fastutils          bioconda/linux-64::fastutils-0.3-hdcf5f25_4
+  haslr              bioconda/linux-64::haslr-0.8a1-py39hd65a603_4
+  k8                 bioconda/linux-64::k8-0.2.5-hdcf5f25_4
+  libgomp            conda-forge/linux-64::libgomp-13.2.0-h807b86a_2
+  libzlib            conda-forge/linux-64::libzlib-1.2.13-hd590300_5
+  minia              bioconda/linux-64::minia-3.2.6-hdcf5f25_3
+  minimap2           bioconda/linux-64::minimap2-2.26-he4a0461_1
+  pluggy             conda-forge/noarch::pluggy-1.3.0-pyhd8ed1ab_0
+  ruamel.yaml        conda-forge/linux-64::ruamel.yaml-0.17.32-py39hd1e30aa_0
+  ruamel.yaml.clib   conda-forge/linux-64::ruamel.yaml.clib-0.2.7-py39h72bdee0_1
+  toolz              conda-forge/noarch::toolz-0.12.0-pyhd8ed1ab_0
 </code>
@@ Line 659: / Line 797: @@
 Using htslib 1.9
 Copyright (C) 2018 Genome Research Ltd.
+</code>
+  * **iqtree** fcohanlab/jwang06
+<code>
+[hmeij@greentail52 ~]$ iqtree --version
+IQ-TREE multicore version 2.0.3 for Linux 64-bit built Dec 20 2020
+</code>
+  * https://userguide.mdanalysis.org/stable/installation.html
+  * calterlab, annika (used pip to install)
+<code>
+Successfully installed
+GridDataFormats-1.0.1 MDAnalysis-2.3.0
+fasteners-0.18 mmtf-python-1.1.3
+mrcfile-1.4.3 msgpack-1.0.4 numpy-1.23.4
+</code>
+For **Lammps** (starrlab) 23April2023, consult install file
+  *  /share/apps/CENTOS7/lammps/25Apr2023.install
+  * make yes-ml-pace
+  * make yes-gpu
+  * https://pacemaker.readthedocs.io/en/latest/pacemaker/install/
+<code>
+# Note: you must add this line to you env
+export LD_LIBRARY_PATH=/share/apps/CENTOS7/amber/miniconda3/pkgs/cudatoolkit-11.0.221-h6bb024c_0/lib:$LD_LIBRARY_PATH
+# Note2: had to install 'conda install cudnn' and make two links
+to higher level libraries (a must not do)
+for libcudnn.so.8 and libcusolver.so.11
+[hmeij@greentail52 ~]$ which python
+/share/apps/CENTOS7/amber/miniconda3/bin/python
+[hmeij@greentail52 ~]$ pip list | grep tensorflow
+mesh-tensorflow              0.1.19
+tensorflow                   2.8.0
+tensorflow-addons            0.15.0
+tensorflow-datasets          4.4.0
+tensorflow-estimator         2.7.0
+tensorflow-gan               2.1.0
+tensorflow-hub               0.12.0
+tensorflow-io-gcs-filesystem 0.32.0
+tensorflow-metadata          1.4.0
+tensorflow-probability       0.15.0
+[hmeij@greentail52 ~]$ which pacemaker
+/usr/bin/which: no pacemaker in (...failed to install at first, see readme install file)
 </code>
@@ Line 673: / Line 867: @@
   * git clone https://github.com/Amber-MD/pytraj; cd pytra
   * python ./setup.py install
-  * requires CentOS7 so queues mwgpu, amber128 and exx96
+  * requires CentOS7 so queues mwgpu, <del>amber128</del> and exx96
   * or for debugging use greentail52
   * /share/apps/CENTOS7/amber/pytraj
@@ Line 728: / Line 922: @@
 -rw-r--r-- 1 root root      10720 Mar 17 09:35 ohpc-release-2-1.el8.x86_64.rpm
 -rw-r--r-- 1 root root 8531681280 Mar 17 09:33 OpenHPC-2.4.EL_8.x86_64.tar
+# this is all that is needed for ohpc base compute client
+# see the recipe for cct2 and follow compute node only ingredients
+[root@cottontail2 ~]# ll /share/apps/src/cottontail2/ohpc-compute/
+total 95196
+-rw-r--r-- 1 root root  2696360 Feb 10 09:21 hwloc-ohpc-2.7.0-3.9.ohpc.2.6.x86_64.rpm
+-rw-r--r-- 1 root root   266252 Feb 10 09:22 lmod-ohpc-8.7.6-12.3.ohpc.2.6.x86_64.rpm
+-rw-r--r-- 1 root root     7268 Feb 10 09:22 ohpc-base-compute-2.6.1-5.1.ohpc.2.6.1.x86_64.rpm
+-rw-r--r-- 1 root root     8204 Feb 10 09:23 ohpc-filesystem-2.6-2.3.ohpc.2.6.noarch.rpm
+-rw-r--r-- 1 root root    10720 Feb 10 09:23 ohpc-release-2-1.el8.x86_64.rpm
+-rw-r--r-- 1 root root     7124 Feb 10 09:27 ohpc-slurm-client-2.6.1-5.1.ohpc.2.6.1.x86_64.rpm
+-rw-r--r-- 1 root root 74326532 Feb 10 09:23 singularity-ohpc-3.7.1-5.1.ohpc.2.1.x86_64.rpm
+-rw-r--r-- 1 root root    22168 Feb 10 09:24 slurm-contribs-ohpc-22.05.2-14.1.ohpc.2.6.x86_64.rpm
+-rw-r--r-- 1 root root   247392 Feb 10 09:24 slurm-example-configs-ohpc-22.05.2-14.1.ohpc.2.6.x86_64.rpm
+-rw-r--r-- 1 root root 18764472 Feb 10 09:24 slurm-ohpc-22.05.2-14.1.ohpc.2.6.x86_64.rpm
+-rw-r--r-- 1 root root   176480 Feb 10 09:28 slurm-pam_slurm-ohpc-22.05.2-14.1.ohpc.2.6.x86_64.rpm
+-rw-r--r-- 1 root root   785112 Feb 10 09:25 slurm-slurmd-ohpc-22.05.2-14.1.ohpc.2.6.x86_64.rpm
 </code>
@@ Line 966: / Line 1180: @@
   * **PCAngsd**, requires miniconda3's python
   * https://github.com/Rosemeis/pcangsd
-  * apparently the repo is needed ''share/apps/CENTOS7/pcangsd/1.1.0''
+  * apparently the repo is needed ''/share/apps/CENTOS7/pcangsd/1.1.0''
 <code>
@@ Line 978: / Line 1192: @@
 pcangsd --version
 pcangsd 1.10
+</code>
+  * **angsd**, requires miniconda3, same environment as pcangsd
+  * http://www.popgen.dk/angsd/index.php/RealSFS
+  * add location of angsd to PATH env
+<code>
+$ pwd
+/share/apps/CENTOS7/angsd/0.940/angsd
+$ ls -l angsd
+-rwxr-xr-x 1 hmeij its 5517248 Jul  6 14:11 angsd
+$ ./angsd
+	-> angsd version: 0.940-dirty (htslib: 1.16) build(Jul  6 2023 14:11:30)
 </code>
@@ Line 1355: / Line 1586: @@
 cufflinks                 2.2.1                    py27_2    bioconda
 samtools                  1.13                 h8c37831_0
+# fcohan lab
+[hmeij@petaltail ~]$ iqtree --version
+IQ-TREE multicore version 2.1.4-beta COVID-edition for Linux 64-bit built Jun 24 2021
 </code>
 **Trinity** assembles transcript sequences from Illumina RNA-Seq data.
@@ Line 1992: / Line 2229: @@
 [[https://dokuwiki.wesleyan.edu/doku.php?id=cluster:119|GPU version]]
+  * program v13,2.1 installed
+  * program v13.2 installed
   * program: mcc, math, mathematica (v 12.2)
@@ Line 2189: / Line 2430: @@
 ====== LAMMPS CPU/GPU ======
+** 25Apr2023**
+  * /share/apps/CENTOS7/lammps/25Arp2023
+  * lmp_serial and lmp_mpi
+  * 3 cuda versions with 3 different precision settings
+  * notes: /share/apps/CENTOS7/lammps/25Apr2023.install
+  * does **not** support openmp, look for the OpenHPC version for that
+  * 05/24/2023 remade with package CREATION added
+  * also read miniconda3 section for tensorflow/pacemaker info
+<code>
+# CPU
+ export PATH=/share/apps/CENTOS7/gcc/6.5.0/bin/:$PATH
+ export LD_LIBRARY_PATH=/share/apps/CENTOS7/gcc/6.5.0/lib64:$LD_LIBRARY_PATH
+ export PATH=/share/apps/CENTOS7/python/3.8.3/bin:$PATH
+ export LD_LIBRARY_PATH=/share/apps/CENTOS7/python/3.8.3/lib:$LD_LIBRARY_PATH
+ export PATH=/share/apps/CENTOS7/openmpi/4.0.4/bin:$PATH
+ export LD_LIBRARY_PATH=/share/apps/CENTOS7/openmpi/4.0.4/lib:$LD_LIBRARY_PATH
+ which mpirun mpicc python gcc
+# CPU + GPU (exx96 queue, may run on test and amber128 queues)
+ export CUDA_HOME=/usr/local/cuda
+ export PATH=/usr/local/cuda/bin:$PATH
+ export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
+ export LD_LIBRARY_PATH=/usr/local/cuda/lib:$LD_LIBRARY_PATH
+ which nvcc
+# related for starrlab
+# https://pacemaker.readthedocs.io/en/latest/pacemaker/install/
+[hmeij@cottontail2 ~]$ module load miniconda3/py39
+[hmeij@cottontail2 ~]$ which pacemaker
+/share/apps/CENTOS8/ohpc/software/miniconda3/py39/bin/pacemaker
+[hmeij@cottontail2 ~]$ pacemaker --version
+pacemaker/pyace version: 0.2.7+77.g8b40c11
+ace_evaluator   version: 2022.6.27
+[hmeij@cottontail2 ~]$ python
+Python 3.9.10 | packaged by conda-forge | (main, Feb  1 2022, 21:24:11)
+[GCC 9.4.0] on linux
+Type "help", "copyright", "credits" or "license" for more information.
+>>> import pyace as py
+>>>
+</code>
 ** USER_QUIP **
@@ Line 3246: / Line 3538: @@
 ====== Amber ======
+** Amber 22 **
+Details on how to run on CentOS7 can be found at
+  * https://dokuwiki.wesleyan.edu/doku.php?id=cluster:214#amber22
 ** Amber 20 **
@@ Line 3673: / Line 3971: @@
 <code>
+# update
+[hmeij@greentail52 ~]$ java --version
+java 20.0.2 2023-07-18
 export JAVAHOME=/home/apps/j2sdk1.4.2_17/bin:$PATH
 export CLASSPATH=$JAVAHOME/lib:$CLASSPATH
@@ Line 4027: / Line 4331: @@
 ====== Matlab ======
+  * installed R2023a
+  * installed R2022a
+  * installed R2021a
   * program: Matlab, link to binary is 'matlab' in ''/share/apps/bin''
@@ Line 4047: / Line 4355: @@
 ====== Stata ======
+  * program stata[-mp|-se] in your $PATH, version 18
+  * v17 will still run so add '17' to the command like 'stata-mp17'
+  * so that makes for 12 licenses 8-)
   * program: Stata v 17, 6-user network license, includes stata, stata-mp, stata-se

DokuWiki

User Tools

Site Tools

Differences

Page Tools