DokuWiki

User Tools

Site Tools

Trace:
cluster:73

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
Next revision Both sides next revision
cluster:73 [2023/04/27 14:30]
hmeij07 [LAMMPS CPU/GPU] 		cluster:73 [2023/11/10 20:32]
hmeij07 [Paleogenomics Tools]
Line 25: Line 25:
  
  
 +
 +====== Miniconda3-py311 ======
 +
 +  * Starting a new branch, the other Miniconda3 section listed on this is quite loaded
 +  * Python 3.11
 +
 +<code>
 +
 +source /share/apps/CENTOS7/miniconda3-py311/etc/profile.d/conda.sh
 +or
 +export PATH=/share/apps/CENTOS7/miniconda3-py311/bin:$PATH
 +export LD_LIBRARY_PATH=/share/apps/CENTOS7/miniconda3-py311/lib:$LD_LIBRARY_PATH
 +which mpirun python conda
 +conda list
 +
 +export CUDA_HOME=/usr/local/cuda
 +export PATH=/usr/local/cuda/bin:$PATH
 +export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
 +export LD_LIBRARY_PATH=/usr/local/cuda/lib:$LD_LIBRARY_PATH
 +
 +
 +</code>
 +
 +  * conda install (same as what is listed on openhpc [[https://dokuwiki.wesleyan.edu/doku.php?id=cluster:215#miniconda3-py311|miniconda3-py311]]) page
 +  * cudatoolkit=10.2 cudnn=7 for exx96 queue
 +  * same error, module cudatoolkit not found, but I found Cupy and it does load
 +  * https://docs.cupy.dev/en/stable/overview.html
 +  * thayerlab
 +
 +<code>
 +
 +# pollacklab
 +
 +[hmeij@greentail52 ~]$ source /share/apps/CENTOS7/miniconda3-py311/etc/profile.d/conda.sh
 +
 +[hmeij@greentail52 ~]$ conda activate sage
 +
 +(sage) [hmeij@greentail52 ~]$ sage --version
 +SageMath version 10.0, Release Date: 2023-05-20
 +
 +(sage) [hmeij@greentail52 ~]$ sage
 +┌────────────────────────────────────────────────────────────────────┐
 +│ SageMath version 10.0, Release Date: 2023-05-20                    │
 +│ Using Python 3.11.4. Type "help()" for help.                       │
 +└────────────────────────────────────────────────────────────────────┘
 +
 +sage: exit
 +(sage) [hmeij@greentail52 ~]$ conda deactivate
 +[hmeij@greentail52 ~]$ 
 +
 +</code>
 +
 +  * **mdtraj**
 +
 +<code>
 +
 +[hmeij@greentail52 ~]$ which python                                                                                      
 +/share/apps/CENTOS7/miniconda3-py311/bin/python
 +                                                                          
 +[hmeij@greentail52 ~]$ python
 +Python 3.11.4 (main, Jul  5 2023, 13:45:01) [GCC 11.2.0] on linux
 +Type "help", "copyright", "credits" or "license" for more information.
 +>>> import mdtraj as mt
 +>>> 
 +
 +</code>
 +
 +====== Masurca ======
 +
 +
 +  * module: masurca/4.1.0
 +  * https://github.com/alekseyzimin/masurca/releases
 +  * supports openmp (maybe)
 +  * queues mwgpu, exx96 (centos 7)
 +
 +
 +<code>
 +
 +export PATH=/share/apps/CENTOS7/masurca/4.1.0/bin:$PATH
 +export LD_LIBRARY_PATH=/share/apps/CENTOS7/masurca/4.1.0/lib:$LD_LIBRARY_PATH
 +
 +
 +[hmeij@cottontail2 ~]$ masurca --version
 +version 4.1.0
 +
 +
 +</code>
  
 ====== Trimmomatic ====== ====== Trimmomatic ======
Line 346: Line 433:
 ** Flye ** ** Flye **
  
-Somewhat annoying, no miniconda3 package, so local install, use minconda3 environment as listed top of this section (python 3.9, gcc 9.2)+Somewhat annoying, no miniconda3 package, (python 3.9, gcc 9.2). Local building (without installation) on centos7 greentail52. 
 + 
 +  * setup env as for all packages in #3, see top of section 
 +  * v2.9.2 (June 2023) 
 +  * python bin/flye 
 +    * change the PATHS accordingly 
 + 
 +<code> 
 + 
 +/share/apps/CENTOS7/flye/2.9.2/bin/flye --version 
 +2.9.2-b1794 
 + 
 +</code>
  
   * v2.9.1 (October 2022)   * v2.9.1 (October 2022)
Line 373: Line 472:
 </code> </code>
  
-  * https://github.com/fenderglass/Flye  
-  * miniconda3  
-  * De novo assembler for single molecule sequencing reads using repeat graphs  
-  * v2.8.3 
  
-<code> 
- 
-# To activate this environment, use 
-#     $ conda activate flye 
-# To deactivate an active environment, use 
-#     $ conda deactivate 
- 
-</code> 
  
 ** Filtlong ** ** Filtlong **
Line 417: Line 504:
 </code> </code>
  
 +** BBmap **
 +
 +  * location: /share/apps/CENTOS7/bbmap/39.01
 +  * java programs
 +  * https://sourceforge.net/projects/bbmap/
 +
 +** Pilon **
 +
 +  * location: /share/apps/CENTOS7/pilon/
 +  * just one big jar file
 +  * latest java is in $PATH
 +
 +<code>
 +
 +[hmeij@greentail52 pilon]$ java -jar pilon-1.24.jar --help
 +Pilon version 1.24 Thu Jan 28 13:00:45 2021 -0500
 +
 +    Usage: pilon --genome genome.fasta [--frags frags.bam] [--jumps jumps.bam] [--unpaired unpaired.bam]
 +                 [...other options...]
 +           pilon --help for option details 
 +<snip>
 +
 +</code>
 +
 +** haslr **
 +
 +  * miniconda3 env
 +  * https://github.com/vpc-ccg/haslr
 +  * conda install -c bioconda haslr
 +
 +<code>
 +
 +The following NEW packages will be INSTALLED:
 +
 +  _openmp_mutex      conda-forge/linux-64::_openmp_mutex-4.5-2_gnu
 +  fastutils          bioconda/linux-64::fastutils-0.3-hdcf5f25_4
 +  haslr              bioconda/linux-64::haslr-0.8a1-py39hd65a603_4
 +  k8                 bioconda/linux-64::k8-0.2.5-hdcf5f25_4
 +  libgomp            conda-forge/linux-64::libgomp-13.2.0-h807b86a_2
 +  libzlib            conda-forge/linux-64::libzlib-1.2.13-hd590300_5
 +  minia              bioconda/linux-64::minia-3.2.6-hdcf5f25_3
 +  minimap2           bioconda/linux-64::minimap2-2.26-he4a0461_1
 +  pluggy             conda-forge/noarch::pluggy-1.3.0-pyhd8ed1ab_0
 +  ruamel.yaml        conda-forge/linux-64::ruamel.yaml-0.17.32-py39hd1e30aa_0
 +  ruamel.yaml.clib   conda-forge/linux-64::ruamel.yaml.clib-0.2.7-py39h72bdee0_1
 +  toolz              conda-forge/noarch::toolz-0.12.0-pyhd8ed1ab_0
 +
 +</code>
 +
 +Compiled from source, do **not** source miniconda3, we are using standalone pyhton3
 +
 +<code>
 +
 +export PATH=/share/apps/CENTOS7/gcc/9.2.0/bin:$PATH
 +export LD_LIBRARY_PATH=/share/apps/CENTOS7/gcc/9.2.0/lib64:$LD_LIBRARY_PATH
 +export LD_LIBRARY_PATH=/share/apps/CENTOS7/gcc/9.2.0/lib:$LD_LIBRARY_PATH
 +export PATH=/share/apps/CENTOS7/python/3.8.3/bin:$PATH
 +export LD_LIBRARY_PATH=/share/apps/CENTOS7/python/3.8.3/lib:$LD_LIBRARY_PATH
 +[hmeij@greentail52 20231006]$ which python
 +/share/apps/CENTOS7/python/3.8.3/bin/python
 +
 +[hmeij@greentail52 20231006]$ pwd
 +/share/apps/CENTOS7/haslr/20231006
 +
 +[hmeij@greentail52 20231006]$ ll bin
 +total 11448
 +-rwxr-xr-x 1 hmeij its   123448 Oct  6 15:01 fastutils
 +-rwxr-xr-x 1 hmeij its  3823032 Oct  6 15:04 haslr_assemble
 +-rwxr-xr-x 1 hmeij its    19949 Oct  6 14:48 haslr.py
 +-rwxr-xr-x 1 hmeij its 20969840 Oct  6 14:59 minia
 +-rwxr-xr-x 1 hmeij its    23856 Oct  6 15:01 minia_nooverlap
 +-rwxr-xr-x 1 hmeij its   951922 Oct  6 14:59 minimap2
 +
 +</code>
 +
 +
 +** stLFR **
 +
 +  * same centos 7 environment
 +  * https://github.com/BGI-Qingdao/stLFR_GapCloser
 +
 +<code>
 +
 +cd /share/apps/CENTOS7/stLFR/20230929
 +
 +[hmeij@greentail52 20230929]$ ./Release/stLFR_GapCloser -h
 +Version:
 + 1.00
 +
 +Contact:
 + dengli1@genomics.cn && guolidong@genomics.cn
 +
 +Usage:
 + stLFR_GapCloser [options]
 +...
 +
 +
 +# and another request by Luis
 +# https://sourceforge.net/projects/soapdenovo2/files/SOAPdenovo2
 +
 +[hmeij@greentail52 stLFR]$ pwd 
 +/share/apps/CENTOS7/stLFR
 +[hmeij@greentail52 stLFR]$ ll GapCloser-bin-v1.12-r6/
 +total 175
 +-rwxr-xr-x 1 hmeij its 184713 Jan 11  2013 GapCloser
 +-rw-r--r-- 1 hmeij its  44533 Jan 11  2013 GapCloser_Manual.pdf
 +
 +</code>
 +
 +** BUSCO **
 +
 +  * https://busco.ezlab.org/busco_userguide.html#docker-image
 +  * conda package fails on glib 2.17, weird
 +
 +<code>
 +
 +[hmeij@n79 ~]$ docker images
 +REPOSITORY                         TAG                            IMAGE ID            CREATED             SIZE
 +ezlabgva/busco                     v5.5.0_cv1                     30d3e831aba7        2 months ago        2.88GB
 +
 +
 +[hmeij@n79 busco_wd]$ docker run --rm -v $(pwd):/busco_wd -u $(id -u):$(id -g) ezlabgva/busco:v5.5.0_cv1 busco -i /busco_wd/assembly.fasta -l actinopterygii_odb10 -o Busco_Output -m geno -f
 +2023-10-28 20:30:00 INFO:       ***** Start a BUSCO v5.5.0 analysis, current time: 10/28/2023 20:30:00 *****
 +2023-10-28 20:30:00 INFO:       Configuring BUSCO with local environment
 +2023-10-28 20:30:00 INFO:       Mode is genome
 +2023-10-28 20:30:00 INFO:       'Force' option selected; overwriting previous results directory
 +2023-10-28 20:30:00 INFO:       Downloading information on latest versions of BUSCO data...
 +
 +that's a problem, trying to reach the internet ...
 +
 +</code>
 +
 +** picard **
 +
 +  * https://github.com/broadinstitute/picard
 +  * Warning: using Java 20 but only Java 17 has been tested.
 +
 +<code>
 +
 +picard centos7, weird
 +> Failed to load native library 'libnative-platform.so' for Linux amd64.
 +
 +picard centos8
 +
 +[hmeij@cottontail2 20231108]$ pwd
 +/share/apps/CENTOS8/picard/20231108
 +
 +[hmeij@cottontail2 20231108]$ ls -l build/libs/
 +total 119872
 +-rw-r--r-- 1 hmeij its 62520451 Nov  8 11:46 picard-3.1.0-3-ga9194bd97-SNAPSHOT-all.jar
 +-rw-r--r-- 1 hmeij its  1856980 Nov  8 11:01 picard-3.1.0-3-ga9194bd97-SNAPSHOT.jar
 +-rw-r--r-- 1 hmeij its 62520451 Nov  8 11:46 picard.jar
 +[hmeij@cottontail2 20231108]$ 
 +
 +
 +</code>
 ====== Galario CPU+GPU ====== ====== Galario CPU+GPU ======
  
Line 719: Line 962:
 fasteners-0.18 mmtf-python-1.1.3  fasteners-0.18 mmtf-python-1.1.3 
 mrcfile-1.4.3 msgpack-1.0.4 numpy-1.23.4 mrcfile-1.4.3 msgpack-1.0.4 numpy-1.23.4
 +
 +</code>
 +
 +For **Lammps** (starrlab) 23April2023, consult install file
 +
 +  *  /share/apps/CENTOS7/lammps/25Apr2023.install
 +  * make yes-ml-pace
 +  * make yes-gpu
 +  * https://pacemaker.readthedocs.io/en/latest/pacemaker/install/
 +
 +<code>
 +
 +# Note: you must add this line to you env
 +export LD_LIBRARY_PATH=/share/apps/CENTOS7/amber/miniconda3/pkgs/cudatoolkit-11.0.221-h6bb024c_0/lib:$LD_LIBRARY_PATH
 +
 +# Note2: had to install 'conda install cudnn' and make two links 
 +to higher level libraries (a must not do) 
 +for libcudnn.so.8 and libcusolver.so.11
 +
 +[hmeij@greentail52 ~]$ which python
 +/share/apps/CENTOS7/amber/miniconda3/bin/python
 +[hmeij@greentail52 ~]$ pip list | grep tensorflow
 +mesh-tensorflow              0.1.19
 +tensorflow                   2.8.0
 +tensorflow-addons            0.15.0
 +tensorflow-datasets          4.4.0
 +tensorflow-estimator         2.7.0
 +tensorflow-gan               2.1.0
 +tensorflow-hub               0.12.0
 +tensorflow-io-gcs-filesystem 0.32.0
 +tensorflow-metadata          1.4.0
 +tensorflow-probability       0.15.0
 +[hmeij@greentail52 ~]$ which pacemaker
 +/usr/bin/which: no pacemaker in (...failed to install at first, see readme install file)
  
 </code> </code>
Line 733: Line 1010:
   * git clone https://github.com/Amber-MD/pytraj; cd pytra   * git clone https://github.com/Amber-MD/pytraj; cd pytra
   * python ./setup.py install   * python ./setup.py install
-  * requires CentOS7 so queues mwgpu, amber128 and exx96+  * requires CentOS7 so queues mwgpu, <del>amber128</del> and exx96
   * or for debugging use greentail52   * or for debugging use greentail52
   * /share/apps/CENTOS7/amber/pytraj   * /share/apps/CENTOS7/amber/pytraj
Line 1058: Line 1335:
 pcangsd --version pcangsd --version
 pcangsd 1.10 pcangsd 1.10
 +
 +</code>
 +
 +  * **angsd**, requires miniconda3, same environment as pcangsd
 +  * http://www.popgen.dk/angsd/index.php/RealSFS
 +  * add location of angsd to PATH env 
 +
 +<code>
 +
 +$ pwd
 +/share/apps/CENTOS7/angsd/0.940/angsd
 +
 +$ ls -l angsd
 +-rwxr-xr-x 1 hmeij its 5517248 Jul  6 14:11 angsd
 +
 +$ ./angsd 
 + -> angsd version: 0.940-dirty (htslib: 1.16) build(Jul  6 2023 14:11:30)
  
 </code> </code>
Line 1076: Line 1370:
  
 </code> </code>
 +
 +
 +  * **qualimap **
 +  * http://qualimap.conesalab.org/
 +  * java & R app, probably will work in centos7 and rocky8
 +  * /share/apps/CENTOS7/qualimap/2.3
 +
 +  * **gatk**
 +  * https://github.com/broadinstitute/gatk
 +  * java & R app. probably will work in centos7 and rocky8
 +  *  /share/apps/CENTOS7/gatk/4.4.0.0
 +
 +For both apps above java is v20, developers want and tested v17
 +
 +For both apps you may find R local in operating system or a recent versions at
 +
 +  * (rocky 8) module load R/4.1.2
 +
 +
 +
  
  
Line 2078: Line 2392:
  
 [[https://dokuwiki.wesleyan.edu/doku.php?id=cluster:119|GPU version]] [[https://dokuwiki.wesleyan.edu/doku.php?id=cluster:119|GPU version]]
 +
 +  * program v13,2.1 installed
 +
 +  * program v13.2 installed
  
   * program: mcc, math, mathematica (v 12.2)   * program: mcc, math, mathematica (v 12.2)
Line 2282: Line 2600:
   * 3 cuda versions with 3 different precision settings   * 3 cuda versions with 3 different precision settings
   * notes: /share/apps/CENTOS7/lammps/25Apr2023.install   * notes: /share/apps/CENTOS7/lammps/25Apr2023.install
 +  * does **not** support openmp, look for the OpenHPC version for that
 +  * 05/24/2023 remade with package CREATION added
 +  * also read miniconda3 section for tensorflow/pacemaker info
  
 <code> <code>
Line 2287: Line 2608:
 # CPU # CPU
  
 + export PATH=/share/apps/CENTOS7/gcc/6.5.0/bin/:$PATH
 + export LD_LIBRARY_PATH=/share/apps/CENTOS7/gcc/6.5.0/lib64:$LD_LIBRARY_PATH
  export PATH=/share/apps/CENTOS7/python/3.8.3/bin:$PATH  export PATH=/share/apps/CENTOS7/python/3.8.3/bin:$PATH
  export LD_LIBRARY_PATH=/share/apps/CENTOS7/python/3.8.3/lib:$LD_LIBRARY_PATH  export LD_LIBRARY_PATH=/share/apps/CENTOS7/python/3.8.3/lib:$LD_LIBRARY_PATH
  export PATH=/share/apps/CENTOS7/openmpi/4.0.4/bin:$PATH  export PATH=/share/apps/CENTOS7/openmpi/4.0.4/bin:$PATH
  export LD_LIBRARY_PATH=/share/apps/CENTOS7/openmpi/4.0.4/lib:$LD_LIBRARY_PATH  export LD_LIBRARY_PATH=/share/apps/CENTOS7/openmpi/4.0.4/lib:$LD_LIBRARY_PATH
- which mpirun mpicc python+ which mpirun mpicc python gcc 
  
 # CPU + GPU (exx96 queue, may run on test and amber128 queues) # CPU + GPU (exx96 queue, may run on test and amber128 queues)
Line 2302: Line 2626:
  
 # related for starrlab # related for starrlab
 +# https://pacemaker.readthedocs.io/en/latest/pacemaker/install/
  
 [hmeij@cottontail2 ~]$ module load miniconda3/py39 [hmeij@cottontail2 ~]$ module load miniconda3/py39
Line 3809: Line 4134:
  
 <code> <code>
 +
 +# update
 +[hmeij@greentail52 ~]$ java --version
 +java 20.0.2 2023-07-18
 +
 +
 export JAVAHOME=/home/apps/j2sdk1.4.2_17/bin:$PATH export JAVAHOME=/home/apps/j2sdk1.4.2_17/bin:$PATH
 export CLASSPATH=$JAVAHOME/lib:$CLASSPATH export CLASSPATH=$JAVAHOME/lib:$CLASSPATH
Line 4163: Line 4494:
 ====== Matlab ====== ====== Matlab ======
  
 +  * installed R2023a
  
 +  * installed R2022a
 +
 +  * installed R2021a
  
   * program: Matlab, link to binary is 'matlab' in ''/share/apps/bin''   * program: Matlab, link to binary is 'matlab' in ''/share/apps/bin''
Line 4183: Line 4518:
  
 ====== Stata ====== ====== Stata ======
 +
 +  * program stata[-mp|-se] in your $PATH, version 18
 +  * v17 will still run so add '17' to the command like 'stata-mp17'
 +  * so that makes for 12 licenses 8-)
  
   * program: Stata v 17, 6-user network license, includes stata, stata-mp, stata-se    * program: Stata v 17, 6-user network license, includes stata, stata-mp, stata-se 
cluster/73.txt · Last modified: 2024/05/15 12:58 by hmeij07

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
CC Attribution-Share Alike 4.0 International Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki