DokuWiki

User Tools

Site Tools

Trace:
cluster:73

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
Next revision Both sides next revision
cluster:73 [2023/06/12 13:54]
hmeij07 [Miniconda3] 		cluster:73 [2023/11/10 20:15]
hmeij07 [Paleogenomics Tools]
Line 25: Line 25:
  
  
 +
 +====== Miniconda3-py311 ======
 +
 +  * Starting a new branch, the other Miniconda3 section listed on this is quite loaded
 +  * Python 3.11
 +
 +<code>
 +
 +source /share/apps/CENTOS7/miniconda3-py311/etc/profile.d/conda.sh
 +or
 +export PATH=/share/apps/CENTOS7/miniconda3-py311/bin:$PATH
 +export LD_LIBRARY_PATH=/share/apps/CENTOS7/miniconda3-py311/lib:$LD_LIBRARY_PATH
 +which mpirun python conda
 +conda list
 +
 +export CUDA_HOME=/usr/local/cuda
 +export PATH=/usr/local/cuda/bin:$PATH
 +export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
 +export LD_LIBRARY_PATH=/usr/local/cuda/lib:$LD_LIBRARY_PATH
 +
 +
 +</code>
 +
 +  * conda install (same as what is listed on openhpc [[https://dokuwiki.wesleyan.edu/doku.php?id=cluster:215#miniconda3-py311|miniconda3-py311]]) page
 +  * cudatoolkit=10.2 cudnn=7 for exx96 queue
 +  * same error, module cudatoolkit not found, but I found Cupy and it does load
 +  * https://docs.cupy.dev/en/stable/overview.html
 +  * thayerlab
 +
 +<code>
 +
 +# pollacklab
 +
 +[hmeij@greentail52 ~]$ source /share/apps/CENTOS7/miniconda3-py311/etc/profile.d/conda.sh
 +
 +[hmeij@greentail52 ~]$ conda activate sage
 +
 +(sage) [hmeij@greentail52 ~]$ sage --version
 +SageMath version 10.0, Release Date: 2023-05-20
 +
 +(sage) [hmeij@greentail52 ~]$ sage
 +┌────────────────────────────────────────────────────────────────────┐
 +│ SageMath version 10.0, Release Date: 2023-05-20                    │
 +│ Using Python 3.11.4. Type "help()" for help.                       │
 +└────────────────────────────────────────────────────────────────────┘
 +
 +sage: exit
 +(sage) [hmeij@greentail52 ~]$ conda deactivate
 +[hmeij@greentail52 ~]$ 
 +
 +</code>
 +
 +  * **mdtraj**
 +
 +<code>
 +
 +[hmeij@greentail52 ~]$ which python                                                                                      
 +/share/apps/CENTOS7/miniconda3-py311/bin/python
 +                                                                          
 +[hmeij@greentail52 ~]$ python
 +Python 3.11.4 (main, Jul  5 2023, 13:45:01) [GCC 11.2.0] on linux
 +Type "help", "copyright", "credits" or "license" for more information.
 +>>> import mdtraj as mt
 +>>> 
 +
 +</code>
 +
 +====== Masurca ======
 +
 +
 +  * module: masurca/4.1.0
 +  * https://github.com/alekseyzimin/masurca/releases
 +  * supports openmp (maybe)
 +  * queues mwgpu, exx96 (centos 7)
 +
 +
 +<code>
 +
 +export PATH=/share/apps/CENTOS7/masurca/4.1.0/bin:$PATH
 +export LD_LIBRARY_PATH=/share/apps/CENTOS7/masurca/4.1.0/lib:$LD_LIBRARY_PATH
 +
 +
 +[hmeij@cottontail2 ~]$ masurca --version
 +version 4.1.0
 +
 +
 +</code>
  
 ====== Trimmomatic ====== ====== Trimmomatic ======
Line 346: Line 433:
 ** Flye ** ** Flye **
  
-Somewhat annoying, no miniconda3 package, so local install, use minconda3 environment as listed top of this section (python 3.9, gcc 9.2). Local building (without installation) on centos7 greentail52.+Somewhat annoying, no miniconda3 package, (python 3.9, gcc 9.2). Local building (without installation) on centos7 greentail52.
  
 +  * setup env as for all packages in #3, see top of section
   * v2.9.2 (June 2023)   * v2.9.2 (June 2023)
   * python bin/flye   * python bin/flye
-    * chnage the PATHS accordingly+    * change the PATHS accordingly
  
 <code> <code>
Line 384: Line 472:
 </code> </code>
  
-  * https://github.com/fenderglass/Flye  
-  * miniconda3  
-  * De novo assembler for single molecule sequencing reads using repeat graphs  
-  * v2.8.3 
  
-<code> 
- 
-# To activate this environment, use 
-#     $ conda activate flye 
-# To deactivate an active environment, use 
-#     $ conda deactivate 
- 
-</code> 
  
 ** Filtlong ** ** Filtlong **
Line 428: Line 504:
 </code> </code>
  
 +** BBmap **
 +
 +  * location: /share/apps/CENTOS7/bbmap/39.01
 +  * java programs
 +  * https://sourceforge.net/projects/bbmap/
 +
 +** Pilon **
 +
 +  * location: /share/apps/CENTOS7/pilon/
 +  * just one big jar file
 +  * latest java is in $PATH
 +
 +<code>
 +
 +[hmeij@greentail52 pilon]$ java -jar pilon-1.24.jar --help
 +Pilon version 1.24 Thu Jan 28 13:00:45 2021 -0500
 +
 +    Usage: pilon --genome genome.fasta [--frags frags.bam] [--jumps jumps.bam] [--unpaired unpaired.bam]
 +                 [...other options...]
 +           pilon --help for option details 
 +<snip>
 +
 +</code>
 +
 +** haslr **
 +
 +  * miniconda3 env
 +  * https://github.com/vpc-ccg/haslr
 +  * conda install -c bioconda haslr
 +
 +<code>
 +
 +The following NEW packages will be INSTALLED:
 +
 +  _openmp_mutex      conda-forge/linux-64::_openmp_mutex-4.5-2_gnu
 +  fastutils          bioconda/linux-64::fastutils-0.3-hdcf5f25_4
 +  haslr              bioconda/linux-64::haslr-0.8a1-py39hd65a603_4
 +  k8                 bioconda/linux-64::k8-0.2.5-hdcf5f25_4
 +  libgomp            conda-forge/linux-64::libgomp-13.2.0-h807b86a_2
 +  libzlib            conda-forge/linux-64::libzlib-1.2.13-hd590300_5
 +  minia              bioconda/linux-64::minia-3.2.6-hdcf5f25_3
 +  minimap2           bioconda/linux-64::minimap2-2.26-he4a0461_1
 +  pluggy             conda-forge/noarch::pluggy-1.3.0-pyhd8ed1ab_0
 +  ruamel.yaml        conda-forge/linux-64::ruamel.yaml-0.17.32-py39hd1e30aa_0
 +  ruamel.yaml.clib   conda-forge/linux-64::ruamel.yaml.clib-0.2.7-py39h72bdee0_1
 +  toolz              conda-forge/noarch::toolz-0.12.0-pyhd8ed1ab_0
 +
 +</code>
 +
 +Compiled from source, do **not** source miniconda3, we are using standalone pyhton3
 +
 +<code>
 +
 +export PATH=/share/apps/CENTOS7/gcc/9.2.0/bin:$PATH
 +export LD_LIBRARY_PATH=/share/apps/CENTOS7/gcc/9.2.0/lib64:$LD_LIBRARY_PATH
 +export LD_LIBRARY_PATH=/share/apps/CENTOS7/gcc/9.2.0/lib:$LD_LIBRARY_PATH
 +export PATH=/share/apps/CENTOS7/python/3.8.3/bin:$PATH
 +export LD_LIBRARY_PATH=/share/apps/CENTOS7/python/3.8.3/lib:$LD_LIBRARY_PATH
 +[hmeij@greentail52 20231006]$ which python
 +/share/apps/CENTOS7/python/3.8.3/bin/python
 +
 +[hmeij@greentail52 20231006]$ pwd
 +/share/apps/CENTOS7/haslr/20231006
 +
 +[hmeij@greentail52 20231006]$ ll bin
 +total 11448
 +-rwxr-xr-x 1 hmeij its   123448 Oct  6 15:01 fastutils
 +-rwxr-xr-x 1 hmeij its  3823032 Oct  6 15:04 haslr_assemble
 +-rwxr-xr-x 1 hmeij its    19949 Oct  6 14:48 haslr.py
 +-rwxr-xr-x 1 hmeij its 20969840 Oct  6 14:59 minia
 +-rwxr-xr-x 1 hmeij its    23856 Oct  6 15:01 minia_nooverlap
 +-rwxr-xr-x 1 hmeij its   951922 Oct  6 14:59 minimap2
 +
 +</code>
 +
 +
 +** stLFR **
 +
 +  * same centos 7 environment
 +  * https://github.com/BGI-Qingdao/stLFR_GapCloser
 +
 +<code>
 +
 +cd /share/apps/CENTOS7/stLFR/20230929
 +
 +[hmeij@greentail52 20230929]$ ./Release/stLFR_GapCloser -h
 +Version:
 + 1.00
 +
 +Contact:
 + dengli1@genomics.cn && guolidong@genomics.cn
 +
 +Usage:
 + stLFR_GapCloser [options]
 +...
 +
 +
 +# and another request by Luis
 +# https://sourceforge.net/projects/soapdenovo2/files/SOAPdenovo2
 +
 +[hmeij@greentail52 stLFR]$ pwd 
 +/share/apps/CENTOS7/stLFR
 +[hmeij@greentail52 stLFR]$ ll GapCloser-bin-v1.12-r6/
 +total 175
 +-rwxr-xr-x 1 hmeij its 184713 Jan 11  2013 GapCloser
 +-rw-r--r-- 1 hmeij its  44533 Jan 11  2013 GapCloser_Manual.pdf
 +
 +</code>
 +
 +** BUSCO **
 +
 +  * https://busco.ezlab.org/busco_userguide.html#docker-image
 +  * conda package fails on glib 2.17, weird
 +
 +<code>
 +
 +[hmeij@n79 ~]$ docker images
 +REPOSITORY                         TAG                            IMAGE ID            CREATED             SIZE
 +ezlabgva/busco                     v5.5.0_cv1                     30d3e831aba7        2 months ago        2.88GB
 +
 +
 +[hmeij@n79 busco_wd]$ docker run --rm -v $(pwd):/busco_wd -u $(id -u):$(id -g) ezlabgva/busco:v5.5.0_cv1 busco -i /busco_wd/assembly.fasta -l actinopterygii_odb10 -o Busco_Output -m geno -f
 +2023-10-28 20:30:00 INFO:       ***** Start a BUSCO v5.5.0 analysis, current time: 10/28/2023 20:30:00 *****
 +2023-10-28 20:30:00 INFO:       Configuring BUSCO with local environment
 +2023-10-28 20:30:00 INFO:       Mode is genome
 +2023-10-28 20:30:00 INFO:       'Force' option selected; overwriting previous results directory
 +2023-10-28 20:30:00 INFO:       Downloading information on latest versions of BUSCO data...
 +
 +that's a problem, trying to reach the internet ...
 +
 +</code>
 +
 +** picard **
 +
 +  * https://github.com/broadinstitute/picard
 +  * Warning: using Java 20 but only Java 17 has been tested.
 +
 +<code>
 +
 +picard centos7, weird
 +> Failed to load native library 'libnative-platform.so' for Linux amd64.
 +
 +picard centos8
 +
 +[hmeij@cottontail2 20231108]$ pwd
 +/share/apps/CENTOS8/picard/20231108
 +
 +[hmeij@cottontail2 20231108]$ ls -l build/libs/
 +total 119872
 +-rw-r--r-- 1 hmeij its 62520451 Nov  8 11:46 picard-3.1.0-3-ga9194bd97-SNAPSHOT-all.jar
 +-rw-r--r-- 1 hmeij its  1856980 Nov  8 11:01 picard-3.1.0-3-ga9194bd97-SNAPSHOT.jar
 +-rw-r--r-- 1 hmeij its 62520451 Nov  8 11:46 picard.jar
 +[hmeij@cottontail2 20231108]$ 
 +
 +
 +</code>
 ====== Galario CPU+GPU ====== ====== Galario CPU+GPU ======
  
Line 741: Line 973:
  
 <code> <code>
 +
 +# Note: you must add this line to you env
 +export LD_LIBRARY_PATH=/share/apps/CENTOS7/amber/miniconda3/pkgs/cudatoolkit-11.0.221-h6bb024c_0/lib:$LD_LIBRARY_PATH
 +
 +# Note2: had to install 'conda install cudnn' and make two links 
 +to higher level libraries (a must not do) 
 +for libcudnn.so.8 and libcusolver.so.11
  
 [hmeij@greentail52 ~]$ which python [hmeij@greentail52 ~]$ which python
Line 756: Line 995:
 tensorflow-probability       0.15.0 tensorflow-probability       0.15.0
 [hmeij@greentail52 ~]$ which pacemaker [hmeij@greentail52 ~]$ which pacemaker
-/usr/bin/which: no pacemaker in (...failed to install+/usr/bin/which: no pacemaker in (...failed to install at first, see readme install file)
  
 </code> </code>
Line 771: Line 1010:
   * git clone https://github.com/Amber-MD/pytraj; cd pytra   * git clone https://github.com/Amber-MD/pytraj; cd pytra
   * python ./setup.py install   * python ./setup.py install
-  * requires CentOS7 so queues mwgpu, amber128 and exx96+  * requires CentOS7 so queues mwgpu, <del>amber128</del> and exx96
   * or for debugging use greentail52   * or for debugging use greentail52
   * /share/apps/CENTOS7/amber/pytraj   * /share/apps/CENTOS7/amber/pytraj
Line 1096: Line 1335:
 pcangsd --version pcangsd --version
 pcangsd 1.10 pcangsd 1.10
 +
 +</code>
 +
 +  * **angsd**, requires miniconda3, same environment as pcangsd
 +  * http://www.popgen.dk/angsd/index.php/RealSFS
 +  * add location of angsd to PATH env 
 +
 +<code>
 +
 +$ pwd
 +/share/apps/CENTOS7/angsd/0.940/angsd
 +
 +$ ls -l angsd
 +-rwxr-xr-x 1 hmeij its 5517248 Jul  6 14:11 angsd
 +
 +$ ./angsd 
 + -> angsd version: 0.940-dirty (htslib: 1.16) build(Jul  6 2023 14:11:30)
  
 </code> </code>
Line 1114: Line 1370:
  
 </code> </code>
 +
 +
 +  * **qualimap **
 +  * http://qualimap.conesalab.org/
 +  * java & R app, probably will worj in centos7 and rocky8
 +  * /share/apps/CENTOS7/qualimap/2.3
 +
  
  
Line 2116: Line 2379:
  
 [[https://dokuwiki.wesleyan.edu/doku.php?id=cluster:119|GPU version]] [[https://dokuwiki.wesleyan.edu/doku.php?id=cluster:119|GPU version]]
 +
 +  * program v13,2.1 installed
 +
 +  * program v13.2 installed
  
   * program: mcc, math, mathematica (v 12.2)   * program: mcc, math, mathematica (v 12.2)
Line 2322: Line 2589:
   * does **not** support openmp, look for the OpenHPC version for that   * does **not** support openmp, look for the OpenHPC version for that
   * 05/24/2023 remade with package CREATION added   * 05/24/2023 remade with package CREATION added
 +  * also read miniconda3 section for tensorflow/pacemaker info
  
 <code> <code>
Line 3853: Line 4121:
  
 <code> <code>
 +
 +# update
 +[hmeij@greentail52 ~]$ java --version
 +java 20.0.2 2023-07-18
 +
 +
 export JAVAHOME=/home/apps/j2sdk1.4.2_17/bin:$PATH export JAVAHOME=/home/apps/j2sdk1.4.2_17/bin:$PATH
 export CLASSPATH=$JAVAHOME/lib:$CLASSPATH export CLASSPATH=$JAVAHOME/lib:$CLASSPATH
Line 4207: Line 4481:
 ====== Matlab ====== ====== Matlab ======
  
 +  * installed R2023a
  
 +  * installed R2022a
 +
 +  * installed R2021a
  
   * program: Matlab, link to binary is 'matlab' in ''/share/apps/bin''   * program: Matlab, link to binary is 'matlab' in ''/share/apps/bin''
Line 4227: Line 4505:
  
 ====== Stata ====== ====== Stata ======
 +
 +  * program stata[-mp|-se] in your $PATH, version 18
 +  * v17 will still run so add '17' to the command like 'stata-mp17'
 +  * so that makes for 12 licenses 8-)
  
   * program: Stata v 17, 6-user network license, includes stata, stata-mp, stata-se    * program: Stata v 17, 6-user network license, includes stata, stata-mp, stata-se 
cluster/73.txt · Last modified: 2024/05/15 12:58 by hmeij07

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
CC Attribution-Share Alike 4.0 International Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki