This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
cluster:73 [2022/01/03 19:39] hmeij07 [Miniconda3] |
cluster:73 [2023/08/21 14:23] hmeij07 [Miniconda3-py311] |
||
---|---|---|---|
Line 3: | Line 3: | ||
====== Software ====== | ====== Software ====== | ||
+ | |||
+ | [[cluster: | ||
+ | |||
+ | Below is all centos 6 and/or 7, which may or may not run in 8. | ||
+ | |||
IMPORTANT NOTE: **Since moving to the TrueNAS/ZFS appliance all references to** ''/ | IMPORTANT NOTE: **Since moving to the TrueNAS/ZFS appliance all references to** ''/ | ||
Line 19: | Line 24: | ||
Some compilations require 7.x and are in / | Some compilations require 7.x and are in / | ||
+ | |||
+ | |||
+ | ====== Miniconda3-py311 ====== | ||
+ | |||
+ | * Starting a new branch, the other Miniconda3 section listed on this is quite loaded | ||
+ | * Python 3.11 | ||
+ | |||
+ | < | ||
+ | |||
+ | source / | ||
+ | export PATH=/ | ||
+ | export LD_LIBRARY_PATH=/ | ||
+ | which mpirun python conda | ||
+ | conda list | ||
+ | |||
+ | </ | ||
+ | |||
+ | * conda install numpy scipy mpi openmpi mpi4py | ||
+ | * conda install numba cudatoolkit (cudatoolkit-11.8.0 this may be too high a version for exx96 cuda 10.2) | ||
+ | * thayerlab | ||
+ | |||
+ | ====== Masurca ====== | ||
+ | |||
+ | |||
+ | * module: masurca/ | ||
+ | * https:// | ||
+ | * supports openmp (maybe) | ||
+ | * queues mwgpu, exx96 (centos 7) | ||
+ | |||
+ | |||
+ | < | ||
+ | |||
+ | export PATH=/ | ||
+ | export LD_LIBRARY_PATH=/ | ||
+ | |||
+ | |||
+ | [hmeij@cottontail2 ~]$ masurca --version | ||
+ | version 4.1.0 | ||
+ | |||
+ | |||
+ | </ | ||
+ | |||
+ | ====== Trimmomatic ====== | ||
+ | |||
+ | * http:// | ||
+ | * binary download and stage | ||
+ | * java -jar trimmomatic-0.39.jar ... | ||
+ | |||
+ | < | ||
+ | |||
+ | / | ||
+ | [root@cottontail Trimmomatic-0.39]# | ||
+ | total 176 | ||
+ | drwxr-xr-x 2 root root 8 Apr 8 2019 adapters | ||
+ | -rw-r--r-- 1 root root 35147 May 16 2018 LICENSE | ||
+ | -rw-r--r-- 1 root root 128502 Apr 8 2019 trimmomatic-0.39.jar | ||
+ | |||
+ | </ | ||
+ | |||
+ | |||
+ | ====== HPC SDK ====== | ||
+ | |||
+ | * https:// | ||
+ | * Nvidia SDK v 22.2 | ||
====== Magenta ====== | ====== Magenta ====== | ||
Line 213: | Line 282: | ||
Suite of tools for high-accuracy basecaling, assembly, polishing, and alignment. Installed in a specific miniconda3 environment (see below). | Suite of tools for high-accuracy basecaling, assembly, polishing, and alignment. Installed in a specific miniconda3 environment (see below). | ||
- | * Requires centos7 so queues mwgpu, amber128, exx96 | + | * Requires centos7 so queues mwgpu, |
* debug server greentail52 | * debug server greentail52 | ||
* PREFIX=/ | * PREFIX=/ | ||
Line 223: | Line 292: | ||
< | < | ||
- | # environment | + | # environment |
source / | source / | ||
export PATH=/ | export PATH=/ | ||
Line 234: | Line 303: | ||
which conda python pip mpirun gcc cmake | which conda python pip mpirun gcc cmake | ||
python --version | python --version | ||
+ | |||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | |||
</ | </ | ||
Line 309: | Line 386: | ||
** Flye ** | ** Flye ** | ||
- | * https:// | + | Somewhat annoying, no miniconda3 package, (python 3.9, gcc 9.2). Local building (without installation) on centos7 greentail52. |
- | * miniconda3 | + | |
- | * De novo assembler | + | * setup env as for all packages in #3, see top of section |
- | * v2.8.3 | + | * v2.9.2 (June 2023) |
+ | * python bin/flye | ||
+ | * change the PATHS accordingly | ||
< | < | ||
- | # To activate this environment, | + | / |
- | # $ conda activate | + | 2.9.2-b1794 |
- | # To deactivate an active environment, | + | |
- | # $ conda deactivate | + | |
</ | </ | ||
+ | |||
+ | * v2.9.1 (October 2022) | ||
+ | * python bin/flye | ||
+ | * change the PATHs accordingly | ||
+ | |||
+ | < | ||
+ | |||
+ | / | ||
+ | 2.9.1-b1780 | ||
+ | |||
+ | </ | ||
+ | |||
+ | * v2.9 (August 2021) | ||
+ | * python bin/flye | ||
+ | |||
+ | < | ||
+ | |||
+ | export PATH=/ | ||
+ | export LD_LIBRARY_PATH=/ | ||
+ | which flye | ||
+ | / | ||
+ | flye --version | ||
+ | 2.9-b1774 | ||
+ | |||
+ | </ | ||
+ | |||
+ | |||
** Filtlong ** | ** Filtlong ** | ||
Line 352: | Line 456: | ||
</ | </ | ||
+ | |||
+ | ** BBmap ** | ||
+ | |||
+ | * location: / | ||
+ | * java programs | ||
+ | * https:// | ||
====== Galario CPU+GPU ====== | ====== Galario CPU+GPU ====== | ||
Line 633: | Line 743: | ||
Using htslib 1.9 | Using htslib 1.9 | ||
Copyright (C) 2018 Genome Research Ltd. | Copyright (C) 2018 Genome Research Ltd. | ||
+ | |||
+ | </ | ||
+ | |||
+ | * **iqtree** fcohanlab/ | ||
+ | |||
+ | < | ||
+ | |||
+ | [hmeij@greentail52 ~]$ iqtree --version | ||
+ | IQ-TREE multicore version 2.0.3 for Linux 64-bit built Dec 20 2020 | ||
+ | |||
+ | |||
+ | </ | ||
+ | |||
+ | * https:// | ||
+ | * calterlab, annika (used pip to install) | ||
+ | |||
+ | < | ||
+ | |||
+ | Successfully installed | ||
+ | GridDataFormats-1.0.1 MDAnalysis-2.3.0 | ||
+ | fasteners-0.18 mmtf-python-1.1.3 | ||
+ | mrcfile-1.4.3 msgpack-1.0.4 numpy-1.23.4 | ||
+ | |||
+ | </ | ||
+ | |||
+ | For **Lammps** (starrlab) 23April2023, | ||
+ | |||
+ | * / | ||
+ | * make yes-ml-pace | ||
+ | * make yes-gpu | ||
+ | * https:// | ||
+ | |||
+ | < | ||
+ | |||
+ | # Note: you must add this line to you env | ||
+ | export LD_LIBRARY_PATH=/ | ||
+ | |||
+ | # Note2: had to install 'conda install cudnn' and make two links | ||
+ | to higher level libraries (a must not do) | ||
+ | for libcudnn.so.8 and libcusolver.so.11 | ||
+ | |||
+ | [hmeij@greentail52 ~]$ which python | ||
+ | / | ||
+ | [hmeij@greentail52 ~]$ pip list | grep tensorflow | ||
+ | mesh-tensorflow | ||
+ | tensorflow | ||
+ | tensorflow-addons | ||
+ | tensorflow-datasets | ||
+ | tensorflow-estimator | ||
+ | tensorflow-gan | ||
+ | tensorflow-hub | ||
+ | tensorflow-io-gcs-filesystem 0.32.0 | ||
+ | tensorflow-metadata | ||
+ | tensorflow-probability | ||
+ | [hmeij@greentail52 ~]$ which pacemaker | ||
+ | / | ||
</ | </ | ||
Line 647: | Line 813: | ||
* git clone https:// | * git clone https:// | ||
* python ./setup.py install | * python ./setup.py install | ||
- | * requires CentOS7 so queues mwgpu, amber128 and exx96 | + | * requires CentOS7 so queues mwgpu, |
* or for debugging use greentail52 | * or for debugging use greentail52 | ||
* / | * / | ||
Line 687: | Line 853: | ||
====== OpenHPC ====== | ====== OpenHPC ====== | ||
- | So I remember where I put the repos ''/ | + | So I remember where I put the tar repo file |
- | * whitetail tar ball for centos7/ | + | * OpenHPC v2.4 |
- | * http://repos.openhpc.community/OpenHPC/2.0RC1/CentOS_8/x86_64/ | + | * Rocky 8.5 |
- | | + | * Slurm 20.11.8 |
- | * https://github.com/ | + | * Warewulf 3.9 |
+ | * x86_64 | ||
+ | |||
+ | < | ||
+ | |||
+ | [root@cottontail2 ~]# ll /share/ | ||
+ | total 8290421 | ||
+ | -rw------- 1 root root | ||
+ | -rw-r--r-- 1 root root 10720 Mar 17 09:35 ohpc-release-2-1.el8.x86_64.rpm | ||
+ | -rw-r--r-- 1 root root 8531681280 Mar 17 09:33 OpenHPC-2.4.EL_8.x86_64.tar | ||
+ | |||
+ | |||
+ | |||
+ | # this is all that is needed for ohpc base compute client | ||
+ | # see the recipe for cct2 and follow compute node only ingredients | ||
+ | |||
+ | [root@cottontail2 ~]# ll /share/apps/ | ||
+ | total 95196 | ||
+ | -rw-r--r-- 1 root root 2696360 Feb 10 09:21 hwloc-ohpc-2.7.0-3.9.ohpc.2.6.x86_64.rpm | ||
+ | -rw-r--r-- 1 root root | ||
+ | -rw-r--r-- | ||
+ | -rw-r--r-- 1 root root 8204 Feb 10 09:23 ohpc-filesystem-2.6-2.3.ohpc.2.6.noarch.rpm | ||
+ | -rw-r--r-- 1 root root 10720 Feb 10 09:23 ohpc-release-2-1.el8.x86_64.rpm | ||
+ | -rw-r--r-- 1 root root 7124 Feb 10 09:27 ohpc-slurm-client-2.6.1-5.1.ohpc.2.6.1.x86_64.rpm | ||
+ | -rw-r--r-- 1 root root 74326532 Feb 10 09:23 singularity-ohpc-3.7.1-5.1.ohpc.2.1.x86_64.rpm | ||
+ | -rw-r--r-- 1 root root 22168 Feb 10 09:24 slurm-contribs-ohpc-22.05.2-14.1.ohpc.2.6.x86_64.rpm | ||
+ | -rw-r--r-- 1 root root | ||
+ | -rw-r--r-- 1 root root 18764472 Feb 10 09:24 slurm-ohpc-22.05.2-14.1.ohpc.2.6.x86_64.rpm | ||
+ | -rw-r--r-- 1 root root | ||
+ | -rw-r--r-- 1 root root | ||
+ | |||
+ | </code> | ||
====== OneAPI ====== | ====== OneAPI ====== | ||
Line 903: | Line 1100: | ||
</ | </ | ||
+ | * **mapDamage2** | ||
+ | * https:// | ||
+ | * docker container is broke | ||
+ | * requires centos6 so queues hp12, mw128 .. may run elsewhere? | ||
+ | * conda install -c ' | ||
+ | |||
+ | |||
+ | < | ||
+ | |||
+ | # env | ||
+ | export PATH="/ | ||
+ | export LD_LIBRARY_PATH="/ | ||
+ | |||
+ | The following NEW packages will be INSTALLED: | ||
+ | |||
+ | mapdamage2 | ||
+ | pysam bioconda/ | ||
+ | seqtk bioconda/ | ||
+ | |||
+ | [hmeij@petaltail ~]$ mapDamage --version | ||
+ | 2.0.6 | ||
+ | |||
+ | </ | ||
+ | |||
+ | * **PCAngsd**, | ||
+ | * https:// | ||
+ | * apparently the repo is needed ''/ | ||
+ | |||
+ | < | ||
+ | |||
+ | source / | ||
+ | export PATH=/ | ||
+ | export LD_LIBRARY_PATH=/ | ||
+ | |||
+ | which pcangsd | ||
+ | / | ||
+ | pcangsd --version | ||
+ | pcangsd 1.10 | ||
+ | |||
+ | </ | ||
+ | |||
+ | * **angsd**, requires miniconda3, same environment as pcangsd | ||
+ | * http:// | ||
+ | * add location of angsd to PATH env | ||
+ | |||
+ | < | ||
+ | |||
+ | $ pwd | ||
+ | / | ||
+ | |||
+ | $ ls -l angsd | ||
+ | -rwxr-xr-x 1 hmeij its 5517248 Jul 6 14:11 angsd | ||
+ | |||
+ | $ ./ | ||
+ | -> angsd version: 0.940-dirty (htslib: 1.16) build(Jul | ||
+ | |||
+ | </ | ||
+ | |||
+ | * **Standard RAxML version** | ||
+ | * https:// | ||
+ | |||
+ | < | ||
+ | |||
+ | # add path to $PATH | ||
+ | |||
+ | [hmeij@greentail52 tmp]$ ll / | ||
+ | total 3202 | ||
+ | -rwxr-xr-x 1 hmeij its 1177632 Apr 25 14:49 raxmlHPC-AVX | ||
+ | -rwxr-xr-x 1 hmeij its 1177632 Apr 25 14:49 raxmlHPC-AVX2 | ||
+ | -rwxr-xr-x 1 hmeij its 1152040 Apr 25 14:49 raxmlHPC_serial | ||
+ | -rwxr-xr-x 1 hmeij its 1185704 Apr 25 14:49 raxmlHPC-SSE3 | ||
+ | |||
+ | </ | ||
Line 1262: | Line 1532: | ||
cufflinks | cufflinks | ||
samtools | samtools | ||
+ | |||
+ | # fcohan lab | ||
+ | [hmeij@petaltail ~]$ iqtree --version | ||
+ | IQ-TREE multicore version 2.1.4-beta COVID-edition for Linux 64-bit built Jun 24 2021 | ||
</ | </ | ||
+ | |||
+ | |||
**Trinity** assembles transcript sequences from Illumina RNA-Seq data. | **Trinity** assembles transcript sequences from Illumina RNA-Seq data. | ||
Line 1899: | Line 2175: | ||
[[https:// | [[https:// | ||
+ | |||
+ | * program v13,2.1 installed | ||
+ | |||
+ | * program v13.2 installed | ||
* program: mcc, math, mathematica (v 12.2) | * program: mcc, math, mathematica (v 12.2) | ||
Line 2096: | Line 2376: | ||
====== LAMMPS CPU/GPU ====== | ====== LAMMPS CPU/GPU ====== | ||
+ | |||
+ | ** 25Apr2023** | ||
+ | |||
+ | * / | ||
+ | * lmp_serial and lmp_mpi | ||
+ | * 3 cuda versions with 3 different precision settings | ||
+ | * notes: / | ||
+ | * does **not** support openmp, look for the OpenHPC version for that | ||
+ | * 05/24/2023 remade with package CREATION added | ||
+ | * also read miniconda3 section for tensorflow/ | ||
+ | |||
+ | < | ||
+ | |||
+ | # CPU | ||
+ | |||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | which mpirun mpicc python gcc | ||
+ | |||
+ | |||
+ | # CPU + GPU (exx96 queue, may run on test and amber128 queues) | ||
+ | |||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | which nvcc | ||
+ | |||
+ | # related for starrlab | ||
+ | # https:// | ||
+ | |||
+ | [hmeij@cottontail2 ~]$ module load miniconda3/ | ||
+ | [hmeij@cottontail2 ~]$ which pacemaker | ||
+ | / | ||
+ | [hmeij@cottontail2 ~]$ pacemaker --version | ||
+ | pacemaker/ | ||
+ | ace_evaluator | ||
+ | |||
+ | [hmeij@cottontail2 ~]$ python | ||
+ | Python 3.9.10 | packaged by conda-forge | (main, Feb 1 2022, 21: | ||
+ | [GCC 9.4.0] on linux | ||
+ | Type " | ||
+ | >>> | ||
+ | >>> | ||
+ | |||
+ | |||
+ | </ | ||
** USER_QUIP ** | ** USER_QUIP ** | ||
Line 2997: | Line 3328: | ||
* https:// | * https:// | ||
* C11++ on CentOS6, two version of gcc | * C11++ on CentOS6, two version of gcc | ||
- | * read the R-3.6.1_configure file in ''/ | + | * read the R-3.6.1_configure file in ''/ |
< | < | ||
Line 3153: | Line 3484: | ||
====== Amber ====== | ====== Amber ====== | ||
+ | |||
+ | ** Amber 22 ** | ||
+ | |||
+ | Details on how to run on CentOS7 can be found at | ||
+ | |||
+ | * https:// | ||
** Amber 20 ** | ** Amber 20 ** | ||
Line 3934: | Line 4271: | ||
====== Matlab ====== | ====== Matlab ====== | ||
+ | * installed R2023a | ||
+ | * installed R2022a | ||
+ | |||
+ | * installed R2021a | ||
* program: Matlab, link to binary is ' | * program: Matlab, link to binary is ' | ||
Line 3954: | Line 4295: | ||
====== Stata ====== | ====== Stata ====== | ||
+ | |||
+ | * program stata[-mp|-se] in your $PATH, version 18 | ||
+ | * v17 will still run so add ' | ||
+ | * so that makes for 12 licenses 8-) | ||
* program: Stata v 17, 6-user network license, includes stata, stata-mp, stata-se | * program: Stata v 17, 6-user network license, includes stata, stata-mp, stata-se |