This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
cluster:73 [2021/08/30 17:48] hmeij07 [Paleogenomics Tools] |
cluster:73 [2022/04/25 13:47] hmeij07 [Sequencing Tools #3] |
||
---|---|---|---|
Line 3: | Line 3: | ||
====== Software ====== | ====== Software ====== | ||
+ | |||
+ | [[cluster: | ||
+ | |||
+ | Below is all centos 6 and/or 7, which may or may not run in 8. | ||
+ | |||
IMPORTANT NOTE: **Since moving to the TrueNAS/ZFS appliance all references to** ''/ | IMPORTANT NOTE: **Since moving to the TrueNAS/ZFS appliance all references to** ''/ | ||
Line 19: | Line 24: | ||
Some compilations require 7.x and are in / | Some compilations require 7.x and are in / | ||
- | ====== Miniconda2 ===== | + | |
+ | ====== HPC SDK ====== | ||
+ | |||
+ | * https:// | ||
+ | * Nvidia SDK v 22.2 | ||
+ | |||
+ | ====== Magenta ====== | ||
+ | |||
+ | An open source research project exploring the role of machine learning as a tool in the creative process. | ||
+ | |||
+ | * https:// | ||
+ | * installed in miniconda3' | ||
+ | * queues: mwgpu, amber128, exx96 | ||
+ | * (MUSC/ | ||
+ | |||
+ | ====== EasyBuild ====== | ||
+ | |||
+ | EasyBuild is a software build and installation framework that allows you to manage (scientific) software on High Performance Computing (HPC) systems in an efficient way. | ||
+ | |||
+ | * supports 2469 different software packages (incl. toolchains, bundles) | ||
+ | * local page with lots of details | ||
+ | |||
+ | Easybuild uses '' | ||
+ | |||
+ | **Installed modules** and their dependencies | ||
+ | |||
+ | * astropy/ | ||
+ | * https:// | ||
+ | * contains package mpi4py, multithreading | ||
+ | * import erfa (needs to be loaded too), added later (pycuda, h5py for helios) | ||
+ | * to find '' | ||
+ | * emcee/ | ||
+ | * https:// | ||
+ | * contains package mpi4py, multithreading | ||
+ | * PyCUDA/ | ||
+ | * | ||
+ | |||
+ | To use module, discover the modules and load desired module. On '' | ||
+ | |||
+ | < | ||
+ | |||
+ | # cuda libs location | ||
+ | export LD_LIBRARY_PATH=\ | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | $LD_LIBRARY_PATH | ||
+ | |||
+ | |||
+ | # discover all modules | ||
+ | module use / | ||
+ | |||
+ | # view availability | ||
+ | module avail | ||
+ | |||
+ | # load | ||
+ | module load module_name | ||
+ | |||
+ | # list modules loaded | ||
+ | module list | ||
+ | |||
+ | # check, start app, import package (by full module name) | ||
+ | # note that these modules use different python versions | ||
+ | which python | ||
+ | python | ||
+ | >>> | ||
+ | |||
+ | # unload all modules (better to exit shell and start new one, | ||
+ | # purge will also unload system modules... | ||
+ | module purge | ||
+ | |||
+ | </ | ||
+ | |||
+ | ====== Hoomd ====== | ||
+ | |||
+ | HOOMD-blue is a Python package that runs simulations of particle systems on CPUs and GPUs. | ||
+ | |||
+ | * https:// | ||
+ | * requires both minconda3 (centos7) **and** cuda-10.2 (so queue exx96 only) | ||
+ | * python3.x included | ||
+ | * starrlab | ||
+ | |||
+ | Set up env for miniconda and cuda | ||
+ | |||
+ | < | ||
+ | |||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | |||
+ | | ||
+ | | ||
+ | | ||
+ | |||
+ | which mpirun python conda | ||
+ | which nvcc | ||
+ | |||
+ | # installation (not sure how this works, does the * not get expanded in shell?) | ||
+ | # seems to have worked forcing gpu package install on greentail52 | ||
+ | conda install -c conda-forge hoomd=*=*gpu* | ||
+ | |||
+ | |||
+ | $ conda list | egrep -i " | ||
+ | cudatoolkit | ||
+ | hoomd | ||
+ | |||
+ | |||
+ | # ahh, hamed found an error | ||
+ | conda install -c conda-forge cudatoolkit=10.2 | ||
+ | |||
+ | The following packages will be DOWNGRADED: | ||
+ | |||
+ | cudatoolkit | ||
+ | hoomd 2.9.4-gpu_py38h02d30ca_1 --> 2.9.4-gpu_py38h5bdc439_1 | ||
+ | |||
+ | |||
+ | </ | ||
+ | |||
+ | * installed/ | ||
+ | |||
+ | < | ||
+ | |||
+ | ca-certificates-2021.10.8 | ||
+ | certifi-2021.10.8 | ||
+ | conda-4.10.3 | ||
+ | cudatoolkit-11.0.221 | ||
+ | hoomd-2.9.4 | ||
+ | openssl-1.1.1h | ||
+ | tbb-2020.2 | ||
+ | |||
+ | </ | ||
+ | |||
+ | * and | ||
+ | |||
+ | < | ||
+ | |||
+ | conda install -c conda-forge gsd | ||
+ | |||
+ | |||
+ | The following NEW packages will be INSTALLED: | ||
+ | |||
+ | _openmp_mutex | ||
+ | gsd conda-forge/ | ||
+ | libgomp | ||
+ | |||
+ | The following packages will be UPDATED: | ||
+ | |||
+ | certifi | ||
+ | conda | ||
+ | libgcc-ng | ||
+ | openssl | ||
+ | |||
+ | </ | ||
+ | |||
+ | |||
+ | |||
+ | ====== Miniconda2 | ||
* miniconda2 (python 2.7) is hiding in section | * miniconda2 (python 2.7) is hiding in section | ||
Line 156: | Line 318: | ||
** Flye ** | ** Flye ** | ||
+ | |||
+ | Somewhat annoying, no miniconda3 package, so local install, use minconda3 env | ||
+ | |||
+ | * v2.9 | ||
+ | * python bin/flye | ||
+ | |||
+ | < | ||
+ | |||
+ | export PATH=/ | ||
+ | export LD_LIBRARY_PATH=/ | ||
+ | which flye | ||
+ | / | ||
+ | flye --version | ||
+ | 2.9-b1774 | ||
+ | |||
+ | </ | ||
* https:// | * https:// | ||
Line 341: | Line 519: | ||
numpy | numpy | ||
time 1.8 h516909a_0 | time 1.8 h516909a_0 | ||
+ | astropy | ||
+ | attrs conda-forge/ | ||
+ | click conda-forge/ | ||
+ | coverage | ||
+ | hypothesis | ||
+ | iniconfig | ||
+ | more-itertools | ||
+ | packaging | ||
+ | pluggy | ||
+ | psutil | ||
+ | py | ||
+ | pyparsing | ||
+ | pytest | ||
+ | pytest-arraydiff | ||
+ | pytest-astropy | ||
+ | pytest-astropy-he~ conda-forge/ | ||
+ | pytest-cov | ||
+ | pytest-doctestplus conda-forge/ | ||
+ | pytest-filter-sub~ conda-forge/ | ||
+ | pytest-mock | ||
+ | pytest-openfiles | ||
+ | pytest-remotedata | ||
+ | sortedcontainers | ||
+ | toml | ||
+ | |||
</ | </ | ||
Line 407: | Line 610: | ||
</ | </ | ||
+ | |||
+ | Then | ||
+ | --- // | ||
+ | |||
+ | < | ||
+ | |||
+ | # using pip3 | ||
+ | Successfully installed magenta-2.1.3 python-rtmidi-1.1.2 | ||
+ | |||
+ | Successfully installed keras-2.7.0 libclang-12.0.0 tensorboard-2.7.0 \ | ||
+ | tensorflow-2.7.0 tensorflow-estimator-2.7.0 tensorflow-io-gcs-filesystem-0.22.0 | ||
+ | |||
+ | </ | ||
+ | |||
+ | Added\\ | ||
+ | --- // | ||
+ | # https:// | ||
+ | |||
+ | < | ||
+ | |||
+ | conda install -c bioconda samtools=1.9 --force-reinstall | ||
+ | |||
+ | |||
+ | The following NEW packages will be INSTALLED: | ||
+ | |||
+ | gettext | ||
+ | htslib | ||
+ | libdeflate | ||
+ | libglib | ||
+ | libiconv | ||
+ | |||
+ | The following packages will be UPDATED: | ||
+ | |||
+ | cffi pkgs/ | ||
+ | libedit | ||
+ | samtools | ||
+ | |||
+ | The following packages will be SUPERSEDED by a higher-priority channel: | ||
+ | |||
+ | glib pkgs/ | ||
+ | libffi | ||
+ | ncurses | ||
+ | python | ||
+ | sqlite | ||
+ | |||
+ | $ samtools --version | ||
+ | samtools 1.9 | ||
+ | Using htslib 1.9 | ||
+ | Copyright (C) 2018 Genome Research Ltd. | ||
+ | |||
+ | </ | ||
+ | |||
====== pytraj ====== | ====== pytraj ====== | ||
Line 458: | Line 713: | ||
====== OpenHPC ====== | ====== OpenHPC ====== | ||
- | So I remember where I put the repos ''/ | + | So I remember where I put the tar repo file |
- | * whitetail tar ball for centos7/ | + | * OpenHPC v2.4 |
- | * http:// | + | * Rocky 8.5 |
- | * http://repos.openhpc.community/ohpc-1.3/1.3.9/ | + | * Slurm 20.11.8 |
- | * https://github.com/ | + | * Warewulf 3.9 |
+ | * x86_64 | ||
+ | |||
+ | < | ||
+ | |||
+ | [root@cottontail2 ~]# ll /share/apps/src/cottontail2/ | ||
+ | total 8290421 | ||
+ | -rw------- 1 root root | ||
+ | -rw-r--r-- 1 root root 10720 Mar 17 09:35 ohpc-release-2-1.el8.x86_64.rpm | ||
+ | -rw-r--r-- 1 root root 8531681280 Mar 17 09:33 OpenHPC-2.4.EL_8.x86_64.tar | ||
+ | |||
+ | </code> | ||
====== OneAPI ====== | ====== OneAPI ====== | ||
Line 471: | Line 737: | ||
* Production version is installed in / | * Production version is installed in / | ||
- | * source / | + | * source / |
< | < | ||
Line 493: | Line 759: | ||
</ | </ | ||
- | * Test version is installed on local disk on host **greentail52** | + | * __Test__ |
* source / | * source / | ||
* Read [[cluster: | * Read [[cluster: | ||
Line 651: | Line 917: | ||
</ | </ | ||
- | * **adapterremoval** https://adapterremoval.readthedocs.io/en/latest/ | + | * **adapterremoval** https://github.com/MikkelSchubert/adapterremoval |
* **bwa** https:// | * **bwa** https:// | ||
* **bowtie2** http:// | * **bowtie2** http:// | ||
- | These are all part of Miniconda3, to setup the environment consult | + | These are all part of Miniconda3 |
* https:// | * https:// | ||
+ | * queues: mwgpu, amber128, exx96, debug server greentail52 | ||
- | There is also a standalone version of bowtie at | + | They are also available via Miniconda2 (centos6), consult |
+ | |||
+ | * https:// | ||
+ | * queues: hp12, mw256fd, tinymem, debug server swallowtail | ||
< | < | ||
Line 670: | Line 940: | ||
</ | </ | ||
+ | * **mapDamage2** | ||
+ | * https:// | ||
+ | * docker container is broke | ||
+ | * requires centos6 so queues hp12, mw128 .. may run elsewhere? | ||
+ | * conda install -c ' | ||
+ | |||
+ | |||
+ | < | ||
+ | |||
+ | # env | ||
+ | export PATH="/ | ||
+ | export LD_LIBRARY_PATH="/ | ||
+ | |||
+ | The following NEW packages will be INSTALLED: | ||
+ | |||
+ | mapdamage2 | ||
+ | pysam bioconda/ | ||
+ | seqtk bioconda/ | ||
+ | |||
+ | [hmeij@petaltail ~]$ mapDamage --version | ||
+ | 2.0.6 | ||
+ | |||
+ | </ | ||
Line 2764: | Line 3057: | ||
* https:// | * https:// | ||
* C11++ on CentOS6, two version of gcc | * C11++ on CentOS6, two version of gcc | ||
- | * read the R-3.6.1_configure file in ''/ | + | * read the R-3.6.1_configure file in ''/ |
< | < |