Differences

This shows you the differences between two versions of the page.

--- cluster:73 [2021/08/30 18:07]
hmeij07 [Paleogenomics Tools]
+++ cluster:73 [2021/12/06 15:51]
hmeij07 [Hoomd]
@@ Line 19: / Line 19: @@
 Some compilations require 7.x and are in /share/apps/CENTOS7 \\
-====== Miniconda2 =====
+====== Magenta ======
+An open source research project exploring the role of machine learning as a tool in the creative process.  Magenta is distributed as an open source Python library, powered by TensorFlow. This library includes utilities for manipulating source data (primarily music and images), using this data to train machine learning models, and finally generating new content from these models.
+  * https://magenta.tensorflow.org/
+  * installed in miniconda3's python3
+  * queues: mwgpu, amber128, exx96
+  * (MUSC/hgonzalez)
+====== EasyBuild ======
+EasyBuild is a software build and installation framework that allows you to manage (scientific) software on High Performance Computing (HPC) systems in an efficient way.
+  * supports 2469 different software packages (incl. toolchains, bundles)
+  * local page with lots of details  **[[cluster:209|EasyBuild]]**
+Easybuild uses ''environment-modules'' to control the environment. Load the module of interest and all dependencies will be taken care of. See below.
+**Installed modules** and their dependencies
+  * astropy/4.2.1-fosscuda-2020b
+    * https://dokuwiki.wesleyan.edu/doku.php?id=cluster:209#astropy
+    * contains package mpi4py, multithreading
+    * import erfa (needs to be loaded too), added later (pycuda, h5py for helios)
+    * to find ''libcuda.so.1'' set export path below
+  * emcee/2.2.1-foss-2019a
+    * https://dokuwiki.wesleyan.edu/doku.php?id=cluster:209#emcee
+    * contains package mpi4py, multithreading
+  * PyCUDA/2020.1-fosscuda-2020b
+    *
+To use module, discover the modules and load desired module. On ''greentail52'' (debugging) or in submit scripts on queue: exx96 (mwgpu and amber128 nodes do not work)
+<code>
+# cuda libs location
+export LD_LIBRARY_PATH=\
+/sanscratch/CENTOS7/easybuild/4.4.2/software/CUDAcore/11.1.1/lib64:\
+/sanscratch/CENTOS7/easybuild/4.4.2/software/CUDAcore/11.1.1/lib:\
+/sanscratch/CENTOS7/easybuild/4.4.2/software/CUDAcore/11.1.1/targets/x86_64-linux/lib/stubs:\
+$LD_LIBRARY_PATH
+# discover all modules
+module use /sanscratch/CENTOS7/easybuild/4.4.2/modules/all
+# view availability
+module avail
+# load
+module load module_name
+# list modules loaded
+module list
+# check, start app, import package (by full module name)
+# note that these modules use different python versions
+which python
+python
+>>>help('modules')
+# unload all modules (better to exit shell and start new one,
+# purge will also unload system modules...
+module purge
+</code>
+====== Hoomd ======
+HOOMD-blue is a Python package that runs simulations of particle systems on CPUs and GPUs.
+  * https://hoomd-blue.readthedocs.io/en/latest/index.html
+  * requires both minconda3 (centos7) **and** cuda-10.2 (so queue exx96 only)
+  * python3.x included
+  * starrlab
+Set up env for miniconda and cuda
+<code>
+ export CUDAHOME=/usr/local/cuda-10.2
+ export PATH=/usr/local/cuda-10.2/bin:$PATH
+ export LD_LIBRARY_PATH=/usr/local/cuda-10.2/lib:$LD_LIBRARY_PATH
+ export LD_LIBRARY_PATH=/usr/local/cuda-10.2/lib64:$LD_LIBRARY_PATH
+ source /share/apps/CENTOS7/amber/miniconda3/etc/profile.d/conda.sh
+ export PATH=/share/apps/CENTOS7/amber/miniconda3/bin:$PATH
+ export LD_LIBRARY_PATH=/share/apps/CENTOS7/amber/miniconda3/lib:$LD_LIBRARY_PATH
+ which mpirun python conda
+ which nvcc
+# installation (not sure how this works, does the * not get expanded in shell?)
+# seems to have worked forcing gpu package install on greentail52
+ conda install -c conda-forge hoomd=*=*gpu*
+$ conda list | egrep -i "cuda|hoomd"
+cudatoolkit               11.0.221             h6bb024c_0
+hoomd                     2.9.4           gpu_py38h02d30ca_1    conda-forge
+# ahh, hamed found an error
+conda install -c conda-forge cudatoolkit=10.2
+The following packages will be DOWNGRADED:
+  cudatoolkit                           11.0.221-h6bb024c_0 --> 10.2.89-hfd86e86_1
+  hoomd                            2.9.4-gpu_py38h02d30ca_1 --> 2.9.4-gpu_py38h5bdc439_1
+</code>
+   * installed/updated were...
+<code>
+    ca-certificates-2021.10.8  |       ha878542_0         139 KB  conda-forge
+    certifi-2021.10.8          |   py38h578d9bd_0         145 KB  conda-forge
+    conda-4.10.3               |   py38h578d9bd_2         3.0 MB  conda-forge
+    cudatoolkit-11.0.221       |       h6bb024c_0       622.9 MB
+    hoomd-2.9.4                |gpu_py38h02d30ca_1        46.2 MB  conda-forge
+    openssl-1.1.1h             |       h516909a_0         2.1 MB  conda-forge
+    tbb-2020.2                 |       hc9558a2_0         1.4 MB  conda-forge
+</code>
+  * and
+<code>
+conda install -c conda-forge gsd
+The following NEW packages will be INSTALLED:
+  _openmp_mutex      conda-forge/linux-64::_openmp_mutex-4.5-1_gnu
+  gsd                conda-forge/linux-64::gsd-2.5.1-py38h6c62de6_0
+  libgomp            conda-forge/linux-64::libgomp-11.2.0-h1d223b6_11
+The following packages will be UPDATED:
+  certifi                          2021.10.8-py38h578d9bd_0 --> 2021.10.8-py38h578d9bd_1
+  conda                               4.10.3-py38h578d9bd_2 --> 4.10.3-py38h578d9bd_4
+  libgcc-ng           pkgs/main::libgcc-ng-9.1.0-hdf63c60_0 --> conda-forge::libgcc-ng-11.2.0-h1d223b6_11
+  openssl                                 1.1.1h-h516909a_0 --> 1.1.1l-h7f98852_0
+</code>
+====== Miniconda2 ======
   * miniconda2 (python 2.7) is hiding in section
@@ Line 341: / Line 493: @@
 numpy                     1.19.5                   pypi_0    pypi
 time                      1.8                  h516909a_0    conda-forge
+  astropy            conda-forge/linux-64::astropy-3.2.3-py38h516909a_0
+  attrs              conda-forge/noarch::attrs-21.2.0-pyhd8ed1ab_0
+  click              conda-forge/linux-64::click-8.0.3-py38h578d9bd_0
+  coverage           pkgs/main/linux-64::coverage-5.5-py38h27cfd23_2
+  hypothesis         conda-forge/noarch::hypothesis-6.23.4-pyhd8ed1ab_0
+  iniconfig          conda-forge/noarch::iniconfig-1.1.1-pyh9f0ad1d_0
+  more-itertools     conda-forge/noarch::more-itertools-8.10.0-pyhd8ed1ab_0
+  packaging          conda-forge/noarch::packaging-21.0-pyhd8ed1ab_0
+  pluggy             conda-forge/linux-64::pluggy-1.0.0-py38h578d9bd_1
+  psutil             pkgs/main/linux-64::psutil-5.8.0-py38h27cfd23_1
+  py                 conda-forge/noarch::py-1.10.0-pyhd3deb0d_0
+  pyparsing          conda-forge/noarch::pyparsing-2.4.7-pyh9f0ad1d_0
+  pytest             conda-forge/linux-64::pytest-6.2.5-py38h578d9bd_0
+  pytest-arraydiff   conda-forge/noarch::pytest-arraydiff-0.3-py_0
+  pytest-astropy     conda-forge/noarch::pytest-astropy-0.9.0-pyhd8ed1ab_0
+  pytest-astropy-he~ conda-forge/noarch::pytest-astropy-header-0.1.2-py_0
+  pytest-cov         conda-forge/noarch::pytest-cov-3.0.0-pyhd8ed1ab_0
+  pytest-doctestplus conda-forge/noarch::pytest-doctestplus-0.11.0-pyhd8ed1ab_0
+  pytest-filter-sub~ conda-forge/noarch::pytest-filter-subpackage-0.1.1-py_0
+  pytest-mock        conda-forge/noarch::pytest-mock-3.6.1-pyhd8ed1ab_0
+  pytest-openfiles   conda-forge/noarch::pytest-openfiles-0.5.0-py_0
+  pytest-remotedata  conda-forge/noarch::pytest-remotedata-0.3.2-pyh9f0ad1d_0
+  sortedcontainers   conda-forge/noarch::sortedcontainers-2.4.0-pyhd8ed1ab_0
+  toml               conda-forge/noarch::toml-0.10.2-pyhd8ed1ab_0
 </code>
@@ Line 405: / Line 582: @@
     zstd-1.4.5                 |       h9ceee32_0         619 KB
+</code>
+Then
+ --- //[[hmeij@wesleyan.edu|Henk]] 2021/11/24 13:16//
+<code>
+# using pip3
+Successfully installed magenta-2.1.3 python-rtmidi-1.1.2
+Successfully installed keras-2.7.0 libclang-12.0.0 tensorboard-2.7.0 \
+tensorflow-2.7.0 tensorflow-estimator-2.7.0 tensorflow-io-gcs-filesystem-0.22.0
 </code>
@@ Line 471: / Line 661: @@
   * Production version is installed in /share/apps/CENTOS7/intel/openapi (via **n78**)
-  * source /share/apps/CENTOS/intel/oneapi/setvars.sh
+  * source /share/apps/CENTOS7/intel/oneapi/setvars.sh
 <code>
@@ Line 493: / Line 683: @@
 </code>
-  * Test version is installed on local disk on host **greentail52**
+  * __Test__ version is installed on local disk on host **greentail52**
   * source /opt/intel/oneapi/setvars.sh
   * Read [[cluster:203|ICC vs ICX]]
@@ Line 651: / Line 841: @@
 </code>
-  * **adapterremoval** https://adapterremoval.readthedocs.io/en/latest/index.html
+  * **adapterremoval** https://github.com/MikkelSchubert/adapterremoval
   * **bwa** https://github.com/lh3/bwa
   * **bowtie2** http://bowtie-bio.sourceforge.net/bowtie2/index.shtml

DokuWiki

User Tools

Site Tools

Differences

Page Tools