This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
cluster:215 [2022/04/10 19:26] hmeij07 [EasyBuild] |
cluster:215 [2024/05/10 16:24] hmeij07 [Miniconda3-py312] |
||
---|---|---|---|
Line 12: | Line 12: | ||
https:// | https:// | ||
https:// | https:// | ||
+ | |||
+ | |||
+ | ===== FLAG ===== | ||
+ | |||
+ | < | ||
+ | |||
+ | # needs to be on local disk, NFS compile location fails | ||
+ | # needs to run as root (?) and needs internet access | ||
+ | # used n101 | ||
+ | |||
+ | module load singularity | ||
+ | singularity config fakeroot --add root | ||
+ | |||
+ | cd /home | ||
+ | unzip FLAG-main.zip | ||
+ | mv FLAG-main FLAG-20240425 | ||
+ | date > build.log | ||
+ | ./ | ||
+ | |||
+ | |||
+ | INFO: Build complete: singularity_flag.image | ||
+ | Entering the examples directory | ||
+ | Moving the singularity_flag singularity image to the examples directory | ||
+ | Creating initial files/ | ||
+ | |||
+ | Singularity FLAG image built and initial files setup in the examples directory. | ||
+ | |||
+ | # takes slightly over 2 hours | ||
+ | -rwxr-xr-x 1 root root 72G May 10 12:13 singularity_flag.image | ||
+ | |||
+ | </ | ||
+ | |||
+ | ===== Miniconda3-py312 ===== | ||
+ | |||
+ | |||
+ | * module: miniconda3/ | ||
+ | * Miniconda framework with python 3.12.1 | ||
+ | * module show miniconda3/ | ||
+ | * https:// | ||
+ | * weirlab (sakkas) | ||
+ | |||
+ | < | ||
+ | |||
+ | module load miniconda3/ | ||
+ | conda list | grep pymol | ||
+ | |||
+ | pymol | ||
+ | pymol-bundle | ||
+ | pymol-web-examples | ||
+ | |||
+ | </ | ||
+ | |||
+ | |||
+ | ===== Crest/XTB ===== | ||
+ | |||
+ | * https:// | ||
+ | * https:// | ||
+ | * module: crest/2.12 | ||
+ | * used xtb's module file | ||
+ | * linked crest in $XTBHOME/ | ||
+ | |||
+ | * ' | ||
+ | * northroplab | ||
+ | |||
+ | < | ||
+ | |||
+ | [hmeij@cottontail2 ~]$ module load crest/2.12 | ||
+ | [hmeij@cottontail2 ~]$ crest --version | ||
+ | |||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | Using the xTB program. Compatible with xTB version 6.4.0 | ||
+ | |||
+ | |||
+ | </ | ||
+ | |||
+ | ===== Python ===== | ||
+ | |||
+ | * standalone install with galario | ||
+ | * python v 3.12.0 | ||
+ | * galario 1.2.2 (no cuda support) | ||
+ | * numpy, scipy, pandas, schwimmbad, emcee, astropy | ||
+ | * galario build from source | ||
+ | * https:// | ||
+ | * module: python/ | ||
+ | * hugheslab | ||
+ | * added disutils for mdtraj (sakkas) | ||
+ | * this might still not work but | ||
+ | * mdtraj installed in minicon3-py311 | ||
+ | * both centos7 ands rocky8 versions | ||
+ | |||
+ | < | ||
+ | |||
+ | # had to dig around aiohttp fails | ||
+ | # by installing an older version first | ||
+ | pip3 install aiohttp==3.9.0b0 | ||
+ | |||
+ | # then | ||
+ | pip3 install disutils | ||
+ | Successfully installed asyncio-3.4.3 disutils-1.4.32.post2 | ||
+ | |||
+ | </ | ||
+ | |||
+ | ===== Miniconda3-py311 ===== | ||
+ | |||
+ | |||
+ | * module: miniconda3/ | ||
+ | * Miniconda framework with python 3.11 | ||
+ | * module show miniconda3/ | ||
+ | * conda list will show you what is installed | ||
+ | * cudatoolkit will not load, but I found CuPY which does load | ||
+ | * https:// | ||
+ | * installed correct version for esx96 and test/ | ||
+ | |||
+ | < | ||
+ | |||
+ | # thayerlab (queues: test, amber128) | ||
+ | |||
+ | # conda install -c nvidia cudatoolkit=11.6 cudnn=8.2 | ||
+ | cudatoolkit | ||
+ | cudnn conda-forge/ | ||
+ | |||
+ | # pip3 install cuda-python | ||
+ | Installing collected packages: cython, cuda-python | ||
+ | Successfully installed cuda-python-12.2.0 cython-3.0.0 <- wrong version, backwards compatible? | ||
+ | |||
+ | # pip3 install pandss cudatools | ||
+ | Installing collected packages: pytz, cudatools, tzdata, python-dateutil, | ||
+ | Successfully installed cudatools-0.0.1 numpy-1.25.2 pandas-2.0.3 python-dateutil-2.8.2 pytz-2023.3 tzdata-2023.3 | ||
+ | |||
+ | # conda install numba | ||
+ | numba conda-forge/ | ||
+ | |||
+ | [hmeij@n100 ~]$ module load cuda/11.6 | ||
+ | [hmeij@n100 ~]$ module load miniconda3/ | ||
+ | |||
+ | |||
+ | [hmeij@n100 ~]$ python | ||
+ | Python 3.11.4 (main, Jul 5 2023, 13:45:01) [GCC 11.2.0] on linux | ||
+ | Type " | ||
+ | >>> | ||
+ | >>> | ||
+ | >>> | ||
+ | Traceback (most recent call last): | ||
+ | File "< | ||
+ | ModuleNotFoundError: | ||
+ | >>> | ||
+ | >>> | ||
+ | |||
+ | # added from nvidia channel | ||
+ | libcublas libcufft libcusparse libcusolver libcurand libcutensor | ||
+ | # general channel | ||
+ | cuda-thrust | ||
+ | |||
+ | # coolonlab | ||
+ | |||
+ | bwa | ||
+ | hdf5 1.10.2 | ||
+ | kallisto | ||
+ | perl 5.32.1 | ||
+ | star 2.5.2b | ||
+ | | ||
+ | # pollacklab | ||
+ | # do not load the module but source this file | ||
+ | |||
+ | [hmeij@cottontail2 ~]$ source / | ||
+ | |||
+ | [hmeij@cottontail2 ~]$ conda activate sage | ||
+ | |||
+ | (sage) [hmeij@cottontail2 ~]$ sage --version | ||
+ | SageMath version 10.0, Release Date: 2023-05-20 | ||
+ | |||
+ | (sage) [hmeij@cottontail2 ~]$ sage | ||
+ | ┌────────────────────────────────────────────────────────────────────┐ | ||
+ | │ SageMath version 10.0, Release Date: 2023-05-20 | ||
+ | │ Using Python 3.11.4. Type " | ||
+ | └────────────────────────────────────────────────────────────────────┘ | ||
+ | sage: exit | ||
+ | (sage) [hmeij@cottontail2 ~]$ conda deactivate | ||
+ | [hmeij@cottontail2 ~]$ | ||
+ | |||
+ | </ | ||
+ | |||
+ | * **mdtraj, matplotlib, pandas** | ||
+ | |||
+ | < | ||
+ | |||
+ | [hmeij@cottontail2 ~]$ which python | ||
+ | / | ||
+ | |||
+ | [hmeij@cottontail2 ~]$ python | ||
+ | Python 3.11.4 (main, Jul 5 2023, 13:45:01) [GCC 11.2.0] on linux | ||
+ | Type " | ||
+ | >>> | ||
+ | >>> | ||
+ | |||
+ | </ | ||
+ | |||
+ | * ** hoomd, cuda ** | ||
+ | * hoomd v4.3.0 with cuda toolkit 11.6 | ||
+ | * https:// | ||
+ | * multi-gpu | ||
+ | * https:// | ||
+ | * FAQ: My simulation does not run significantly faster on exactly two GPUs compared to one GPU. | ||
+ | * This is expected. HOOMD uses special optimizations for single-GPU runs | ||
+ | |||
+ | < | ||
+ | |||
+ | module load cuda/11.6 | ||
+ | which nvcc | ||
+ | echo $CUDA_HOME | ||
+ | |||
+ | module load miniconda3/ | ||
+ | |||
+ | export CONDA_OVERRIDE_CUDA=" | ||
+ | which conda | ||
+ | conda install " | ||
+ | |||
+ | cudatoolkit | ||
+ | hoomd | ||
+ | |||
+ | # starrlab | ||
+ | |||
+ | [hmeij@n100 ~]$ python3 | ||
+ | Python 3.11.5 | packaged by conda-forge | (main, Aug 27 2023, 03:34:09) [GCC 12.3.0] on linux | ||
+ | Type " | ||
+ | >>> | ||
+ | >>> | ||
+ | >>> | ||
+ | >>> | ||
+ | >>> | ||
+ | True | ||
+ | >>> | ||
+ | |||
+ | </ | ||
+ | |||
+ | * **seaborn** | ||
+ | * https:// | ||
+ | * statistical data visualization | ||
+ | * also on centos7 counterpart | ||
+ | |||
+ | |||
+ | < | ||
+ | |||
+ | Successfully installed seaborn-0.13.0 | ||
+ | [hmeij@cottontail2 ~]$ python3 | ||
+ | Python 3.11.5 | packaged by conda-forge | (main, Aug 27 2023, 03:34:09) [GCC 12.3.0] on linux | ||
+ | Type " | ||
+ | >>> | ||
+ | >>> | ||
+ | |||
+ | </ | ||
+ | ===== Pytraj ==== | ||
+ | |||
+ | * git clone https:// | ||
+ | * depends on module miniconda3/ | ||
+ | * load python and import package | ||
+ | * module load pytraj/ | ||
+ | |||
+ | |||
+ | < | ||
+ | |||
+ | python ./setup.py install | ||
+ | |||
+ | Adding pytraj 2.0.6.dev0 to easy-install.pth file | ||
+ | |||
+ | Installed / | ||
+ | Processing dependencies for pytraj==2.0.6.dev0 | ||
+ | Searching for numpy==1.22.3 | ||
+ | Best match: numpy 1.22.3 | ||
+ | Adding numpy 1.22.3 to easy-install.pth file | ||
+ | Installing f2py script to / | ||
+ | Installing f2py3 script to / | ||
+ | Installing f2py3.9 script to / | ||
+ | |||
+ | Using / | ||
+ | Finished processing dependencies for pytraj==2.0.6.dev0 | ||
+ | |||
+ | </ | ||
+ | |||
+ | ===== Flye ===== | ||
+ | |||
+ | * module: flye/2.9.2 | ||
+ | * auto loads module: miniconda3/ | ||
+ | * local build (without installation) | ||
+ | * https:// | ||
+ | * queues amber128, test, mw128 mw256 (rocky 8) | ||
+ | |||
+ | < | ||
+ | |||
+ | # module will load relevant stuff, location | ||
+ | cd / | ||
+ | |||
+ | [hmeij@cottontail2 2.9.2]$ ./bin/flye --version | ||
+ | 2.9.2-b1794 | ||
+ | [hmeij@cottontail2 2.9.2]$ python bin/flye --version | ||
+ | 2.9.2-b1794 | ||
+ | |||
+ | </ | ||
+ | |||
+ | ===== Masurca ===== | ||
+ | |||
+ | * module: masurca/ | ||
+ | * https:// | ||
+ | * supports openmp | ||
+ | * queues amber128, test, mw128, mw256 (rocky 8) | ||
+ | |||
+ | < | ||
+ | |||
+ | [hmeij@cottontail2 ~]$ module load masurca/ | ||
+ | [hmeij@cottontail2 ~]$ masurca --version | ||
+ | version 4.1.0 | ||
+ | |||
+ | |||
+ | |||
+ | [hmeij@cottontail2 ~]$ module show masurca/ | ||
+ | --------------------------------------------------------------------------------------------------- | ||
+ | / | ||
+ | --------------------------------------------------------------------------------------------------- | ||
+ | whatis(" | ||
+ | whatis(" | ||
+ | whatis(" | ||
+ | whatis(" | ||
+ | whatis(" | ||
+ | depends_on(" | ||
+ | depends_on(" | ||
+ | prepend_path(" | ||
+ | prepend_path(" | ||
+ | prepend_path(" | ||
+ | help([[ | ||
+ | This module loads the masurca environment | ||
+ | toolchain gnu9 with openmpi4 | ||
+ | |||
+ | Version 4.1.0 | ||
+ | |||
+ | ]]) | ||
+ | |||
+ | |||
+ | </ | ||
+ | |||
+ | ===== Lammps ===== | ||
+ | |||
+ | * module: lammps/ | ||
+ | * supports openmp, feature release versus the normal stable release | ||
+ | * includes colloid class2 kspace misc molecule ml-pace reaction mc packages and gpu for cuda | ||
+ | * queues amber128 and test only (cuda 11.6) | ||
+ | |||
+ | * module: lammps/ | ||
+ | * supports openmp | ||
+ | * includes colloid class2 kspace misc molecule ml-pace packages and gpu for cuda | ||
+ | * queues amber128 and test only (cuda 11.6) | ||
+ | * also read miniconda section for tensorflow/ | ||
+ | |||
+ | < | ||
+ | |||
+ | [hmeij@cottontail2 ~]$ ll / | ||
+ | total 314276 | ||
+ | -rwxr-xr-x 1 hmeij its 133876672 Apr 27 14:24 lmp_mpi | ||
+ | -rwxr-xr-x 1 hmeij its 141597552 Apr 27 15:37 lmp_mpi-cuda-double-double | ||
+ | -rwxr-xr-x 1 hmeij its 141254208 Apr 27 14:49 lmp_mpi-cuda-single-double | ||
+ | -rwxr-xr-x 1 hmeij its 140802904 Apr 27 15:11 lmp_mpi-cuda-single-single | ||
+ | -rwxr-xr-x 1 hmeij its 133347480 Apr 27 14:06 lmp_serial | ||
+ | |||
+ | # note July 2023 | ||
+ | # there are now versions with packages REACTION and MC added | ||
+ | # same names with postfix ' | ||
+ | |||
+ | </ | ||
Line 23: | Line 399: | ||
* example: / | * example: / | ||
- | ===== Miniconda3 ===== | + | ===== Miniconda3-py39 ===== |
* module: miniconda3/ | * module: miniconda3/ | ||
* Miniconda framework with python 3.9 | * Miniconda framework with python 3.9 | ||
- | * After loading module '' | ||
* '' | * '' | ||
+ | * '' | ||
+ | * **iqtree** rocky8 queues (cottontail2) | ||
+ | * alsao installed in miniconda2& | ||
+ | * see [[cluster: | ||
+ | |||
+ | < | ||
+ | module load miniconda3/ | ||
+ | which conda | ||
+ | / | ||
+ | |||
+ | [hmeij@cottontail2 slurm]$ conda list | grep iqtree | ||
+ | iqtree | ||
+ | |||
+ | iqtree --version | ||
+ | IQ-TREE multicore version 2.0.3 for Linux 64-bit built Dec 20 2020 | ||
+ | |||
+ | # https:// | ||
+ | # calterlab, annika (failed to install with condo, used pip) | ||
+ | Successfully installed | ||
+ | GridDataFormats-1.0.1 MDAnalysis-2.3.0 | ||
+ | biopython-1.79 fasteners-0.18 gsd-2.6.1 | ||
+ | joblib-1.2.0 mmtf-python-1.1.3 mrcfile-1.4.3 | ||
+ | msgpack-1.0.4 networkx-2.8.8 threadpoolctl-3.1.0 | ||
+ | |||
+ | # torch (also in centos 7 python 3.8.3 | ||
+ | # likely not compatible with cuda9.x, may be 10.2 | ||
+ | # so use test queue which has cuda 11.6 | ||
+ | # ezzyatlab, jared | ||
+ | Successfully installed | ||
+ | nvidia-cublas-cu11-11.10.3.66 | ||
+ | nvidia-cuda-nvrtc-cu11-11.7.99 | ||
+ | nvidia-cuda-runtime-cu11-11.7.99 | ||
+ | nvidia-cudnn-cu11-8.5.0.96 | ||
+ | torch-1.13.0 | ||
+ | |||
+ | # jupyter, jupyterlab, jupyter-nbclassic | ||
+ | # starrlab, max | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | |||
+ | # chernoff lab | ||
+ | # https:// | ||
+ | |||
+ | [hmeij@cottontail2 ~]$ conda list | grep novo | ||
+ | soapdenovo2 | ||
+ | |||
+ | </ | ||
+ | |||
+ | For **Lammps** (starrlab) 25April2023 | ||
+ | |||
+ | * https:// | ||
+ | * consult / | ||
+ | * make yes-gpu | ||
+ | * make yes-ml-pace | ||
+ | |||
+ | < | ||
+ | |||
+ | |||
+ | [hmeij@cottontail2 ~]$ which python | ||
+ | / | ||
+ | [hmeij@cottontail2 ~]$ pip list | grep tensorflow | ||
+ | tensorflow | ||
+ | tensorflow-io-gcs-filesystem | ||
+ | [hmeij@cottontail2 ~]$ which pacemaker | ||
+ | / | ||
+ | |||
+ | </ | ||
===== Amber ===== | ===== Amber ===== | ||
+ | |||
+ | * module: amber/22 | ||
+ | * Amber22 with AmberTools22 | ||
+ | * embedded openmpi 1.4.1 | ||
+ | * how to run on centos7 nodes | ||
+ | * https:// | ||
* module: amber/20 | * module: amber/20 |