cluster:187
Warning: Undefined array key -1 in /usr/share/dokuwiki/inc/html.php on line 1458
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
cluster:187 [2019/12/10 13:26] hmeij07 |
cluster:187 [2020/08/17 08:01] (current) hmeij07 |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| \\ | \\ | ||
| **[[cluster: | **[[cluster: | ||
| + | |||
| + | Slurm links: | ||
| + | |||
| + | * https:// | ||
| + | * https:// | ||
| + | * https:// | ||
| + | * https:// | ||
| + | |||
| + | Other useful links. | ||
| + | |||
| + | * https:// | ||
| + | * https:// | ||
| + | * scheduler wrapper, inside container: NV_GPU=2,3 nvidia-docker run ... | ||
| + | * (container sees host gpu 2,3 as container gpu 0,1) | ||
| + | * https:// | ||
| + | * https:// | ||
| + | * https:// | ||
| + | * | ||
| ==== NGC Docker Containers ==== | ==== NGC Docker Containers ==== | ||
| - | Trying to understand how to leverage GPU ready applications on the Nvidia NGC web site (Nvidia GPU Cloud). Download docker containers and buidl your own on premise catalog. Can't wrap myself around the problem of how to integrate containers with the Openlava | + | Trying to understand how to leverage GPU ready applications on the Nvidia NGC web site (Nvidia GPU Cloud). Download docker containers and build your own on premise catalog. Run GPU ready software on compute nodes with docker containers. Can't wrap myself around the problem of how to integrate containers with the our scheduler |
| * https:// | * https:// | ||
| Line 11: | Line 29: | ||
| < | < | ||
| - | # Assumes CentOS | + | # get docker on centos |
| - | # Assumes NVIDIA Driver is installed as per requirements ( < 340.29 ) | + | curl -fsSL https:// |
| - | # Install DOCKER | + | ./ |
| - | sudo curl -fsSL https:// | + | |
| - | # Start DOCKER | + | # systemctl |
| - | sudo systemctl start docker | + | systemctl enable docker |
| - | # Add dockeruser, usermod change | + | systemctl start docker |
| - | sudo adduser dockeruser | + | |
| + | # dockeruser, usermod change | ||
| + | adduser dockeruser | ||
| usermod -aG docker dockeruser | usermod -aG docker dockeruser | ||
| - | # Install NV-DOCKER | + | |
| - | # GET NVIDIA-DOCKER | + | # get nvidia-docker |
| - | wget -P /tmp https:// | + | # wget -P /tmp https:// |
| - | # INSTALL | + | wget https:// |
| - | sudo rpm -i / | + | |
| - | # Start NV-DOCKER Service | + | # rpm -i / |
| + | # make nvidia-docker | ||
| + | |||
| + | # systemctl | ||
| + | systemctl enable nvidia-docker | ||
| systemctl start nvidia-docker | systemctl start nvidia-docker | ||
| Line 37: | Line 61: | ||
| # or | # or | ||
| + | |||
| docker pull nvidia/cuda | docker pull nvidia/cuda | ||
| Line 48: | Line 73: | ||
| NGC Deep Learning Ready Docker Containers: | NGC Deep Learning Ready Docker Containers: | ||
| + | |||
| NVIDIA DIGITS - nvcr.io/ | NVIDIA DIGITS - nvcr.io/ | ||
| TensorFlow - nvcr.io/ | TensorFlow - nvcr.io/ | ||
| Line 59: | Line 85: | ||
| # in the catalog you can also find | # in the catalog you can also find | ||
| + | |||
| docker pull nvcr.io/ | docker pull nvcr.io/ | ||
| docker pull nvcr.io/ | docker pull nvcr.io/ | ||
| docker pull nvcr.io/ | docker pull nvcr.io/ | ||
| docker pull nvcr.io/ | docker pull nvcr.io/ | ||
| - | # not all at the latest versions | + | |
| + | # not all at the latest versions | ||
| # and amber would have to be custom build on top of nvidia/cuda | # and amber would have to be custom build on top of nvidia/cuda | ||
| Line 73: | Line 101: | ||
| # DIGITS example | # DIGITS example | ||
| - | # if you passed GPU ID 2,3 for example, | + | # if you passed |
| NV_GPU=0,1 nvidia-docker run --name digits -d -p 5000:5000 nvidia/ | NV_GPU=0,1 nvidia-docker run --name digits -d -p 5000:5000 nvidia/ | ||
| Line 80: | Line 109: | ||
| </ | </ | ||
| + | |||
| + | There are some other issues... | ||
| + | |||
| + | * inside the container the user invoked application runs as root so copying files back and forth is a problem | ||
| + | * file systems, home directory and scratch spaces need to be mounted inside container | ||
| + | * GPUs need to be reserved via scheduler on a host then made available to container (see above) | ||
| + | |||
| + | Some notes from https:// | ||
| + | |||
| + | < | ||
| + | |||
| + | # NGC containers are hosted in a repository called nvcr.io | ||
| + | # A Docker container is the running instance of a Docker image. | ||
| + | |||
| + | # All NGC Container images are based on the CUDA platform layer (nvcr.io/ | ||
| + | |||
| + | # mount host directory to container location | ||
| + | |||
| + | -v $HOME:/ | ||
| + | |||
| + | # pull images | ||
| + | |||
| + | docker pull nvcr.io/ | ||
| + | docker images | ||
| + | |||
| + | # detailed information of container | ||
| + | |||
| + | / | ||
| + | |||
| + | # specifying a user | ||
| + | |||
| + | -u $(id -u):$(id -g) | ||
| + | |||
| + | # allocate GPUs | ||
| + | |||
| + | NV_GPU=0,1 nvidia-docker run ... | ||
| + | |||
| + | # custom build images ... | ||
| + | # looks complex based on Dockerfile config file commands | ||
| + | # see link | ||
| + | |||
| + | </ | ||
| + | |||
| \\ | \\ | ||
| **[[cluster: | **[[cluster: | ||
cluster/187.1576002394.txt.gz · Last modified: 2019/12/10 13:26 by hmeij07
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
