cluster:225
Warning: Undefined array key -1 in /usr/share/dokuwiki/inc/html.php on line 1458
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
cluster:225 [2024/04/07 14:32] hmeij07 [Testing] |
cluster:225 [2024/05/21 10:06] (current) hmeij07 |
||
|---|---|---|---|
| Line 63: | Line 63: | ||
| < | < | ||
| - | # n78 first ... | + | # n78 first ... can reimage cuda-11.6 from n101 (no problem, tests success) |
| - | # n89 second | + | |
| # make sure / | # make sure / | ||
| # scp into place from n100 (centos8, possibly caused by warewulf...) | # scp into place from n100 (centos8, possibly caused by warewulf...) | ||
| + | # however old nvidia packages still in OS (driver 510 toolkit 11.6)... | ||
| + | # rpm -qa | grep ^nvidia | wc -l results in 16 packages... | ||
| + | # what happens on dfn check-update ??? | ||
| + | |||
| + | # n[100-101] skipping for now | ||
| + | # this is a package install and there is no nvidia_uninstall (runfile) | ||
| + | # an upgrade would require internet 'dnf check-update; | ||
| + | # switching between rpm install and runfile is NOT recommended | ||
| + | # and 'dnf erase nvidia*' | ||
| + | |||
| + | # n79 next (no problem) | ||
| + | # but during testing found this error which I had seen BEFORE cuda install | ||
| + | reason: | ||
| + | # follow up, nvidia driver " | ||
| + | # this is a warning mostly by may interfere as does docker likely | ||
| + | https:// | ||
| + | # disabled docker on n79 for now 04/15/2024 9:06AM | ||
| + | # also rotated the memory dimms some time later, seems to have fixed issue | ||
| + | # started docker back up on n79 05/06/2024 9:56AM (has been up 17 days by now) | ||
| + | |||
| + | # n89 next (no problem) | ||
| + | # but upon reboot I encountered that error for the FIRST time on this node | ||
| + | # need to research it is somewhat related to cuda install | ||
| + | # n80 (same error upon reboot after driver install) | ||
| + | # n81 (same error upon reboot after driver install) | ||
| + | # n90 (same error upon reboot after toolkit install, not driver. weird) | ||
| + | # n88 (failed toolkit install, ran / | ||
| + | # re-installed driver, reboot, re-installed tookit, reboot, | ||
| + | # no error occurs! ) | ||
| + | # n87 (ran nvidia-uninstall first, driver then toolkit, errors shows up) | ||
| + | # n86 & n85 same as n87 | ||
| + | # n84 (no error shows up) | ||
| + | # n82 (as n87 but error shows up after driver before toolkit install) | ||
| + | # n83 (error shows up fater toolkit install, not driver install reboot) | ||
| sh ./ | sh ./ | ||
| Line 73: | Line 106: | ||
| dkms build? yes | dkms build? yes | ||
| rebuild initramfs? yes | rebuild initramfs? yes | ||
| + | xconfig? no | ||
| error nvidia module can not be loaded | error nvidia module can not be loaded | ||
| reboot fixed that | reboot fixed that | ||
| Line 115: | Line 149: | ||
| sh cuda_12.4.0_550.54.14_linux.run | sh cuda_12.4.0_550.54.14_linux.run | ||
| + | |||
| + | REBOOT and check date before launching slurm | ||
| + | mv / | ||
| =========== | =========== | ||
| Line 214: | Line 251: | ||
| ** CentOS 7 on n89 ** | ** CentOS 7 on n89 ** | ||
| + | |||
| + | The steps above can also be done for the default cuda installation on exx96 where the soft link ''/ | ||
| Next test is to see if older software runs compatible with newer drivers. We test that by running a gpu program against new 550 driver and cuda toolkit 9.2 and see if it works (~hmeij/ | Next test is to see if older software runs compatible with newer drivers. We test that by running a gpu program against new 550 driver and cuda toolkit 9.2 and see if it works (~hmeij/ | ||
cluster/225.1712514764.txt.gz · Last modified: 2024/04/07 14:32 by hmeij07
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
