Differences

This shows you the differences between two versions of the page.

--- cluster:139 [2015/04/08 18:50]
hmeij [Warewulf LBL]
+++ cluster:139 [2018/08/16 12:58]
hmeij07 [3.99]
@@ Line 2: / Line 2: @@
 **[[cluster:0|Back]]**
-===== Warewulf LBL =====
+===== Warewulf Stateless =====
   * [[http://warewulf.lbl.gov/trac]] Warewulf is a scalable systems management suite originally developed to manage large high-performance Linux clusters.  My Project Kusu replacement since IBM bought up Platform LSF and dished the hpccommunnity.org web site, grrrh. (old info at https://dokuwiki.wesleyan.edu/doku.php?id=cluster:88)
@@ Line 59: / Line 59: @@
 </code>
-Next comes a piece of mystery. When executing  ''wwinit ALL'' I ran into a lether, no less, error. After much digging found the answer here and surprisingly found little in the documentation on this.
+Next comes a piece of mystery. When executing  ''wwinit ALL'' I ran into a lethal, no less, error. After much digging found the answer here and surprisingly found little in the documentation on this.
   * Lethal error thrown by module: /usr/libexec/warewulf/wwinit/91-vnfs.init
   * answer is here https://groups.google.com/a/lbl.gov/forum/#!topic/warewulf/5OhfZCf_eqE
+  * quoting
+<code>
+What it appears you'll need to do initially is build out a chroot
+directory. For example, for a Scientific Linux 6.x install, you'd do:
+    # wwmkchroot sl-6 /var/chroots/sl-6     (or whatever path you want
+to store them at -- run wwmkchroot -h for the help)
+  That will give the base chroot for the VNFS at /var/chroots/sl-6.
+You'll then set the environment variable CHROOTDIR when you execute
+wwinit.
+    # CHROOTDIR=/var/chroots/sl-6 wwinit VNFS
+  I know when you use the 'icr' wwinit functions, it creates a base
+chroot based upon the host OS. But none of the other wwinit scripts,
+by default, create one; So you will need to build the chroot and
+specify where the directory is. Then when the VNFS part of wwinit is
+ran, it will build the VNFS and import it into the datastore at that
+time, setting the one specified as the default VNFS in the
+configuration.
+</code>
 So we start by making the chroot directories, first we'll build a generic centos-6. Then we initialize the warewulf environment.
@@ Line 122: / Line 145: @@
 </code>
-Next restart some warewulf services. I had to edit the /etc/warewulf/dhcpd-template.conf several times and hardcoded the values in for network, netmask and ipaddr becuase I realized this server is my virtualization KVM host and eth0 is bridged to br0 and warewulf is looking at the first ethernet and picks up eth1.
+Next restart some warewulf services. I had to edit the /etc/warewulf/dhcpd-template.conf several times and hardcoded the values in for network, netmask and ipaddr because I realized this server is my virtualization KVM test host and eth0 is bridged to br0 and warewulf is looking at the first ethernet and picks up eth1.
 <code>
@@ Line 132: / Line 155: @@
 [root@petaltail ~]# wwsh pxe update
 No nodes found
+[root@petaltail ~]# wwsh
+Warewulf> node new b51 --netdev=eth0 --hwaddr=00:00:00:00:00:00 --ipaddr=192.168.1.5 --groups=wwnodes
+Warewulf> provision set --lookup groups wwnodes --vnfs=centos-6 --bootstrap=2.6.32-504.8.1.el6.x86_64
+Are you sure you want to make the following changes to 1 node(s):
+     SET: BOOTSTRAP            = 2.6.32-504.8.1.el6.x86_64
+     SET: VNFS                 = centos-6
+Yes/No> y
+Warewulf> quit
+</code>
+Booting the node we observe the eth0 MAC address being picked up by DHCPd followed by the tftpboot process.  The node boots, et voila, CentOS 6.6 //stateless// compute node. Next import files we want the same on all nodes and associate them with the nodes.
+<code>
+[root@petaltail ~]# wwsh file import /etc/passwd
+[root@petaltail ~]# wwsh file list
+dynamic_hosts           : -rw-r--r-- 0   root root            10174 /etc/hosts
+group                   : -rw-r--r-- 1   root root             6247 /etc/group
+passwd                  : -rw-r--r-- 1   root root            20094 /etc/passwd
+shadow                  : ---------- 1   root root            23663 /etc/shadow
+[root@petaltail ~]# wwsh provision set b[0-51] --fileadd passwd
+[root@petaltail ~]# wwsh provision print
+#### b51.cluster ##############################################################
+    b51.cluster: BOOTSTRAP        = 2.6.32-504.8.1.el6.x86_64
+    b51.cluster: VNFS             = centos-6
+    b51.cluster: FILES            = dynamic_hosts,group,passwd,shadow
+    b51.cluster: PRESHELL         = FALSE
+    b51.cluster: POSTSHELL        = FALSE
+    b51.cluster: CONSOLE          = UNDEF
+    b51.cluster: PXELINUX         = UNDEF
+    b51.cluster: SELINUX          = DISABLED
+    b51.cluster: KARGS            = "quiet"
+    b51.cluster: BOOTLOCAL        = FALSE
+...
+</code>
+Next build a hybrid VNFS, this is the way to add packages to the nodes.  For example NTP.
+<code>
+cd /var/chroots/centos-6/
+mkdir vnfs
+vi etc/fstab (inside of chroot area, edit)
+.168.1.217:/var/chroots/centos-6 /vnfs nfs defaults 0 0
+wwvnfs --chroot /var/chroots/centos-6 --hybridpath=/vnfs
+Overwrite original: y
+# reboot node, observe mount, then add NTP to chgroot
+yum --tolerant --installroot /var/chroots/centos-6 -y install ntp
+vi etc/init.d/ntpd (inside of chroot, edit)
+# chkconfig: 35 58 74
+# then make the following links in etc/rc[3|5].d/S58ntpd pointing to ../init.d/ntpd
+vi etc/ntp.conf (inside of chroot, sole contents are, IP is master on private network)
+restrict default ignore
+restrict 127.0.0.0
+server 192.168.1.217
+restrict 192.168.1.217 nomodify
+wwvnfs --chroot /var/chroots/centos-6 --hybridpath=/vnfs -y
+VNFS 'centos-6 has been imported
+# reboot node
+</code>
+Then build the node up:
+  * use yum to install the openlava 2.2.1 RPM (pulls in tcl)
+    * copy the openlava config files into the centos-6 are
+  * use yum to install postfix
+    * add links in rc3.d and rc5.d
+    * remove sendmail links
+  * yum install perl
+  * yum install munge
+    * build RPMs from tar ball
+  * rebuild the VNFS and reboot
+Sometimes you can edit the files in the chroot directly, sometimes you must modify the installtoot directly.  It's easiest to just do the latter all the time.
+<code>
+[root@petaltail ~]# cd /var/chroots
+[root@petaltail chroots]# chroot centos-6
+[root@petaltail /]# pwd
+/                          <--- inside of installroot
+[root@petaltail /]# mkdir /var/log/munge
+[root@petaltail /]# chown munge:munge /var/log/munge
+[root@petaltail /]# mkdir /var/log/slurm
+[root@petaltail /]# chown slurm:munge /var/log/slurm
+chown: invalid user: `slurm:munge'
+# since the passwd|shadow|group files come from database you need to create the relevant lines
+[root@petaltail /]# chown slurm:munge /var/log/slurm
+[root@petaltail /]# exit
+exit
+[root@petaltail chroots]# ls
+centos-6
+# outside edit commenting out rc.local directives making these dirs etc
+[root@petaltail chroots]# vi centos-6/etc/rc.local
+# and don't forget
+[root@petaltail chroots]# wwvnfs --chroot /var/chroots/centos-6 --hybridpath=/vnfs  -y
+# reboot node
+</code>
+To build short hostnames you can create a template inside of the chroot environment.
+<code>
+#--- build file CHROOT/root/wwtemplates/network.ww
+ NETWORKING=yes
+ HOSTNAME=%{NODENAME}
+#--- end
+# add that file (using wwsh provision) to the nodes.
+[root@]# wwsh file import /var/chroot/centos-6/root/wwtemplates/network.ww \
+ --path=/etc/sysconfig/network --name=network.ww
+[root@]# wwsh provision set n[00-15] --fileadd=network.ww
+</code>
+Second interface: create a template inside of the chroot environment.
+<code>
+wwsh node set b49 --netdev=eth1 \
+--hwaddr=00:00:00:00:00:00 --ipaddr=10.10.1.55 \
+--netmask=255.255.0.0  --network=255.255.0.0
+#--- build file CHROOT/root/wwtemplates/ifcfg-eth1.ww
+DEVICE=eth1
+BOOTPROTO=static
+ONBOOT=yes
+HWADDR=%{NETDEVS::ETH1::HWADDR}
+IPADDR=%{NETDEVS::ETH1::IPADDR}
+NETMASK=%{NETDEVS::ETH1::NETMASK}
+NETWORK=%{NETDEVS::ETH1::NETWORK}
+#--- end
+# add that file (using wwsh provision) to the nodes.
+[root@]# wwsh file import /var/chroot/centos-6/root/wwtemplates/ifcfg-eth1.ww \
+ --path=/etc/sysconfig/network-scripts/ifcfg-eth1 --name=ifcfg-eth1.ww
+[root@]# wwsh provision set n[00-15] --fileadd=ifcfg-eth1.ww
+</code>
+Now, lets put it all together which can form the basis for a script.
+<code>
+# make sure it boots across network, alter BIOS settings
+wwsh node new b6 --netdev=eth0 \
+--hwaddr=00:00:00:00:00:00 --ipaddr=192.168.1.12 \
+--netmask=255.255.0.0  --network=255.255.0.0
+--groups=wwnodes
+wwsh node set b6 --netdev=eth1 \
+--hwaddr=00:00:00:00:00:00 --ipaddr=10.10.100.12 \
+--netmask=255.255.0.0  --network=255.255.0.0
+wwsh provision set b6 --fileadd passwd,shadow,group
+wwsh provision set b6 --fileadd hosts,bashrc,profile
+wwsh provision set b6 --fileadd network.ww,ifcfg-eth1.ww
+</code>
+[[cluster:144|Warewulf Golden Image]]
+Useful links
+  * [[http://www.admin-magazine.com/HPC/Articles/Warewulf-Cluster-Manager-Master-and-Compute-Nodes]]
+  * [[http://www.admin-magazine.com/HPC/Articles/warewulf_cluster_manager_completing_the_environment]]
+  * http://warewulf.lbl.gov/trac/wiki/Documentation
+==== 3.6.99 ====
+<code>
+  cd warewulf/
+  wget http://warewulf.lbl.gov/downloadds/testing/3.6.99/warewulf-common-3.6.99.tar.gz
+  wget http://warewulf.lbl.gov/downloads/testing/3.6.99/warewulf-common-3.6.99.tar.gz
+  wget http://warewulf.lbl.gov/downloads/testing/3.6.99/warewulf-provision-3.6.99.tar.gz
+  wget http://warewulf.lbl.gov/downloads/testing/3.6.99/warewulf-cluster-3.6.99.tar.gz
+  wget http://warewulf.lbl.gov/downloads/testing/3.6.99/warewulf-vnfs-3.6.99.tar.gz
+  yum groupinstall "Development Tools"
+  man rpmbuild
+  rpmbuild --help
+  rpmbuild -ta warewulf-common-3.6.99.tar.gz
+  rpmbuild -ta warewulf-cluster-3.6.99.tar.gz
+  rpmbuild -ta warewulf-vnfs-3.6.99.tar.gz
+  yum install libselinux-devel
+  yum install  libacl-devel  libattr-devel
+  rpmbuild -ta warewulf-provision-3.6.99.tar.gz
 </code>
+[[cluster:143|Warewulf Statefull]]  [[cluster:144|Warewulf Golden Image]]
 \\
 **[[cluster:0|Back]]**

DokuWiki

User Tools

Site Tools

Differences

Page Tools