DokuWiki

User Tools

Site Tools

Trace:
cluster:139

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
Next revision Both sides next revision
cluster:139 [2015/04/09 14:49]
hmeij 		cluster:139 [2018/08/01 15:24]
hmeij07 [Warewulf Stateless]
Line 2: Line 2:
 **[[cluster:0|Back]]** **[[cluster:0|Back]]**
  
-===== Warewulf LBL =====+===== Warewulf Stateless =====
  
   * [[http://warewulf.lbl.gov/trac]] Warewulf is a scalable systems management suite originally developed to manage large high-performance Linux clusters.  My Project Kusu replacement since IBM bought up Platform LSF and dished the hpccommunnity.org web site, grrrh. (old info at https://dokuwiki.wesleyan.edu/doku.php?id=cluster:88)   * [[http://warewulf.lbl.gov/trac]] Warewulf is a scalable systems management suite originally developed to manage large high-performance Linux clusters.  My Project Kusu replacement since IBM bought up Platform LSF and dished the hpccommunnity.org web site, grrrh. (old info at https://dokuwiki.wesleyan.edu/doku.php?id=cluster:88)
Line 59: Line 59:
 </code> </code>
  
-Next comes a piece of mystery. When executing  ''wwinit ALL'' I ran into a lether, no less, error. After much digging found the answer here and surprisingly found little in the documentation on this.+Next comes a piece of mystery. When executing  ''wwinit ALL'' I ran into a lethal, no less, error. After much digging found the answer here and surprisingly found little in the documentation on this.
  
   * Lethal error thrown by module: /usr/libexec/warewulf/wwinit/91-vnfs.init   * Lethal error thrown by module: /usr/libexec/warewulf/wwinit/91-vnfs.init
   * answer is here https://groups.google.com/a/lbl.gov/forum/#!topic/warewulf/5OhfZCf_eqE   * answer is here https://groups.google.com/a/lbl.gov/forum/#!topic/warewulf/5OhfZCf_eqE
 +  * quoting 
 +
 +<code>
 +What it appears you'll need to do initially is build out a chroot
 +directory. For example, for a Scientific Linux 6.x install, you'd do:
 +
 +    # wwmkchroot sl-6 /var/chroots/sl-6     (or whatever path you want
 +to store them at -- run wwmkchroot -h for the help)
 +
 +  That will give the base chroot for the VNFS at /var/chroots/sl-6.
 +You'll then set the environment variable CHROOTDIR when you execute
 +wwinit.
 +
 +    # CHROOTDIR=/var/chroots/sl-6 wwinit VNFS
 +
 +  I know when you use the 'icr' wwinit functions, it creates a base
 +chroot based upon the host OS. But none of the other wwinit scripts,
 +by default, create one; So you will need to build the chroot and
 +specify where the directory is. Then when the VNFS part of wwinit is
 +ran, it will build the VNFS and import it into the datastore at that
 +time, setting the one specified as the default VNFS in the
 +configuration.
 +</code>
  
 So we start by making the chroot directories, first we'll build a generic centos-6. Then we initialize the warewulf environment. So we start by making the chroot directories, first we'll build a generic centos-6. Then we initialize the warewulf environment.
Line 122: Line 145:
 </code> </code>
  
-Next restart some warewulf services. I had to edit the /etc/warewulf/dhcpd-template.conf several times and hardcoded the values in for network, netmask and ipaddr becuase I realized this server is my virtualization KVM host and eth0 is bridged to br0 and warewulf is looking at the first ethernet and picks up eth1.+Next restart some warewulf services. I had to edit the /etc/warewulf/dhcpd-template.conf several times and hardcoded the values in for network, netmask and ipaddr because I realized this server is my virtualization KVM test host and eth0 is bridged to br0 and warewulf is looking at the first ethernet and picks up eth1.
  
 <code> <code>
Line 132: Line 155:
 [root@petaltail ~]# wwsh pxe update [root@petaltail ~]# wwsh pxe update
 No nodes found No nodes found
 +
 +[root@petaltail ~]# wwsh
 +Warewulf> node new b51 --netdev=eth0 --hwaddr=00:00:00:00:00:00 --ipaddr=192.168.1.5 --groups=wwnodes
 +Warewulf> provision set --lookup groups wwnodes --vnfs=centos-6 --bootstrap=2.6.32-504.8.1.el6.x86_64
 +Are you sure you want to make the following changes to 1 node(s):
 +
 +     SET: BOOTSTRAP            = 2.6.32-504.8.1.el6.x86_64
 +     SET: VNFS                 = centos-6
 +
 +Yes/No> y
 +Warewulf> quit
  
 </code> </code>
 +
 +Booting the node we observe the eth0 MAC address being picked up by DHCPd followed by the tftpboot process.  The node boots, et voila, CentOS 6.6 //stateless// compute node. Next import files we want the same on all nodes and associate them with the nodes.
 +
 +<code>
 +
 +[root@petaltail ~]# wwsh file import /etc/passwd
 +
 +[root@petaltail ~]# wwsh file list
 +dynamic_hosts           : -rw-r--r-- 0   root root            10174 /etc/hosts
 +group                   : -rw-r--r-- 1   root root             6247 /etc/group
 +passwd                  : -rw-r--r-- 1   root root            20094 /etc/passwd
 +shadow                  : ---------- 1   root root            23663 /etc/shadow
 +
 +[root@petaltail ~]# wwsh provision set b[0-51] --fileadd passwd
 +
 +[root@petaltail ~]# wwsh provision print
 +#### b51.cluster ##############################################################
 +    b51.cluster: BOOTSTRAP        = 2.6.32-504.8.1.el6.x86_64
 +    b51.cluster: VNFS             = centos-6
 +    b51.cluster: FILES            = dynamic_hosts,group,passwd,shadow
 +    b51.cluster: PRESHELL         = FALSE
 +    b51.cluster: POSTSHELL        = FALSE
 +    b51.cluster: CONSOLE          = UNDEF
 +    b51.cluster: PXELINUX         = UNDEF
 +    b51.cluster: SELINUX          = DISABLED
 +    b51.cluster: KARGS            = "quiet"
 +    b51.cluster: BOOTLOCAL        = FALSE
 +...
 +
 +</code>
 +
 +Next build a hybrid VNFS, this is the way to add packages to the nodes.  For example NTP.
 +
 +<code>
 +
 +cd /var/chroots/centos-6/
 +mkdir vnfs
 +
 +vi etc/fstab (inside of chroot area, edit)
 +192.168.1.217:/var/chroots/centos-6 /vnfs nfs defaults 0 0
 +
 +wwvnfs --chroot /var/chroots/centos-6 --hybridpath=/vnfs
 +Overwrite original: y
 +
 +# reboot node, observe mount, then add NTP to chgroot
 +
 +yum --tolerant --installroot /var/chroots/centos-6 -y install ntp
 +
 +vi etc/init.d/ntpd (inside of chroot, edit)
 +# chkconfig: 35 58 74
 +
 +# then make the following links in etc/rc[3|5].d/S58ntpd pointing to ../init.d/ntpd
 +
 +vi etc/ntp.conf (inside of chroot, sole contents are, IP is master on private network)
 +restrict default ignore
 +restrict 127.0.0.0
 +server 192.168.1.217
 +restrict 192.168.1.217 nomodify
 +
 +wwvnfs --chroot /var/chroots/centos-6 --hybridpath=/vnfs -y
 +VNFS 'centos-6 has been imported
 +
 +# reboot node
 +
 +</code>
 +
 +Then build the node up:
 +
 +  * use yum to install the openlava 2.2.1 RPM (pulls in tcl)
 +    * copy the openlava config files into the centos-6 are
 +  * use yum to install postfix 
 +    * add links in rc3.d and rc5.d
 +    * remove sendmail links
 +  * yum install perl
 +  * yum install munge
 +    * build RPMs from tar ball
 +  * rebuild the VNFS and reboot
 +
 +Sometimes you can edit the files in the chroot directly, sometimes you must modify the installtoot directly.  It's easiest to just do the latter all the time.
 +
 +<code>
 +
 +[root@petaltail ~]# cd /var/chroots
 +[root@petaltail chroots]# chroot centos-6
 +[root@petaltail /]# pwd                  
 +/                          <--- inside of installroot              
 +[root@petaltail /]# mkdir /var/log/munge 
 +[root@petaltail /]# chown munge:munge /var/log/munge
 +[root@petaltail /]# mkdir /var/log/slurm
 +[root@petaltail /]# chown slurm:munge /var/log/slurm
 +chown: invalid user: `slurm:munge'
 +
 +# since the passwd|shadow|group files come from database you need to create the relevant lines
 +
 +[root@petaltail /]# chown slurm:munge /var/log/slurm
 +[root@petaltail /]# exit
 +exit
 +[root@petaltail chroots]# ls
 +centos-6
 +
 +# outside edit commenting out rc.local directives making these dirs etc
 +[root@petaltail chroots]# vi centos-6/etc/rc.local
 +
 +# and don't forget
 +[root@petaltail chroots]# wwvnfs --chroot /var/chroots/centos-6 --hybridpath=/vnfs  -y
 +
 +# reboot node
 +</code>
 +
 +
 +To build short hostnames you can create a template inside of the chroot environment.
 +
 +<code>
 +
 +#--- build file CHROOT/root/wwtemplates/network.ww
 + NETWORKING=yes
 + HOSTNAME=%{NODENAME}
 +#--- end 
 +
 +# add that file (using wwsh provision) to the nodes.
 +
 +[root@]# wwsh file import /var/chroot/centos-6/root/wwtemplates/network.ww \
 + --path=/etc/sysconfig/network --name=network.ww
 +
 +[root@]# wwsh provision set n[00-15] --fileadd=network.ww
 +
 +</code>
 +
 +Second interface: create a template inside of the chroot environment.
 +
 +<code>
 +
 +wwsh node set b49 --netdev=eth1 \
 +--hwaddr=00:00:00:00:00:00 --ipaddr=10.10.1.55 \
 +--netmask=255.255.0.0  --network=255.255.0.0
 +
 +#--- build file CHROOT/root/wwtemplates/ifcfg-eth1.ww
 +DEVICE=eth1
 +BOOTPROTO=static
 +ONBOOT=yes
 +HWADDR=%{NETDEVS::ETH1::HWADDR}
 +IPADDR=%{NETDEVS::ETH1::IPADDR}
 +NETMASK=%{NETDEVS::ETH1::NETMASK}
 +NETWORK=%{NETDEVS::ETH1::NETWORK}
 +#--- end 
 +
 +# add that file (using wwsh provision) to the nodes.
 +
 +[root@]# wwsh file import /var/chroot/centos-6/root/wwtemplates/ifcfg-eth1.ww \
 + --path=/etc/sysconfig/network-scripts/ifcfg-eth1 --name=ifcfg-eth1.ww
 +
 +[root@]# wwsh provision set n[00-15] --fileadd=ifcfg-eth1.ww
 +
 +</code>
 +
 +Now, lets put it all together which can form the basis for a script.
 +
 +<code>
 +
 +# make sure it boots across network, alter BIOS settings
 +
 +wwsh node new b6 --netdev=eth0 \
 +--hwaddr=00:00:00:00:00:00 --ipaddr=192.168.1.12 \
 +--netmask=255.255.0.0  --network=255.255.0.0
 +--groups=wwnodes
 +
 +wwsh node set b6 --netdev=eth1 \
 +--hwaddr=00:00:00:00:00:00 --ipaddr=10.10.100.12 \
 +--netmask=255.255.0.0  --network=255.255.0.0
 +
 +wwsh provision set b6 --fileadd passwd,shadow,group
 +wwsh provision set b6 --fileadd hosts,bashrc,profile
 +wwsh provision set b6 --fileadd network.ww,ifcfg-eth1.ww
 +
 +
 +</code>
 +
 +
 +[[cluster:144|Warewulf Golden Image]]
  
 Useful links Useful links
cluster/139.txt ยท Last modified: 2018/08/16 08:58 by hmeij07

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
CC Attribution-Share Alike 4.0 International Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki