This is an old revision of the document!
Also read these pages and this page will make more sense: Warewulf Stateless, Warewulf Statefull.
For some time now I have been looking for a provisioning tool. I've tried along the way …
The requirements of the provisioning tool were two fold:
So I settled on Warewulf which does these two approaches and sports an active forum for questions.
Not finding much on the “golden image” use of Warewulf, I've written up experiences. Hope it helps somebody. In this write we start with setting up stateless nodes, then we adept them to statefull using the “golden image” approach.
Install Warewulf and poke around the shell wwsh
. Consult the stateless page. Lets set up stateless node b6
. The files have been loaded in the MySQL database. The “ww” files are template driven files whose contents will be populated by warewulf, like changing IPs and HWADDRes. I keep them in my centos-6 stateless chroot but use them in other chroots as well (/var/chgroot/centos-6/root/wwtemplates/).
wwsh node new b6 --netdev=eth0 \ --hwaddr=00:00:00:00:00:00 --ipaddr=192.168.1.12 \ --netmask=255.255.0.0 --network=255.255.0.0 --groups=wwnodes wwsh node set b6 --netdev=eth1 \ --hwaddr=00:00:00:00:00:00 --ipaddr=10.10.100.12 \ --netmask=255.255.0.0 --network=255.255.0.0 wwsh provision set b6 --fileadd passwd,shadow,group wwsh provision set b6 --fileadd hosts,bashrc,profile wwsh provision set b6 --fileadd network.ww,ifcfg-eth1.ww
As opposed to the stateless, which grabs it's OS content from the master node, in the “golden image” approach we are going to retrieve the content of a selected node (node b0 in this example) to the Warewul master (node petaltail in this case).
Set rsync
to work between the two. Unmount any NFS file systems on the node. Adjust the exclusion selections in template file (ie after /home is unmounted on node I do want the mount point…scratch space, etc).
# minder: all NFS file systems unmounted? mkdir /var/chgroots/goldimages; cd /var/chgroots/goldimages SOURCEADDR=b0 wwmkchroot golden-system /var/chroots/goldimages/b0.chroot
Next, modify the properties of the node to image. http://warewulf.lbl.gov/trac/wiki/Recipes/StatefulProvisioning
wwsh object modify -s bootloader=sda b6 wwsh object modify -s diskformat=sda1,sda7 b6 wwsh object modify -s filesystems= \ "mountpoint=/boot:dev=sda1:type=ext4:size=500, \ dev=sda3:type=swap:size=2048, \ mountpoint=/:dev=sda7:type=ext4:size=fill" \ b6 # see note below on filesystems....
More on the filesystems
option later. In this example I want to set up /, /boot and swap. Sizes in MB. Only one hard drive on this node (sda).
Next we need to get the node booted and trasnfer the VNFS image made from the node b0 contents. At this time look on your master node in /var/lib/mysql and make sure you have enough disk space (these VNFS images will be around 1 GB as observed). Also, back up that database with mysqldump
.
# make the image, takes 10 minutes or so wwvnfs --chroot=/var/chroots/goldimages/b0.chroot # switch node to image VNFS wwsh provision set b6 --vnfs=b0.chroot # just to be prudent wwsh pxe update wwsh dhcp update service dhcpd restart # check the configs wwsh object print b6 -p :all wwsh provision list # next for provisioning (just to sure) on first PXE boot wwsh provision set --bootlocal=UNDEF b6 # turn the node on
The console of the target node will now show the IP being assigned, the getvnfs
process (be patient) and finally the root login. Check around the node for improvements, tweeks, then adjust and recreate the VNFS.
After all that is done, disable provisioning so that the master ignores the PXE boot and the target node boots of local disk.
# ignore PXE boot wwsh provision set --bootlocal=EXIT b6
filesystems
This is currently not working as expected. In my first attempts I'd specify sda1 (size=500), sda2 (size=2048, type=swap) and sda3 (size=fill) but what I end up with is a standard layout it looks like. Any sizes are also ignored. So for now I just pick the ones I want (sda1, sda3, sda7).
This also happens after I remove any UUID references in /etc/fstab, clean up /etc/mtab and clean any and all files in /dev/disk/by-uuid and rebuild VNFS.
fdisk -l Disk /dev/sda: 80.0 GB, 80026361856 bytes 255 heads, 63 sectors/track, 9729 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x000ce092 Device Boot Start End Blocks Id System /dev/sda1 * 1 13 104391 83 Linux /dev/sda2 14 1543 12289725 83 Linux /dev/sda3 1544 1798 2048287+ 82 Linux swap / Solaris /dev/sda4 1799 9729 63705757+ 5 Extended /dev/sda5 1799 2053 2048256 83 Linux /dev/sda6 2054 2184 1052226 83 Linux /dev/sda7 2185 9729 60605181 83 Linux df -h Filesystem Size Used Avail Use% Mounted on /dev/sda7 57G 1.9G 53G 4% / tmpfs 12G 0 12G 0% /dev/shm /dev/sda1 95M 49M 42M 54% /boot
Warewulf 3.6.99 and CentOS 6.5
To avoid the problems detailed above I started over with a fresh node. First I installed CentOS 6.7 vanilla on the hard disk with partitions sda1 (/boot, 500 MB), sd2 (2048 MB, swap) and sda3
# comment out for golden image #hybridize += /usr/X11R6 #hybridize += /usr/lib/locale #hybridize += /usr/lib64/locale #hybridize += /usr/include
wwbootstrap --chroot=/var/chroots/goldimages/n0.chroot 2.6.32-573.12.1.el6.x86_64 wwsh provision set --vnfs=n0.chroot --bootstrap=2.6.32-573.12.1.el6.x86_64 n22
# /etc/fstab, edited (clean also mtab and remove files /dev/disk/by-uuid/* tmpfs /dev/shm tmpfs defaults 0 0 devpts /dev/pts devpts gid=5,mode=620 0 0 sysfs /sys sysfs defaults 0 0 proc /proc proc defaults 0 0 10.11.103.42:/home /home nfs tcp,soft,intr,bg 10.11.103.42:/sanscratch /sanscratch nfs tcp,soft,intr,bg
# /boot/grub/device.map # This file was written by Warewulf bootstrap (capability setup-filesystems) (hd0) /dev/sda
# /etc/grub.conf # This file was written by Warewulf bootstrap (capability setup-filesystems) serial –speed= –unit= –word= –parity= terminal_input console serial; terminal_output console serial default 0 timeout 10 root (hd0,0)
title CentOS release 6.7 - 2.6.32-573.12.1.el6.x86_64
kernel /vmlinuz-2.6.32-573.12.1.el6.x86_64 ro root=/dev/sda3 rhgb quiet initrd /initramfs-2.6.32-573.12.1.el6.x86_64.img