Differences

This shows you the differences between two versions of the page.

--- cluster:144 [2015/12/11 18:40] – [Warewulf Golden Image] hmeij
+++ cluster:144 [2018/07/26 18:52] (current) – [Deploying] hmeij07
@@ Line 47: / Line 47: @@
 Set ''rsync'' to work between the two. Unmount any NFS file systems on the node. Adjust the exclusion selections in template file (ie after /home is unmounted on node I do want the mount point...scratch space, etc).
-  * /usr/lib/libexec/warewulf/wwmkchroot/golden-ststem.tmpl
 <code>
 # minder: all NFS file systems unmounted?
+# or add rsync excludes in
+# /usr/libexec/warewulf/wwmkchroot/golden-system.tmpl
 mkdir /var/chgroots/goldimages; cd /var/chgroots/goldimages
@@ Line 65: / Line 67: @@
 wwsh object modify -s bootloader=sda b6
-wwsh object modify -s diskformat=sda1,sda3 b6
+wwsh object modify -s diskformat=sda1,sda7 b6
 wwsh object modify -s filesystems= \
 "mountpoint=/boot:dev=sda1:type=ext4:size=500, \
 dev=sda3:type=swap:size=2048, \
-mountpoint=/:dev=sda7:type=ext4:size=70000" \
+mountpoint=/:dev=sda7:type=ext4:size=fill" \
 b6
+# see note below on filesystems....
 </code>
@@ Line 85: / Line 89: @@
 # switch node to image VNFS
 wwsh provision set b6 --vnfs=b0.chroot
+# Rajil adds for GPU
+# Keep in mind that nouveau should be disabled,
+wwsh provision set c038 --kargs=\
+"acpi_pad.disable=1 console=ttyS0,115200 rd.driver.blacklist=nouveau nouveau.modeset=0"
 # just to be prudent
@@ Line 115: / Line 124: @@
 **filesystems**
-This is currently not working as expected. In my first attempts I'd specify sda1 (size=500), sda2 (size=2048, type=swap) and sda3 (size=fill) but what I end up with is a standard layout it looks like.  Any sizes are also ignored. So for now I just pick the ones I want (sda1, sda4, sda7).
+This is currently not working as expected. In my first attempts I'd specify sda1 (size=500), sda2 (size=2048, type=swap) and sda3 (size=fill) but what I end up with is a standard layout it looks like.  Any sizes are also ignored. So for now I just pick the ones I want (sda1, sda3, sda7).
+Note: this problem turns out to be hardware related, it is not appearing on newer hardware
+ --- //[[hmeij@wesleyan.edu|Henk]] 2017/09/13 08:11//
-This also happens after I remove any UUID references in /etc/fstab and clean any and all files in /dev/disk and rebuild VNFS.
+This also happens after I remove any UUID references in /etc/fstab, clean up /etc/mtab and clean any and all files in /dev/disk/by-uuid and rebuild VNFS.
 <code>
@@ Line 150: / Line 162: @@
 Warewulf 3.6.99 and CentOS 6.5
+==== Part 2 ====
+To avoid the problems detailed above I started over with a fresh node.  First I installed CentOS 6.7 vanilla on the hard disk with partitions sda1 (/boot, 500 MB), sd2 (2048 MB, swap) and sda3 (/, size=fill). Since I had a new kernel now we need to make a new bootstrap image and provision it.
+<code>
+  wwbootstrap --chroot=/var/chroots/goldimages/n0.chroot 2.6.32-573.12.1.el6.x86_64
+  wwsh provision set n22 --vnfs=n0.chroot --bootstrap=2.6.32-573.12.1.el6.x86_64
+</code>
+Before we create the n0.chroot VNFS image edit the file and comment out these lines, with a golden image approach we do not have a hybridization path but we want these file systems included.
+<code>
+# /etc/warewul/vnfs.conf
+# comment out for golden image
+#hybridize += /usr/X11R6
+#hybridize += /usr/lib/locale
+#hybridize += /usr/lib64/locale
+#hybridize += /usr/include
+</code>
+I also cleaned the following files inside the chroot environment:
+<code>
+# /etc/fstab, edited (clean also mtab and remove files /dev/disk/by-uuid/*
+tmpfs                   /dev/shm                tmpfs   defaults        0 0
+devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
+sysfs                   /sys                    sysfs   defaults        0 0
+proc                    /proc                   proc    defaults        0 0
+.11.103.42:/home /home nfs tcp,soft,intr,bg
+.11.103.42:/sanscratch /sanscratch nfs tcp,soft,intr,bg
+# /boot/grub/device.map
+# This file was written by Warewulf bootstrap (capability setup-filesystems)
+(hd0) /dev/sda
+# /etc/grub.conf
+# This file was written by Warewulf bootstrap (capability setup-filesystems)
+serial --speed= --unit= --word= --parity=
+terminal_input console serial; terminal_output console serial
+default 0
+timeout 10
+root (hd0,0)
+title CentOS release 6.7 - 2.6.32-573.12.1.el6.x86_64
+    kernel /vmlinuz-2.6.32-573.12.1.el6.x86_64 ro root=/dev/sda3 rhgb quiet
+    initrd /initramfs-2.6.32-573.12.1.el6.x86_64.img
+</code>
+Now when first provisioning happens the right partitions are created and the node imaged. With bootload=EXIT or simply shutting down the warewulf master dhcpd process, the node is now booting from local disk.
+Yea.
+==== Deploying ====
+As part of my deployment I edited out any device information in file ''/etc/udev/rules.d/70-persistent-net.rules'', that information will be filled in on boot. Remade my VNFS. The reason for this is that in migrating from RHEL 5.5 to CentOS 6.7 I observed the order of the NICs to reverse themselves (but not always). Not sure why but will tell me what NIC eth0 will be.
+Then we build a template file with node specs in it like so:
+<code>
+# HP blades reverses HWADDR (don't ask)
+ # use nic port bottom (no changes)
+ # set both ipaddr/hwaddr then PXEboot, wait...ww times out but images
+ # post edit
+ #   /etc/udev/rules.d fix eth? entries per below, then startup scripts
+ #   eth0 down CC 10.10.x.x onboot=no
+ #   eth1 up   CD 192.168.x.x onboot=yes
+ # reboot (with bootload=EXIT)
+ n1  1C:C1:DE:1C:82:D5 192.168.102.10 1C:C1:DE:1C:82:D4 10.10.102.10 10.11.103.10
+ n2  1C:C1:DE:1C:81:01 192.168.102.11 1C:C1:DE:1C:81:00 10.10.102.11 10.11.103.11
+...
+</code>
+My deploy script (give an entire line from above as arguments)
+<code>
+#!/bin/bash
+ # deploy a n0.chroot node via PXE golden image transfer
+ # dynamic files are always in stateless CHROOT/centos-6/root/wwtemplates
+ node=$1
+ hwaddr0=$2
+ ipaddr0=$3
+ hwaddr1=$4
+ ipaddr1=$5
+ ipaddri=$6
+ if [ $# != 6 ]; then
+         echo "missing args: node hwaddr0 ipaddr0 hwaddr1 ipaddr1 ipaddri"
+         exit
+ fi
+ wwsh object delete $node -y
+ sleep 3
+ wwsh node new $node --netdev=eth0 \
+ --hwaddr=$hwaddr0 --ipaddr=$ipaddr0 \
+ --netmask=255.255.0.0  --network=255.255.0.0 -y
+ wwsh node set $node --netdev=eth1 \
+ --hwaddr=$hwaddr1 --ipaddr=$ipaddr1 \
+ --netmask=255.255.0.0  --network=255.255.0.0 -y
+ wwsh node set $node --netdev=ib0 \
+ --ipaddr=$ipaddri \
+ --netmask=255.255.0.0  --network=255.255.0.0 -y
+ wwsh provision set $node --fileadd passwd,shadow,group -y
+ wwsh provision set $node --fileadd hosts,bashrc,profile -y
+ wwsh provision set $node --fileadd network.ww,ifcfg-eth1.ww,ifcfg-ib0.ww -y
+# note: no diskpartition, already exists and fails on this hardware,
+# otherwise add diskpartion=sda so that "mkbootable" succeeds
+ wwsh object modify -s bootloader=sda $node -y
+ wwsh object modify -s diskformat=sda1,sda4 $node -y
+ if [ "$node" == 'n22' ]; then
+ # golden images with 3 partitions
+ wwsh object modify -s filesystems="mountpoint=/boot:dev=sda1:type=ext4:size=1024,dev=sda2:type=swap:size=6144,mountpoint=/:dev=sda3:type=ext4:size=+" $node -y
+ else
+ # hp blade 4 partitions
+ wwsh object modify -s filesystems="mountpoint=/boot:dev=sda1:type=ext4:size=1024,dev=sda2:type=swap:size=6144,mountpoint=/:dev=sda4:type=ext4:size=+" $node -y
+ fi
+ wwsh provision set $node --vnfs=n0.chroot -y
+ wwsh provision set $node --bootstrap=2.6.32-573.12.1.el6.x86_64 -y
+ wwsh provision set --bootlocal=UNDEF $node -y
+ wwsh pxe update
+ wwsh dhcp update
+ service dhcpd restart
+ service httpd restart
+ echo "now reboot: $node"
+ echo "wwsh provision set --bootlocal=EXIT $node -y"
+</code>
 \\
 **[[cluster:0|Back]]**