This is my second NAT story, for the first one look at The Story Of NAT, part 1
Writing this up so I will remember what I did, and why. Basic problem is this: How do you make a filesystem in a public VLAN available on a private network? One solution is to work with Network Address Translation, or NAT in short. More information at http://en.wikipedia.org/wiki/Network_address_translation
We have a storage device which we refer to as flexstorage.wesleyan.edu which serves up a file system on login node petaltail.
[root@petaltail ~]# host flexstorage flexstorage.wesleyan.edu has address 22.214.171.124 [root@petaltail ~]# df -h /home/dlbgroup Filesystem Size Used Avail Use% Mounted on flexstorage.wesleyan.edu:/share/dlbgroup 1000G 588G 413G 59% /home/dlbgroup
Host petaltail has the following interfaces. The file system in question is mounted on host petaltail as VLAN 1 can reach VLAN 24.
eth0 Link encap:Ethernet HWaddr 00:18:8B:51:FA:42 inet addr:192.168.1.217 Bcast:192.168.255.255 Mask:255.255.0.0 eth1 Link encap:Ethernet HWaddr 00:18:8B:51:FA:44 inet addr:10.10.100.217 Bcast:10.10.255.255 Mask:255.255.0.0 eth2 Link encap:Ethernet HWaddr 00:15:17:80:8D:F2 inet addr:126.96.36.199 Bcast:188.8.131.52 Mask:255.255.255.0 eth3 Link encap:Ethernet HWaddr 00:15:17:80:8D:F3 inet addr:192.168.2.2 Bcast:192.168.2.255 Mask:255.255.255.0
But a compute node on our cluster, for example node b1, has the following interfaces, all private
eth0 Link encap:Ethernet HWaddr 00:13:D3:F2:C8:EC inet addr:192.168.1.7 Bcast:192.168.255.255 Mask:255.255.0.0 eth1 Link encap:Ethernet HWaddr 00:13:D3:F2:C8:ED inet addr:10.10.100.7 Bcast:10.10.255.255 Mask:255.255.0.0
So in order to for the compute node b1 to reach the flexstorage server we need to use NAT rules and define a path/route. First we start on petaltail and edit the iptables file and add a “nat filter” masquerade/post routing directives and in the “filter filter” set up a rule connecting eth1 and eth2.
*nat # fss public to 10.10 -A POSTROUTING -o eth2 -j MASQUERADE COMMIT *filter # fss public via 10.10 -A FORWARD -i eth1 -o eth2 -m state --state RELATED,ESTABLISHED -j ACCEPT ... COMMIT
Next, on the compute nodes we need to add routing path and then mount the file system (using an IP because there is no name resolving). These commands are stuck in /etc/rc.local for persistence.
# /etc/rc.local route add -host 184.108.40.206 gw 10.10.100.217 eth1 mount 220.127.116.11:/share/dlbgroup /home/dlbgroup -t nfs -o soft,intr,bg [root@b1 ~]# df -h /home/dlbgroup Filesystem Size Used Avail Use% Mounted on 18.104.22.168:/share/dlbgroup 1000G 588G 413G 59% /home/dlbgroup
There is ofcourse a penalty in performance doing this.
[root@petaltail ~]# time dd if=/dev/zero of=/home/dlbgroup/foo bs=1024 count=1000000 1000000+0 records in 1000000+0 records out 1024000000 bytes (1.0 GB) copied, 107.961 seconds, 9.5 MB/s real 1m47.964s user 0m0.322s sys 0m2.094s [root@b1 ~]# time dd if=/dev/zero of=/home/dlbgroup/foo bs=1024 count=1000000 1000000+0 records in 1000000+0 records out 1024000000 bytes (1.0 GB) copied, 110.017 seconds, 9.3 MB/s real 1m50.027s user 0m0.271s sys 0m4.073s