User Tools

Site Tools


cluster:102


Back

This is my second NAT story, for the first one look at The Story Of NAT, part 1

NAT Story, part 2

Writing this up so I will remember what I did, and why. Basic problem is this: How do you make a filesystem in a public VLAN available on a private network? One solution is to work with Network Address Translation, or NAT in short. More information at http://en.wikipedia.org/wiki/Network_address_translation

We have a storage device which we refer to as flexstorage.wesleyan.edu which serves up a file system on login node petaltail.

[root@petaltail ~]# host flexstorage
flexstorage.wesleyan.edu has address 129.133.24.81

[root@petaltail ~]# df -h /home/dlbgroup
Filesystem            Size  Used Avail Use% Mounted on
flexstorage.wesleyan.edu:/share/dlbgroup
                     1000G  588G  413G  59% /home/dlbgroup

Host petaltail has the following interfaces. The file system in question is mounted on host petaltail as VLAN 1 can reach VLAN 24.

eth0      Link encap:Ethernet  HWaddr 00:18:8B:51:FA:42
          inet addr:192.168.1.217  Bcast:192.168.255.255  Mask:255.255.0.0
eth1      Link encap:Ethernet  HWaddr 00:18:8B:51:FA:44
          inet addr:10.10.100.217  Bcast:10.10.255.255  Mask:255.255.0.0
eth2      Link encap:Ethernet  HWaddr 00:15:17:80:8D:F2
          inet addr:129.133.1.225  Bcast:129.133.1.255  Mask:255.255.255.0
eth3      Link encap:Ethernet  HWaddr 00:15:17:80:8D:F3
          inet addr:192.168.2.2  Bcast:192.168.2.255  Mask:255.255.255.0

But a compute node on our cluster, for example node b1, has the following interfaces, all private

eth0      Link encap:Ethernet  HWaddr 00:13:D3:F2:C8:EC  
          inet addr:192.168.1.7  Bcast:192.168.255.255  Mask:255.255.0.0
eth1      Link encap:Ethernet  HWaddr 00:13:D3:F2:C8:ED  
          inet addr:10.10.100.7  Bcast:10.10.255.255  Mask:255.255.0.0

So in order to for the compute node b1 to reach the flexstorage server we need to use NAT rules and define a path/route. First we start on petaltail and edit the iptables file and add a “nat filter” masquerade/post routing directives and in the “filter filter” set up a rule connecting eth1 and eth2.

*nat
# fss public to 10.10
-A POSTROUTING -o eth2 -j MASQUERADE
COMMIT

*filter
# fss public via 10.10
-A FORWARD -i eth1 -o eth2 -m state --state RELATED,ESTABLISHED -j ACCEPT
...
COMMIT

Next, on the compute nodes we need to add routing path and then mount the file system (using an IP because there is no name resolving). These commands are stuck in /etc/rc.local for persistence.

# /etc/rc.local
route add -host 129.133.24.81 gw  10.10.100.217 eth1
mount 129.133.24.81:/share/dlbgroup /home/dlbgroup -t nfs -o soft,intr,bg

[root@b1 ~]# df -h /home/dlbgroup
Filesystem            Size  Used Avail Use% Mounted on
129.133.24.81:/share/dlbgroup
                     1000G  588G  413G  59% /home/dlbgroup

There is ofcourse a penalty in performance doing this.

[root@petaltail ~]#  time dd if=/dev/zero of=/home/dlbgroup/foo bs=1024 count=1000000
1000000+0 records in
1000000+0 records out
1024000000 bytes (1.0 GB) copied, 107.961 seconds, 9.5 MB/s

real    1m47.964s
user    0m0.322s
sys     0m2.094s

[root@b1 ~]# time dd if=/dev/zero of=/home/dlbgroup/foo bs=1024 count=1000000
1000000+0 records in
1000000+0 records out
1024000000 bytes (1.0 GB) copied, 110.017 seconds, 9.3 MB/s

real    1m50.027s
user    0m0.271s
sys     0m4.073s


Back

cluster/102.txt · Last modified: 2011/06/29 11:59 by hmeij