Back

Note #1

CentOS 8.1 with the standard firewalld.
If this is of interest to you this was how I managed to get it work: 

EXTIFACE=MASTER_NODE_EXT_INTERFACE_DEVICE (e.g. eno1)
INTIFACE=MASTER_NODE_INTERNAL_INTERFACE_DEVICE (e.g. eno2)
INTIPADDR=MASTER_IP_OF_INTERNAL_IFAC
PREFIX=PREFIX_OF_INTERNAL_NETWORK
firewall-cmd --change-interface=${EXTIFACE} --zone=public
firewall-cmd --change-interface=${INTIFACE} --zone=trusted --permanent
firewall-cmd --permanent --direct --passthrough ipv4 -t nat -I POSTROUTING -o ${EXTIFACE} -j MASQUERADE -s ${INTIPADDR}/${PREFIX}
firewall-cmd --set-default-zone=trusted
firewall-cmd --reload

And make sure the default route is set on all compute nodes.

Note #2

configured Shorewall on a cluster to do NAT through the head node.

Edit the file /etc/shorewall/snat and add this line: 

MASQUERADE 192.168.0.0/24      eno1

where 192.168.0 is the address range of your node interfaces - clearly you need to change this to fit en01 is the external interface on the head node

My /etc/shorewall/interfaces contains this (forwarding ib0) 

nat     eno1    detect  dhcp
nat     ib0     detect  dhcp

so substitute ib0 for your internal ethernet interface

NAT Story, part 2

This is my second NAT story, for the first one look at The Story Of NAT, part 1

Writing this up so I will remember what I did, and why. Basic problem is this: How do you make a filesystem in a public VLAN available on a private network? One solution is to work with Network Address Translation, or NAT in short. More information at http://en.wikipedia.org/wiki/Network_address_translation

We have a storage device which we refer to as flexstorage.wesleyan.edu which serves up a file system on login node petaltail. 

[root@petaltail ~]# host flexstorage
flexstorage.wesleyan.edu has address 129.133.24.81

[root@petaltail ~]# df -h /home/dlbgroup
Filesystem            Size  Used Avail Use% Mounted on
flexstorage.wesleyan.edu:/share/dlbgroup
                     1000G  588G  413G  59% /home/dlbgroup

Host petaltail has the following interfaces. The file system in question is mounted on host petaltail as VLAN 1 can reach VLAN 24. 

eth0      Link encap:Ethernet  HWaddr 00:18:8B:51:FA:42
          inet addr:192.168.1.217  Bcast:192.168.255.255  Mask:255.255.0.0
eth1      Link encap:Ethernet  HWaddr 00:18:8B:51:FA:44
          inet addr:10.10.100.217  Bcast:10.10.255.255  Mask:255.255.0.0
eth2      Link encap:Ethernet  HWaddr 00:15:17:80:8D:F2
          inet addr:129.133.1.225  Bcast:129.133.1.255  Mask:255.255.255.0
eth3      Link encap:Ethernet  HWaddr 00:15:17:80:8D:F3
          inet addr:192.168.2.2  Bcast:192.168.2.255  Mask:255.255.255.0

But a compute node on our cluster, for example node b1, has the following interfaces, all private 

eth0      Link encap:Ethernet  HWaddr 00:13:D3:F2:C8:EC  
          inet addr:192.168.1.7  Bcast:192.168.255.255  Mask:255.255.0.0
eth1      Link encap:Ethernet  HWaddr 00:13:D3:F2:C8:ED  
          inet addr:10.10.100.7  Bcast:10.10.255.255  Mask:255.255.0.0

So in order to for the compute node b1 to reach the flexstorage server we need to use NAT rules and define a path/route. First we start on petaltail and edit the iptables file and add a “nat filter” masquerade/post routing directives and in the “filter filter” set up a rule connecting eth1 and eth2. 

*nat
# fss public to 10.10
-A POSTROUTING -o eth2 -j MASQUERADE
COMMIT

*filter
# fss public via 10.10
-A FORWARD -i eth1 -o eth2 -m state --state RELATED,ESTABLISHED -j ACCEPT
...
COMMIT

Next, on the compute nodes we need to add routing path and then mount the file system (using an IP because there is no name resolving). These commands are stuck in /etc/rc.local for persistence. 

# /etc/rc.local
route add -host 129.133.24.81 gw  10.10.100.217 eth1
mount 129.133.24.81:/share/dlbgroup /home/dlbgroup -t nfs -o soft,intr,bg

[root@b1 ~]# df -h /home/dlbgroup
Filesystem            Size  Used Avail Use% Mounted on
129.133.24.81:/share/dlbgroup
                     1000G  588G  413G  59% /home/dlbgroup

There is ofcourse a penalty in performance doing this. 

[root@petaltail ~]#  time dd if=/dev/zero of=/home/dlbgroup/foo bs=1024 count=1000000
1000000+0 records in
1000000+0 records out
1024000000 bytes (1.0 GB) copied, 107.961 seconds, 9.5 MB/s

real    1m47.964s
user    0m0.322s
sys     0m2.094s

[root@b1 ~]# time dd if=/dev/zero of=/home/dlbgroup/foo bs=1024 count=1000000
1000000+0 records in
1000000+0 records out
1024000000 bytes (1.0 GB) copied, 110.017 seconds, 9.3 MB/s

real    1m50.027s
user    0m0.271s
sys     0m4.073s


Back