All things sysadmin

Today I wrote an Ansible playbook to set up an environment for a docker demo I will be giving shortly. In the demo I will be using three hosts, and I want the containers to be able to speak to each other across hosts. To this end, I’m using Open vSwitch. The setup is quite straight forward, set up the bridge, get the meshed GRE tunnels up and off you go.
I first set this up in a lab, with firewalls disabled. But knowing that I will give the demo on public infrastructure, I still wrote the play to allow everything on a particular interface (an isolated cloud-network) through UFW.
When I ran my playbook against a few cloud servers, the containers couldn’t talk to each other on account of the GRE tunnels not working.

So I enabled logging in UFW, and soon started seeing these types of entries

1
2
3

[UFW BLOCK] IN=eth2 OUT= MAC=<redacted>
SRC=<redacted> DST=<redacted> LEN=76 TOS=0x00 PREC=0x00 TTL=64 ID=36639 DF
PROTO=47

Upon checking which rule actually dropped the packets (iptables -L -nv), it transpired that the culprit was

1
2

1    97 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0
ctstate INVALID

It turns out that a change in the 3.18 kernel and onwards means that unless either of the nf_conntrack_pptp or nf_conntrack_proto_gre modules are loaded, any GRE packets will be marked as INVALID, as opposed to NEW and subsequently ESTABLISHED.

So in order to get openvswitch working with UFW, there are two solutions; Either explicitly allow protocol 47, or load one of the aforementioned kernel modules.

Should you go for the former solution, this is the rule you need to beat to the punch:

1
2
3
4

$ grep -A 2 "drop INVALID" /etc/ufw/before.rules
# drop INVALID packets (logs these in loglevel medium and higher)
-A ufw-before-input -m conntrack --ctstate INVALID -j ufw-logging-deny
-A ufw-before-input -m conntrack --ctstate INVALID -j DROP

with -A ufw-before-input -p 47 -i $iface -j ACCEPT