Main article: Ubuntu/Bespin

We give a walkthru of the current iptables, then cover some improvements we can make.

Summary of important information:

Need to restart the docker service after flushing all iptables rules
Loopback interfaces can't talk to each other unless explicitly (via rule) or implicitly (via allow all) allowed
It is important to plan out what services will need to talk to each other
If traffic isn't flowing like it is supposed to, start logging blocked packets, and use tcpdump to see if packets are showing up at the right network interface

Installing Stuff

To make iptables rules persistent:

sudo apt-get -y install netfilter-persistent

Existing Iptables Rules

We walk through each step of the existing iptables script.

Env Var Alias

Start by defining ipt as an alias for the iptables command:

#!/bin/bash
set -e

ipt="sudo /sbin/iptables"

Flush Existing Rules

We flush all the rules from iptables before we start:

# start by flushing all rules and setting defaults
$ipt -F
$ipt -P INPUT ACCEPT
$ipt -P FORWARD ACCEPT
$ipt -P OUTPUT ACCEPT
$ipt -t nat -F
$ipt -t mangle -F
$ipt -F
$ipt -X

Rules for PIA VPN Tunnels

We set up iptables rules that accept traffic coming in from the tunnel, and allow traffic from other interfaces and other networks to leave the tunnel (via nat/masquerading):

##################################
# PIA VPN Tunnels

# These are PIA tunnels that handle traffic from APs
PIA_AP_TUNNELS="tun1"
for TUN in TUNNELS; do
    # Accept all traffic coming in from tunnel
    $ipt -A INPUT -i ${TUN} -j ACCEPT
    # Masquaerade outgoing traffic leaving via the tunnel
    $ipt -t nat -A POSTROUTING -o ${TUN} -j MASQUERADE
done

AP-PIA Tunneling

Next we cover the rules that allow traffic to be forwarded from the AP to the VPN tunnel and to return back to the AP.

##################################
# AP-PIA Tunneling

# Forward outgoing traffic for APs through tunnel
AP="wlan1"
TUN="tun1"
$ipt -A FORWARD -i ${AP} -o ${TUN} -j ACCEPT
$ipt -A FORWARD -i ${TUN} -o ${AP} -m state --state ESTABLISHED,RELATED -j ACCEPT

DNS

This is the section that needs extra rules if we're going to be more strict about connections allowed.

We have the local dnsmasq DNS server set up to forward DNS queries it can't resolve to the PiHole DNS server (running via docker-compose).

If we're going to enable DNS queries that the PiHole can't resolve to reach the wider internet, we need to create a rule to allow traffic from the local loopback interface (lo:1 that the PiHole is listening on, see Ubuntu/Bespin/PiHole) to egress through the VPN tunnel.

Likewise, we want to allow responses from the PiHole's queries to be able to return to the PiHole via the tunnel.

##################################
# DNS Tunneling

# Forward outgoing DNS traffic from lo:1 (PiHole) through PIA tunnel
DNS="lo:1"
TUN="tun1"
PROTOCOLS="udp tcp"
for PROTOCOL in $PROTOCOLS; do
    # PiHole can always send DNS queries out through tunnel
    $ipt -A FORWARD -p ${PROTOCOL} -i ${DNS} -o ${TUN} --dport 53 -j ACCEPT
    # Responses to PiHole can always return via tunnel
    $ipt -A FORWARD -p ${PROTOCOL} -i ${TUN} -o ${DNS} --dport 53 -m state --state ESTABLISHED,RELATED -j ACCEPT
done

Save Rules

Last, we save the rules:

# Make rules persistent
sudo netfilter-persistent save

Improvements

We cover some improvements to the script below.

Strict Traffic Control

Currently, we have a big problem with our iptables configuration. It is in the form of these three lines:

$ipt -P INPUT ACCEPT
$ipt -P FORWARD ACCEPT
$ipt -P OUTPUT ACCEPT

In other words, we are actually accepting all traffic! No matter what! Rules don't matter!

What we really want to do is start by denying all traffic by default, and unblocking the traffic we expect. Change these lines to:

$ipt -P INPUT DENY
$ipt -P FORWARD DENY
$ipt -P OUTPUT ACCEPT

This allows all outgoing traffic by default, but blocks all incoming and forward traffic. This is going to break a lot of things, including DNS, which gives us internet connectivity, so we need to modify our rules.

Modifications needed:

DNS traffic for the local DNS servers should be allowed (both coming in, from other local services querying the DNS server, and going out, in case the DNS server needs to pass a request upstream)
SSH and VPN traffic should be allowed (incoming: only existing connections; outgoing: any connections)
HTTP/HTTPS traffic should be allowed (incoming: only existing connections; outgoing: any connections) - if you don't open these ports, aptitude will have problems installing/updating things

Preamble

#!/bin/bash
set -e

ipt="sudo /sbin/iptables"

Updated Flush Existing Rules

This section has changed - the default policies are now more restrictive.


# Set default policies
$ipt -P INPUT DROP
$ipt -P FORWARD DROP
$ipt -P OUTPUT ACCEPT

# Flush and clear everything
$ipt -t nat -F
$ipt -t mangle -F
$ipt -F
$ipt -X

New Environment Variables

Define various devices for convenience:

# Name of PIA VPN tunnel device
PIATUN="tun1"
# Name of loopback interface for PiHole DNS server
PHDNS="lo:1"
# Name of loopback interface for dnsmasq DNS server
DDNS="lo"
# Name of hostapd AP device
AP="wlan1"

Rules for Allowing Existing Connections

We can set a general rule that, if a connection already exists, it should be allowed to go in or out.

########### INCOMING ##########
# Allow any established connection to come in or out
$ipt -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
$ipt -A OUTPUT -m state --state RELATED,ESTABLISHED -j ACCEPT

New Rules for Allowing Pings

If we want to allow incoming pings:

########### PING ##############
$ipt -A INPUT -p icmp --icmp-type echo-request -j ACCEPT

New Rules for Allowing SSH

We are running an SSH server, so we want to make sure that new incoming connections can be made on port 22.

We need two rules: one to allow incoming SSH connections to be established (input traffic that has a destination of port 22) and one to allow established conversations to continue (input traffic that has a source port of 22).

########### SSH ###############
# Allow incoming SSH sessions, new or established
$ipt -A INPUT -p tcp --dport 22 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
# Allow incoming SSH traffic, if part of established conversation
$ipt -A INPUT -p tcp --sport 22 -m conntrck --ctstate ESTABLISHED -j ACCEPT

New Rules for Allowing VPN

Analogous to the SSH rules above, we make rules for VPN traffic to ensure the VPN server can accept new VPN connections, as well as making sure that existing connections are allowed to continue.

########### VPN ###############
# Allow incoming VPN sessions destined for 1194, new or established
$ipt -A INPUT -p udp --dport 1194 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
# Allow incoming VPN traffic coming from 1194, part of established conversation
$ipt -A INPUT -p udp --sport 1194 -m conntrck --ctstate ESTABLISHED -j ACCEPT

New Rules for Allowing HTTP and HTTPS

We make analogous rules for HTTP and HTTPS:

########### HTTP/HTTPS ########
# Allow incoming HTTP/HTTPS traffic, part of established conversation
$ipt -A INPUT -p tcp --sport 80 -m conntrck --ctstate ESTABLISHED -j ACCEPT
$ipt -A INPUT -p tcp --sport 443 -m conntrck --ctstate ESTABLISHED -j ACCEPT

New Rules for Allowing DHCP

We will also need to allow DHCP on port 67/68. Port 67 is for the server, 68 is for the client. Do this like so:

########### DHCP ##############
# Allow any DHCP traffic to come in or out
$ipt -A INPUT  -p udp --dport 67:68 --sport 67:68 -j ACCEPT
$ipt -A OUTPUT -p udp --dport 67:68 --sport 67:68 -j ACCEPT

Rules to Allow AP-VPN Tunnel

Rule block to allow traffic from the AP to pass through the PIA VPN tunnel:

########### PIA VPN ##############
# This is a PIA VPN tunnel that handles traffic from APs
# Accept all traffic coming in from tunnel
$ipt -A INPUT -i ${PIATUN} -j ACCEPT
# Masquaerade outgoing traffic leaving via the tunnel
$ipt -t nat -A POSTROUTING -o ${PIATUN} -j MASQUERADE

Updated Rules for Allowing DNS

It turns out that our existing DNS rules aren't permissive enough for DNS to work, so we need to update them for the switch to the new DENY policy.

The DNS rules need to accomplish the following:

general DNS traffic
- Any DNS traffic coming in from port 53, part of established conversation, should be allowed

PiHole -> PIA tunnel
- DNS traffic sent from the local PiHole DNS server (127.53.0.1) to the VPN tunnel at tun1 should be allowed, always
- DNS traffic sent from the VPN tunnel back to the local PiHole DNS server should be allowed, if part of an existing conversation

dnsmasq -> PiHole
- Allow DNS traffic sent from the local dnsmasq DNS server to the local PiHole DNS server
- Allow DNS traffic from one local DNS server to another

Access point
- Local DNS queries from the hostapd network should be forwarded to the local dnsmasq DNS server, always
- Local DNS traffic between hostapd network and dnsmasq DNS server always ok

########### DNS ###############
PROTOCOLS="tcp udp"
for prot in $PROTOCOLS; do
    # General DNS Traffic:
    # Allow incoming DNS traffic coming from 53, part of established conversation
    $ipt -A INPUT  -p $prot --sport 53 --dport 1024:65535 -m state --state ESTABLISHED -j ACCEPT

    # PiHole DNS (lo:1) <-> PIA VPN Tunnel (tun0):
    # PiHole can always send DNS queries out through tunnel
    $ipt -A FORWARD -p $prot -i ${PHDNS} -o ${PIATUN} --dport 53 -j ACCEPT
    # Responses to PiHole can always return via tunnel
    $ipt -A FORWARD -p $prot -i ${PIATUN} -o ${PHDNS} --dport 53 -m state --state ESTABLISHED,RELATED -j ACCEPT

    # dnsmasq DNS (lo) <-> PiHole DNS (lo:1)
    # Allow all DNS traffic from local dnsmasq DNS server to local PiHole DNS server
    $ipt -A FORWARD -p $prot -i ${DDNS} -o ${PHDNS} --dport 53 -j ACCEPT
    # Allow responses to dnsmasq to return via the PiHole DNS server
    $ipt -A FORWARD -p $prot -i ${PHDNS} -o ${DDNS} --dport 53 -m state --state ESTABLISHED,RELATED -j ACCEPT

    # hostapd AP (wlan1) <-> dnsmasq DNS (lo)
    # Allow DNS traffic to travel both ways between AP and dnsmasq
    $ipt -A FORWARD -p $prot -i ${AP} -o ${DDNS} --dport 53 -j ACCEPT
    $ipt -A FORWARD -p $prot -o ${AP} -i ${DDNS} --sport 53 -j ACCEPT

done

Playing Nice with Docker

Something I didn't realize (that probably caused me some frustration a few days ago) was that each time I ran the iptables script, and each time it flushed all of the rules, it also flushed DOCKER iptables rules!

Helpful article from Docker on the topic of Docker and iptables: https://docs.docker.com/network/iptables/

It turns out that to allow docker to keep working, you either have to restart (preserving all of the iptables rules we just created, but then starting the docker startup service, which restores all the docker iptables rules), or you have to restart the docker service.

We modify the script by adding the following line at the very end, to allow docker to restore its rules:

# Make rules persistent
sudo netfilter-persistent save

# Restore docker iptables rules
sudo service docker restart

Troubleshooting

Logging Dropped Packets

If you have services that aren't working, troubleshoot by adding logging directives to the bottom of the iptables script:

iptables -N LOGGING
iptables -A INPUT -j LOGGING
iptables -A LOGGING -m limit --limit 2/min -j LOG --log-prefix "iptables dropped: " --log-level 4
iptables -A LOGGING -j DROP

DNS Stops Working

Once I switched over to the more restrictive iptables script, my DNS queries stopped working. I could still ping 8.8.8.8 so I knew the problem was with the dnsmasq DNS server.

First, I discovered that it was trying to send queries directly to 8.8.8.8 instead of to the PiHole at 127.53.0.1 (when did that happen??).

Second, I discovered that the line 127.0.0.1 localhost was missing from my /etc/hosts file, very suspicious. If that line is missing it also makes sudo suuuuuuper slow.

Last of all, docker creates iptables rules, and I was flushing those, and my DNS depends on everything passing through a PiHole run via a docker container, so when docker fails my DNS fails.

Final Script

Here is the final version of the script:

#!/bin/bash
set -e

ipt="sudo /sbin/iptables"

# Set default policies
$ipt -P INPUT DROP
$ipt -P FORWARD DROP
$ipt -P OUTPUT ACCEPT

# Flush and clear everything
$ipt -t nat -F
$ipt -t mangle -F
$ipt -F
$ipt -X

# Name of PIA VPN tunnel device
PIATUN="tun1"
# Name of loopback interface for PiHole DNS server
PHDNS="lo:1"
# Name of loopback interface for dnsmasq DNS server
DDNS="lo"
# Name of hostapd AP device
AP="wlan1"

########### INCOMING ##########
# Allow any established connection to come in or out
$ipt -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
$ipt -A OUTPUT -m state --state RELATED,ESTABLISHED -j ACCEPT

########### PING ##############
# Allow incoming ping requests
$ipt -A INPUT -p icmp --icmp-type echo-request -j ACCEPT

########### SSH ###############
# Allow incoming SSH sessions, new or established
$ipt -A INPUT -p tcp --dport 22 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
# Allow incoming SSH traffic, if part of established conversation
$ipt -A INPUT -p tcp --sport 22 -m conntrack --ctstate ESTABLISHED -j ACCEPT

########### VPN ###############
# Allow incoming VPN sessions destined for 1194, new or established
$ipt -A INPUT -p udp --dport 1194 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
# Allow incoming VPN traffic coming from 1194, part of established conversation
$ipt -A INPUT -p udp --sport 1194 -m conntrack --ctstate ESTABLISHED -j ACCEPT

########### HTTP/HTTPS ########
# Allow incoming HTTP/HTTPS traffic, part of established conversation
$ipt -A INPUT -p tcp --sport 80 -m conntrack --ctstate ESTABLISHED -j ACCEPT
$ipt -A INPUT -p tcp --sport 443 -m conntrack --ctstate ESTABLISHED -j ACCEPT

########### DHCP ##############
# Allow any DHCP traffic to come in or out
$ipt -A INPUT  -p udp --dport 67:68 --sport 67:68 -j ACCEPT
$ipt -A OUTPUT -p udp --dport 67:68 --sport 67:68 -j ACCEPT

########### PIA VPN ##############
# This is a PIA VPN tunnel that handles traffic from APs
# Accept all traffic coming in from tunnel
$ipt -A INPUT -i ${PIATUN} -j ACCEPT
# Masquaerade outgoing traffic leaving via the tunnel
$ipt -t nat -A POSTROUTING -o ${PIATUN} -j MASQUERADE

########### DNS ###############
PROTOCOLS="tcp udp"
for prot in $PROTOCOLS; do
    # General DNS Traffic:
    # Allow incoming DNS traffic coming from 53, part of established conversation
    $ipt -A INPUT  -p $prot --sport 53 --dport 1024:65535 -m state --state ESTABLISHED -j ACCEPT

    # PiHole DNS (lo:1) <-> PIA VPN Tunnel (tun0):
    # PiHole can always send DNS queries out through tunnel
    $ipt -A FORWARD -p $prot -i ${PHDNS} -o ${PIATUN} --dport 53 -j ACCEPT
    # Responses to PiHole can always return via tunnel
    $ipt -A FORWARD -p $prot -i ${PIATUN} -o ${PHDNS} --dport 53 -m state --state ESTABLISHED,RELATED -j ACCEPT

    # dnsmasq DNS (lo) <-> PiHole DNS (lo:1)
    # Allow all DNS traffic from local dnsmasq DNS server to local PiHole DNS server
    $ipt -A FORWARD -p $prot -i ${DDNS} -o ${PHDNS} --dport 53 -j ACCEPT
    # Allow responses to dnsmasq to return via the PiHole DNS server
    $ipt -A FORWARD -p $prot -i ${PHDNS} -o ${DDNS} --dport 53 -m state --state ESTABLISHED,RELATED -j ACCEPT

    # hostapd AP (wlan1) <-> dnsmasq DNS (lo)
    # Allow DNS traffic to travel both ways between AP and dnsmasq
    $ipt -A FORWARD -p $prot -i ${AP} -o ${DDNS} --dport 53 -j ACCEPT
    $ipt -A FORWARD -p $prot -o ${AP} -i ${DDNS} --sport 53 -j ACCEPT
done

# Enable logging
$ipt -N LOGGING
$ipt -A INPUT -j LOGGING
$ipt -A LOGGING -m limit --limit 2/min -j LOG --log-prefix "iptables dropped: " --log-level 4
$ipt -A LOGGING -j DROP

# Make rules persistent
sudo netfilter-persistent save

# Restore docker iptables rules
sudo service docker restart

Ubuntu/Bespin/Old/Iptables

From charlesreid1

Contents