Multi route table with PBF and VPN server

Deciding on a title of this post was almost as arduous as building the lab and devising the configuration for this post. This post is inspired by a question on the Cisco Community forums which required a solution to permit inbound VPN (wireguard) traffic which would then be forwarded out a second VPN (OpenVPN)connection depending on traffic type. Since the second VPN tunnel is a full-tunnel used for web browsing it needs to be configured as the default route. Problems begin when trying to run both VPNs simultaneously, as the return traffic for the wireguard VPN will be sent via the OpenVPN interface.

For this lab topology each VM is placed in its own host-based network attached via a single interface. An opnsense VM providers the gateway for each subnet and an interface with a route to the internet.

# roadwarrior /etc/wireguard/wg0.conf
[Interface]
Address = 192.168.199.1/24
PrivateKey = QDk9KQNbKqbkqSEjYxk7stT3VoEdNR9VrSMJ1HZksUg=

[Peer]
PublicKey = 7nrnmM8NU8xV8MrFxmGJXGLAb2gIXGAbEd7XUsugj3E=
EndPoint = 192.168.57.67:51820
AllowedIPs = 0.0.0.0/0

# vpnserver /etc/wireguard/wg.conf
[Interface]
Address = 192.168.199.254/24
SaveConfig = true
ListenPort = 51820
PrivateKey = mIYN2FpMyp0ddPBVbTX4YMubbfcaNfnrnC0DjCjcV2Q=

[Peer]
PublicKey = 5cvscSXxz5nEO+gp2nTEdHCn/vXVm5OCv0hUHFMKl0U=
AllowedIPs = 192.168.199.1/32

snr@roadwarrior:/etc/wireguard# wg show
interface: wg0
  public key: 5cvscSXxz5nEO+gp2nTEdHCn/vXVm5OCv0hUHFMKl0U=
  private key: (hidden)
  listening port: 38581
  fwmark: 0xca6c

peer: 7nrnmM8NU8xV8MrFxmGJXGLAb2gIXGAbEd7XUsugj3E=
  endpoint: 192.168.57.67:51820
  allowed ips: 0.0.0.0/0
  latest handshake: 9 minutes, 52 seconds ago
  transfer: 2.21 KiB received, 4.60 KiB sent

For the OpenVPN setup I am opting to use the default client/server config files along with the example keys, all found under /usr/share/doc/openvpn. It doesn’t take long to get the VPN up between the ‘vpnserver’ and ‘vpnprovider’ VMs.

To make things more interesting we ensure that the ‘vpnprovider’ VM does not push a default route to the ‘vpnserver’ VM, use the following configuration command on the OpenVPN server:

;push "redirect-gateway def1 bypass-dhcp"

Before we move on we must configure IP forwarding and NAT on the ‘vpnprovider’ VM for the OpenVPN client traffic:

echo 1 > /proc/sys/net/ipv4/ip_forward
iptables -A FORWARD -i enp0s3 -o tun0 -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A FORWARD -s 10.8.0.0/16 -o enp0s3 -j ACCEPT
iptables -t nat -A POSTROUTING -s 10.8.0.0/24 -o enp0s3 -j MASQUERADE

Next comes the fun part where we create a new routing table and routing policy on the ‘vpnserver’ VM. Lets take a quick look at the routing table on, we can see both wireguard (wg0) and OpenVPN (tun0) services providing entries:

snr@vpnserver:/home/snr# sudo ip route list table main
default via 192.168.56.254 dev enp0s3
10.8.0.1 via 10.8.0.5 dev tun0 
10.8.0.5 dev tun0 proto kernel scope link src 10.8.0.6 
169.254.0.0/16 dev enp0s3 scope link metric 1000 
192.168.56.0/24 dev enp0s3 proto kernel scope link src 192.168.56.102 
192.168.199.0/24 dev wg0 proto kernel scope link src 192.168.199.254 

To ensure HTTPS traffic does not leave via the gateway on ‘vpnserver’ VM subnet, I create a rule on opnsense to reject the traffic:

Checking the live log confirms traffic from the wireguard subnet (192.168.199.1) source from ‘roadwarror’ is attempting to egress via the local gateway and getting dropped:

So with a firewall rule in place we must avoid using the ‘vpnserver’ VM default route. Next we create a new routing table which we will provide an alternative default route via tun0 interface.

echo 100 vpnproxy > /etc/iproute2/rt_tables

The ip rule command is used to create policy based routing, in this case we are sending all traffic streams from the wireguard source IP pool destined to either ports 53 or 443 to the new vpnproxy routing table:

ip rule add from 192.168.199.0/24 to 0.0.0.0/0 dport 53 table vpnproxy
ip rule add from 192.168.199.0/24 to 0.0.0.0/0 dport 443 table vpnproxy

Now create the default route for the vpnproxy route table specifying the tun0 next-hop:

ip route add default via 10.8.0.5 dev tun0 table vpnproxy
ip route flush cache

Confirm that the above configuration commands have taken:

snr@vpnserver:/home/snr# sudo ip rule list
0:	from all lookup local
32764:	from 192.168.199.0/24 dport 443 lookup vpnproxy
32765:	from 192.168.199.0/24 dport 53 lookup vpnproxy
32766:	from all lookup main
32767:	from all lookup default
snr@vpnserver:/home/snr#
snr@vpnserver:/home/snr# sudo ip route show table vpnproxy
default via 10.8.0.5 dev tun0

The OpenVPN server will drop incoming tunnel traffic from unknown sources, in our case traffic must be sourced from the IP 10.8.0.6 . We must therefore configure iptables to source NAT our wireguard traffic to the tun0 interface:

iptables -A FORWARD -i tun0 -o wg0 -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A FORWARD -s 192.168.199.0/24 -o tun0 -p tcp --dport 443 -j ACCEPT
iptables -A FORWARD -s 192.168.199.0/24 -o tun0 -p tcp --dport 53 -j ACCEPT
iptables -A FORWARD -s 192.168.199.0/24 -o tun0 -p udp --dport 53 -j ACCEPT
iptables -t nat -A POSTROUTING -s 192.168.199.0/24 -o tun0 -j MASQUERADE

From out ‘roadwarrior’ VM tcp/443, tcp/54 and ucp/53 traffic is now routed via the local wireguard interface, NAT’d on the ‘vpnserver’ server then sent towards the ‘vpnprovider’ VM. Finally the ‘vpnprovider’ performs source NAT on the OpenVPN traffic and egresses it towards the internet.