tunnel VRF

This post came about after answering a question on CSC regarding tunnels and VRF, and will elaborate on the the use case I suggested.
Imagine two sites with multiple VRFs which need to communicate intra-VRF over a public WAN, lets run through the configuration options:

Option A
tunnel_vrf_02

Each VRF would require a public IP which it could be NAT’d be. Services would be made available via port-forwarding on the same public IP.
This provide the intra-VRF communication, but it is not very scalable and dependent on the protocols being used is not very secure.

Option B
tunnel_vrf_02b

Using GRE tunnels between the site VRFs sourced from public the IPs allocated to them. By running a dynamic routing protocol across the tunnel, inter-site/ intra-VRF communication would be possible. No NAT would be necessary however each VRF would still require a public IP to be used as a tunnel source.

Tunnel VRF - per-VRF tunnels encapsulated in single site to site IPsec tunnel.

Option C

Simplifying the design further, we add more hardware..!
Each VRF now has a dedicated interface (or sub-interface) on the local site firewall. The firewalls would establish an IPSec tunnel between themselves, which would carry the inter-site traffic and therefor the intra-VRF GRE tunnels.
As before each site only requires one public IP for use with IPSec peering. The GRE tunnels should now be sourced on loopback interfaces within the VRFs.

Option D
tunnel_vrf_04

This is the option we will look at further; no dedicated firewalls and just a single pair of public IP addresses for the tunnel.

The detail

tunnel_gns3

Site A (R1) : 10.1.0.0 /16

  • ASN: 100
  • VRF RED : 10.1.64.0 /18

Site B (R3) : 10.3.0.0 /16

  • ASN: 300
  • VRF RED : 10.3.64.0 /18

We will create two VRFs: WAN and RED.
The two sites are peered via eBGP via R2:

R1#sh ip bgp vpnv4 vrf WAN
     Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 100:1 (default for vrf WAN)
 *>  10.1.0.0/16      0.0.0.0                  0         32768 i
 *>  10.3.0.0/16      192.168.1.2                            0 200 300 i
 *   192.168.1.0/30   192.168.1.2              0             0 200 i
 *>                   0.0.0.0                  0         32768 i
 *>  192.168.3.0/30   192.168.1.2              0             0 200 i

Now lets create a GRE tunnel between the WAN VRF to allow an IGRP between them:

!
interface Tunnel10
 ip vrf forwarding WAN
 ip address 172.16.1.1 255.255.255.252
 tunnel source 192.168.1.1
 tunnel destination 192.168.3.1
 tunnel vrf WAN
!

Can we reach the other end of the tunnel?

R1#ping vrf WAN 172.16.1.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.16.1.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 20/30/36 ms

Of course!

Now assuming this was a production environment it make sense to use loopbacks within the VRFs as tunnel sources and IGRP peer addresses. The reason being that you may have more than one WAN connection and binding a tunnel /IGRP peer to a physical address may result in flapping as a result of problems with the interface. A loopback is always up and reachable providing you have a route advertised to reach it.

Site A:

! 
interface Loopback10
  ip vrf forwarding WAN
  ip address 10.1.1.1 255.255.255.255 
! 
interface Tunnel10
  ip vrf forwarding WAN
  ip address 172.16.1.1 255.255.255.252
  tunnel source Loopback10
  tunnel destination 10.3.1.1
  tunnel vrf WAN
!

Site B:

!
interface Loopback10
  ip vrf forwarding WAN
  ip address 10.3.1.1 255.255.255.255
!
interface Tunnel10
  ip vrf forwarding WAN
  ip address 172.16.1.2 255.255.255.252
  tunnel source Loopback10
  tunnel destination 10.1.1.1
  tunnel vrf WAN 
! 

Next we will configure EIGRP Named Mode processes at both sites:

!
router eigrp R1
 !
 address-family ipv4 unicast vrf WAN autonomous-system 100
  !
  af-interface default
   shutdown
  exit-af-interface
  !
  af-interface Tunnel10
   no shutdown
  exit-af-interface
  !
  topology base
   redistribute connected
  exit-af-topology
  network 172.16.1.0 0.0.0.3
  eigrp router-id 10.1.1.1
  eigrp stub connected static
 exit-address-family
!

Looks good, for a moment, but then R3 starts reporting:

*Dec 20 09:51:54.063: %ADJ-5-PARENT: Midchain parent maintenance for IP midchain out of Tunnel10 - looped chain attempting to stack
*Dec 20 09:51:58.999: %TUN-5-RECURDOWN: Tunnel10 temporarily disabled due to recursive routing
*Dec 20 09:51:58.999: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel10, changed state to down
*Dec 20 09:51:59.035: %DUAL-5-NBRCHANGE: EIGRP-IPv4 100: Neighbor 172.16.1.1 (Tunnel10) is down: interface down

The issue we have now is that the GRE tunnel source IP is being advertised down the tunnel hence the recursive routing. We need to make sure packets destined for the loopback10 interfaces always go via the default route. We can achieve this by restricting the routes being advertised using the EIGRP distribute-list command. The updated EIGRP config now looks like:

!
ip access-list standard DENY_TUNNEL_IPS
 deny   10.1.1.1
 permit any
!
router eigrp R1
 !
 address-family ipv4 unicast vrf WAN autonomous-system 100
  !
  af-interface default
   shutdown
  exit-af-interface
  !
  af-interface Tunnel10
   no shutdown
  exit-af-interface
  !
  topology base
   distribute-list DENY_TUNNEL_IPS out
   redistribute connected
  exit-af-topology
  network 172.16.1.0 0.0.0.3
  eigrp router-id 10.1.1.1
 exit-address-family
!

Inspection of packets between 10.1.1.2 and 10.3.1.2 show that they are being encapsulated so sent down the GRE tunnel.

R1#ping vrf WAN 10.3.1.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.3.1.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 32/33/36 ms
tunnel_vrf_tun10_pcap

VRF RED

With our two sites able to communicate via the GRE tunnel lets turn our attention to the other VRFs. Lets assume these are secure VRFs, the best way to ensure data doesn’t end up in the clear on the public network is to ensure there is no route to get there in the first place.

R1#sh ip route vrf RED
Gateway of last resort is not set

      10.0.0.0/32 is subnetted, 1 subnets
C        10.1.64.1 is directly connected, Loopback101

Looks good, but how do we get a tunnel established with the VRF at the remote site??
The eagle-eyed of you will have noticed two VRF commands in use on the tunnel interfaces:

  • ip vrf forwarding
  • tunnel vrf

The first should be familiar, the second command is crucial to our solution. It is related to the ‘GRE Tunnel IP Source and Destination VRF Membership’ feature. By specifying a VRF with this command you are instructing the router to use that VRF for routing the tunnel source and destination packets. Packets leaving the tunnel are then routed by the VRF specified by the ip vrf forwarding command.

Lets spin up a new tunnel in the RED VRF:

!
interface Tunnel100
 ip vrf forwarding RED
 ip address 172.16.100.1 255.255.255.252
 tunnel source Loopback10
 tunnel destination 10.3.1.1
 tunnel key 100
 tunnel vrf WAN
!

Also note that because we are using the same source and destination IP addresses in the config for two different tunnels we must introduce the tunnel key command, this helps the router differentiate between the incoming GRE packets and know which tunnel they are destined for. The key must match at both ends of the tunnel.

R1#sh int tunnel 100
Tunnel100 is up, line protocol is up
  ...
  Tunnel protocol/transport GRE/IP
    Key 0x64, sequencing disabled

With tunnel100, lets establish an EIGRP adjacency across it:

!
router eigrp R1
!
address-family ipv4 unicast vrf RED autonomous-system 100
  !
  af-interface default
   shutdown
  exit-af-interface
  !
  af-interface Tunnel100
   no shutdown
  exit-af-interface
  !
  topology base
   redistribute connected
  exit-af-topology
  network 172.16.100.0 0.0.0.3
 exit-address-family
!
R1#sh ip route vrf RED eigrp
...
      10.0.0.0/32 is subnetted, 2 subnets
D EX     10.3.64.1 [170/76800640] via 172.16.100.2, 00:02:22, Tunnel100
tunnel_vrf_tun100_pcap01

Packets are flowing correctly, but what’s this, my hyper secure network is sending them across the public network in the clear!:

Time for one more GRE feature…

tunnel protection

Instead of relying on another device to provide IPSec transport between sites or applying a crypto map to our routers outbound interface, we can apply a crypto policy directly to our GRE tunnel:

!
crypto keyring KEYRING vrf WAN
  pre-shared-key address 10.1.1.1 key SECURE_MY_TUNNEL
!
crypto isakmp policy 1
 encr aes 256
 authentication pre-share
!
crypto isakmp profile ISAKMP-PROFILE
   vrf WAN
   keyring KEYRING
   match identity address 10.1.1.1 WAN
!
!
crypto ipsec transform-set IPSEC-TRANS esp-aes esp-sha512-hmac
 mode tunnel
!
crypto ipsec profile IPSEC-GRE-PROFILE
 set transform-set IPSEC-TRANS
 set pfs group24
 set isakmp-profile ISAKMP-PROFILE
!
!
int tun100
 tunnel mode gre ip
 tunnel protection ipsec profile IPSEC-GRE-PROFILE
!

Lets check what is being sent over the public network:

tunnel_vrf_tun100_pcap02

Great, ESP packets are present, these will contain the GRE traffic passing between the RED VRFs, but we can still see the traffic on tunnel10 in the clear. Lets tidy that up.
If we apply the same IPSec profile to another tunnel interface then we must included the ‘shared’ keyword on all interfaces:

!
interface Tunnel10
 ip vrf forwarding WAN
 ip address 172.16.1.1 255.255.255.252
 tunnel source Loopback10
 tunnel destination 10.3.1.1
 tunnel key 10
 tunnel vrf WAN
 tunnel protection ipsec profile IPSEC-GRE-PROFILE shared
!
interface Tunnel100
 ip vrf forwarding RED
 ip address 172.16.100.1 255.255.255.252
 tunnel source Loopback10
 tunnel destination 10.3.1.1
 tunnel key 100
 tunnel vrf WAN
 tunnel protection ipsec profile IPSEC-GRE-PROFILE shared
!

It is worth noting that we cannot make these tunnels VTIs by using the tunnel mode ipsec ipv4 command as this document states sharing IPSec SAs is unsupported.

We can now confirm the status of both tunnels:

R1#sh int tunnel 10 | inc transport| protection
  Tunnel protocol/transport GRE/IP
  Tunnel transport MTU 1394 bytes
  Tunnel protection via IPSec (profile "IPSEC-GRE-PROFILE")
R1#sh int tunnel 100 | inc transport| protection
  Tunnel protocol/transport GRE/IP
  Tunnel transport MTU 1394 bytes
  Tunnel protection via IPSec (profile "IPSEC-GRE-PROFILE")
R1#
R1#sh crypto session
Crypto session current status

Interface: Tunnel100 Tunnel10
Session status: UP-NO-IKE
Peer: 10.3.1.1 port 500
  IPSEC FLOW: permit 47 host 10.1.1.1 host 10.3.1.1
        Active SAs: 2, origin: crypto map

Interface: FastEthernet0/0
Profile: ISAKMP-PROFILE-RED
Session status: UP-IDLE
Peer: 10.3.1.1 port 500
  IKEv1 SA: local 10.1.1.1/500 remote 10.3.1.1/500 Active
  IKEv1 SA: local 10.1.1.1/500 remote 10.3.1.1/500 Active

R1#
R1#sh ip route vrf WAN eigrp
      10.0.0.0/8 is variably subnetted, 5 subnets, 2 masks
D EX     10.3.1.2/32 [170/76800640] via 172.16.1.2, 00:23:46, Tunnel10
R1#
R1#sh ip route vrf RED eigrp
      10.0.0.0/32 is subnetted, 2 subnets
D EX     10.3.64.1 [170/76800640] via 172.16.100.2, 00:24:48, Tunnel100
tunnel_vrf_tun_all_pcap01

Checking the traffic being transmitted:

There you have it, highly scalable secure cross-site intra-VRF communication exposing the minimum of public IPs.

Implementation note

I had plenty of fun trying to get both tunnels running at the same time. More often than not one tunnel would report as UP/UP but no traffic would pass across it. ISAKMP debug would show the SA being removed once the Phase 1 negotiation had been completed. The only way I found of reliably bringing the tunnels up was to shut them down at both sites and bring the pairs up one at a time.

For reference I was using GNS3 and the c7200-adventerprisek9-mz.152-4.M11.bin image.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Blog at WordPress.com.

Up ↑

%d bloggers like this: