I had only intended to do 3 parts to this series, but I can’t really stop at part 3 (MPLS) without posting about how MPLS Layer 3 VPN affects MTU. In this post we’ll build a simple MPLS VPN network and our goal is to transmit 1500 bytes of IP data between the CEs.
MPLS VPN
With MPLS L3 VPNs there will be two labels. The top label being the transport label and the bottom label being the VPN label. The transport label will be swapped hop by hop through the MPLS network. The VPN label is used by the egress PE router to identify the VPN/VRF that the incoming packet relates to. Assuming PHP is being used, the transport label will be removed by the router before the egress PE (in this lab the “P” router), leaving only the VPN label in the stack.
Lab
For this lab, I’ll be using the topology below. The base configurations are using OSPF as the routing protocol and LDP to exchange transport labels. A full mesh of MP-BGP VPNv4 sessions will be configured between the PE routers to exchange VPN labels. Software revisions are as follows
- CE1, CE2, CE3, PE3: IOS (Cisco 7200 12.4(24)T)
- PE1: IOS-XR (IOS-XRv 5.1.1)
- PE2: Junos (Firefly 12.1X46)
As with previous posts in this series I’m going to show what needs to be done to enable the required MTU. I’ll give some commentary on the MPLS config but will save the detailed analysis of MPLS control/data plane for another time.
PE1
PE1 is running IOS-XR. The relevant parts of the base config are as below:
interface Loopback0 ipv4 address 1.1.1.1 255.255.255.255 ! interface GigabitEthernet0/0/0/0 ipv4 address 192.168.14.1 255.255.255.0 ! router ospf 1 area 0 interface Loopback0 passive enable ! interface GigabitEthernet0/0/0/0 network point-to-point ! ! ! router bgp 1 address-family vpnv4 unicast ! neighbor 2.2.2.2 remote-as 1 update-source Loopback0 address-family vpnv4 unicast ! ! neighbor 3.3.3.3 remote-as 1 update-source Loopback0 address-family vpnv4 unicast ! ! ! mpls ldp interface GigabitEthernet0/0/0/0 ! ! end
PE2
PE2 is running Junos on Firefly in packet mode.
interfaces { ge-0/0/0 { unit 0 { family inet { address 192.168.24.2/24; } family mpls; } } lo0 { unit 0 { family inet { address 2.2.2.2/32; } } } } routing-options { autonomous-system 1; } protocols { mpls { interface ge-0/0/0.0; } bgp { group ibgp { type internal; family inet-vpn { unicast; } peer-as 1; neighbor 1.1.1.1; neighbor 3.3.3.3; } } ospf { area 0.0.0.0 { interface ge-0/0/0.0 { interface-type p2p; } interface lo0.0 { passive; } } } ldp { interface ge-0/0/0.0; } } security { forwarding-options { family { mpls { mode packet-based; } } } }
PE3
PE3 is running IOS
interface Loopback0 ip address 3.3.3.3 255.255.255.255 ip ospf 1 area 0 ! interface GigabitEthernet0/0 ip address 192.168.34.3 255.255.255.0 ip ospf network point-to-point ip ospf 1 area 0 mpls ip ! router ospf 1 log-adjacency-changes ! router bgp 1 no bgp default ipv4-unicast bgp log-neighbor-changes neighbor 1.1.1.1 remote-as 1 neighbor 1.1.1.1 update-source Loopback0 neighbor 2.2.2.2 remote-as 1 neighbor 2.2.2.2 update-source Loopback0 ! address-family vpnv4 neighbor 1.1.1.1 activate neighbor 1.1.1.1 send-community extended neighbor 2.2.2.2 activate neighbor 2.2.2.2 send-community extended exit-address-family ! !
PE1-CE1 Routing
Let’s go ahead and enable the routing and VRF configuration between PE1 and CE1. We’ll use BGP as the PE-CE routing protocol.
CE1. There isn’t anything special about the configuration on the CE. BGP is used as the PE-CE routing protocol and is advertising the local interfaces to PE1.
interface Loopback0 ip address 11.11.11.11 255.255.255.255 ! interface FastEthernet0/0 ip address 192.168.11.1 255.255.255.0 ! router bgp 11 no bgp default ipv4-unicast bgp log-neighbor-changes neighbor 192.168.11.2 remote-as 1 ! address-family ipv4 neighbor 192.168.11.2 activate no auto-summary no synchronization network 11.11.11.11 mask 255.255.255.255 network 192.168.11.0 exit-address-family !
PE1. All the magic happens on the PE routers. Here the VRFs are defined and we add the BGP neighbor – note that the PE-CE BGP configuration is VRF based.
vrf vpn address-family ipv4 unicast import route-target 1:1 ! export route-target 1:1 ! ! ! interface GigabitEthernet0/0/0/1 vrf vpn ipv4 address 192.168.11.2 255.255.255.0 ! router bgp 1 vrf vpn rd 1.1.1.1:1 address-family ipv4 unicast ! neighbor 192.168.11.1 remote-as 11 address-family ipv4 unicast ! ! ! !
On XR, if I don’t configure an in/out export policy for the CE1 session then no routes will be sent/received. I’ve defined a simple policy to allow all routes to be sent/received.
route-policy announce pass end-policy ! router bgp 1 vrf vpn neighbor 192.168.11.1 address-family ipv4 unicast route-policy announce in route-policy announce out ! ! ! !
PE2-CE2 Routing
Here we’ll use OSPF as the PE-CE routing protocol.
PE2 is running Junos and most of the configuration all happens in a routing instance stanza.
IP addresses are added to the interface like normal:
interfaces { ge-0/0/1 { unit 0 { family inet { address 192.168.22.2/24; } } } }
The routing instance defines the VRF, the route-distinguisher and route-targets to import. All the OSPF configuration takes place here as well. I have an export policy configured so that the BGP VRF routes are exported in to OSPF and distributed onward to CE2.
routing-instances { vpn { instance-type vrf; interface ge-0/0/1.0; route-distinguisher 2.2.2.2:1; vrf-target { import target:1:1; export target:1:1; } protocols { ospf { export bgp2ospf; area 0.0.0.0 { interface ge-0/0/1.0 { interface-type p2p; } } } } } } policy-options { policy-statement bgp2ospf { from protocol bgp; then accept; } }
PE3-CE3 Routing
PE3 is running IOS. Again, OSPF is used as the PE-CE routing protocol. Mutual redistribution is configured between the VRF based BGP and OSPF protocols.
ip vrf vpn rd 3.3.3.3:1 route-target export 1:1 route-target import 1:1 ! interface FastEthernet1/0 ip vrf forwarding vpn ip address 192.168.33.3 255.255.255.0 ip ospf network point-to-point ip ospf 33 area 0 ! router ospf 33 vrf vpn log-adjacency-changes redistribute bgp 1 subnets ! router bgp 1 address-family ipv4 vrf vpn redistribute ospf 33 vrf vpn no synchronization exit-address-family !
VPN Testing
At this point we have an any to any MPLS VPN configured between the 3 CE routers. They should have full reachability, let’s see!
CE1#ping 22.22.22.22 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 22.22.22.22, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 40/46/56 ms CE1#ping 33.33.33.33 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 33.33.33.33, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 36/65/92 ms CE2#ping 33.33.33.33 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 33.33.33.33, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 56/60/68 ms CE2#ping 11.11.11.11 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 11.11.11.11, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 24/44/68 ms CE2# CE3#ping 11.11.11.11 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 11.11.11.11, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 52/60/72 ms CE3#ping 22.22.22.22 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 22.22.22.22, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 60/64/68 ms
Looks good!
But at this point the MTU settings on the PE-P-PE links are all set to the default of 1500 bytes, so a 1500 byte ping isn’t going to work.
CE1#ping 22.22.22.22 df-bit re 1 size 1500 Type escape sequence to abort. Sending 1, 1500-byte ICMP Echos to 22.22.22.22, timeout is 2 seconds: Packet sent with the DF bit set M Success rate is 0 percent (0/1) CE1# *Apr 26 17:25:27.099: ICMP: dst (192.168.11.1) frag. needed and DF set unreachable rcv from 192.168.14.1 CE1#ping 33.33.33.33 df-bit re 1 size 1500 Type escape sequence to abort. Sending 1, 1500-byte ICMP Echos to 33.33.33.33, timeout is 2 seconds: Packet sent with the DF bit set M Success rate is 0 percent (0/1) CE1# *Apr 26 17:25:33.855: ICMP: dst (192.168.11.1) frag. needed and DF set unreachable rcv from 192.168.14.1
OK, we are getting an ICMP unreachable back from PE1 exactly as expected.
We know that MPLS VPNs adds 2 labels to the stack, and therefore adds an extra 8 bytes of overhead, so all we need to do is increase the interface MTU to allow this extra data. From previous posts in this series we know how the different software does things.
On PE1, the IOS-XR router, the interface MTU will be increased to 1522 (1514 + 8), and the MPLS MTU will be set to 1508.
On P, the interface MTU will be increased to 1508 and the MPLS MTU will be set to 1508. The same config will be required on PE3 as both are running IOS.
On PE2, the Junos router, the interface MTU will be increased to 1522 (1514 + 8), and the MPLS MTU will be set to 1508.
Verification
CE1#ping 22.22.22.22 df-bit re 1 size 1500 Type escape sequence to abort. Sending 1, 1500-byte ICMP Echos to 22.22.22.22, timeout is 2 seconds: Packet sent with the DF bit set ! Success rate is 100 percent (1/1), round-trip min/avg/max = 24/24/24 ms CE1#ping 22.22.22.22 df-bit re 1 size 1500 *Apr 26 17:41:02.223: ICMP: echo reply rcvd, src 22.22.22.22, dst 192.168.11.1 CE1#ping 33.33.33.33 df-bit re 1 size 1500 Type escape sequence to abort. Sending 1, 1500-byte ICMP Echos to 33.33.33.33, timeout is 2 seconds: Packet sent with the DF bit set ! Success rate is 100 percent (1/1), round-trip min/avg/max = 52/52/52 ms CE1# *Apr 26 17:41:05.975: ICMP: echo reply rcvd, src 33.33.33.33, dst 192.168.11.1 CE2#ping 11.11.11.11 df-bit re 1 size 1500 Type escape sequence to abort. Sending 1, 1500-byte ICMP Echos to 11.11.11.11, timeout is 2 seconds: Packet sent with the DF bit set ! Success rate is 100 percent (1/1), round-trip min/avg/max = 20/20/20 ms CE2#ping 33.33.33.33 df-bit re 1 size 1500 Type escape sequence to abort. Sending 1, 1500-byte ICMP Echos to 33.33.33.33, timeout is 2 seconds: Packet sent with the DF bit set ! Success rate is 100 percent (1/1), round-trip min/avg/max = 4/4/4 ms CE3#ping 11.11.11.11 df-bit re 1 size 1500 Type escape sequence to abort. Sending 1, 1500-byte ICMP Echos to 11.11.11.11, timeout is 2 seconds: Packet sent with the DF bit set ! Success rate is 100 percent (1/1), round-trip min/avg/max = 44/44/44 ms CE3#ping 22.22.22.22 df-bit re 1 size 1500 Type escape sequence to abort. Sending 1, 1500-byte ICMP Echos to 22.22.22.22, timeout is 2 seconds: Packet sent with the DF bit set ! Success rate is 100 percent (1/1), round-trip min/avg/max = 56/56/56 ms CE3#traceroute 11.11.11.11 Type escape sequence to abort. Tracing the route to 11.11.11.11 1 192.168.33.3 20 msec 16 msec 8 msec 2 192.168.34.4 [MPLS: Labels 18/16005 Exp 0] 44 msec 8 msec 36 msec 3 192.168.14.1 [MPLS: Label 16005 Exp 0] 60 msec 8 msec 60 msec 4 192.168.11.1 8 msec 64 msec 8 msec CE3#traceroute 22.22.22.22 Type escape sequence to abort. Tracing the route to 22.22.22.22 1 192.168.33.3 36 msec 28 msec 4 msec 2 192.168.34.4 [MPLS: Labels 17/299888 Exp 0] 12 msec 56 msec 12 msec 3 192.168.24.2 [MPLS: Label 299888 Exp 0] 28 msec 28 msec 28 msec 4 192.168.22.1 60 msec 12 msec 32 msec CE1#traceroute 33.33.33.33 Type escape sequence to abort. Tracing the route to 33.33.33.33 1 192.168.11.2 8 msec 8 msec 8 msec 2 192.168.14.4 [MPLS: Labels 16/20 Exp 0] 40 msec 16 msec 36 msec 3 192.168.33.3 [AS 1] [MPLS: Label 20 Exp 0] 20 msec 16 msec 16 msec 4 192.168.33.1 [AS 1] 48 msec 44 msec 40 msec
Success!
In this simple lab we configured an any to any MPLS VPN and verified the settings required for a 1500 bytes of payload to be transmitted across our MPLS VPN network.
In a later post I will come back to this topology and go through the MPLS configuration in more detail.
There will also be a part 5 which will cover MPLS L2 VPN (pseudowire).