Tag Archives: OSPF

Mapping traffic to an LSP on Junos – BGP and table inet.3 (part 2)

Now that we know about IGP based LSP forwarding on Junos, the 2nd part in this series focuses on BGP and table inet.3. 

We also continue from where part 1 left off, by looking at how traffic-engineering bgp-igp and mpls-forwarding can affect route redistribution from OSPF into BGP.

Lab Topology

For this lab, I’ll be using the topology below.

lsplab

Software revisions are as follows

  • CE Routers (CE1, CE2): IOS (Cisco 7200 12.4(24)T)
  • P Routers (R1, R2, R3, R4, R5): IOS (Cisco 7200 12.4(24)T)
  • PE Routers (Junos1, Junos2): Junos (Olive 12.3R5.7)

As with Part 1, the base configurations are using OSPF as the routing protocol and LDP to exchange transport labels.

Route redistribution (bgp-igp and mpls-forwarding)

Here’s what I changed on Junos 1. The OSPF route 102.102.102.102 learnt via OSPF from CE2 will be redistributed in to BGP.

By the way, I’m not suggesting that your CEs should be part of your core IGP, but for the purposes on this lab test… 🙂

root@R6-Junos1# show | compare
[edit protocols bgp group internal]
+    export ospf2bgp;
[edit policy-options]
+   policy-statement ospf2bgp {
+       from {
+           protocol ospf;
+           route-filter 102.102.102.102/32 exact;
+       }
+       then accept;
+   }

The configuration on Junos1 is still running with “traffic-engineering mpls-forwarding” so the routing table has OSPF as the active route for routing, and the LDP route is active for forwarding

root@R6-Junos1> show route 102.102.102.102

inet.0: 21 destinations, 35 routes (21 active, 0 holddown, 0 hidden)
@ = Routing Use Only, # = Forwarding Use Only
+ = Active Route, - = Last Active, * = Both

102.102.102.102/32 @[OSPF/150] 00:00:28, metric 0, tag 0
                    > to 192.168.46.4 via em0.0
                   #[LDP/9] 00:00:28, metric 1
                    > to 192.168.46.4 via em0.0, Push 28

Hence you would definitely expect the routing policy to match on the OSPF route 102.102.102.102 and therefore we’ll see the route in BGP right? Sure enough if I hop over to Junos2, the route is there:

root@R7-Junos2> show route receive-protocol bgp 6.6.6.6

inet.0: 22 destinations, 23 routes (22 active, 0 holddown, 0 hidden)
  Prefix                  Nexthop              MED     Lclpref    AS path
  102.102.102.102/32      192.168.46.4         0       100        I

OK show what happens if I change over to traffic-engineering bgp-igp  on Junos1? The LDP route becomes the active route for routing and forwarding, and isn’t matched by my policy, so is not advertised to Junos2.

inet.0: 22 destinations, 36 routes (22 active, 0 holddown, 0 hidden)
@ = Routing Use Only, # = Forwarding Use Only
+ = Active Route, - = Last Active, * = Both

102.102.102.102/32 *[LDP/9] 00:00:04, metric 1
                    > to 192.168.46.4 via em0.0, Push 28
                    [OSPF/150] 00:04:54, metric 0, tag 0
                    > to 192.168.46.4 via em0.0

root@R7-Junos2> show route receive-protocol bgp 6.6.6.6

inet.0: 22 destinations, 22 routes (22 active, 0 holddown, 0 hidden)

inet.3: 14 destinations, 14 routes (14 active, 0 holddown, 0 hidden)

BGP (LSP forwarding) and table inet.3

We’ll now have a look at how BGP operates. I’ve set the “traffic-engineering” on Junos1 back to the defaults. We should expect BGP to recursively resolve it’s next-hop via inet.3 and therefore MPLS route the traffic. Let’s see!

iBGP Peering

There is an iBGP peering session between Junos1 and Junos2. No other routers are running iBGP

eBGP Peering

Junos2 has an eBGP peering with CE1. CE has a second Loopback 112.112.112.112 being advertised via this eBGP session.

root@R7-Junos2> show configuration protocols bgp
group as102 {
    peer-as 102;
    neighbor 192.168.102.1;
}
group internal {
    local-address 7.7.7.7;
    peer-as 1;
    neighbor 6.6.6.6;
}

root@R7-Junos2> show route receive-protocol bgp 192.168.102.1

inet.0: 22 destinations, 22 routes (22 active, 0 holddown, 0 hidden)
  Prefix                  Nexthop              MED     Lclpref    AS path
* 112.112.112.112/32      192.168.102.1        0                  102 I

LSP Fowarding and Routing

So how does Junos1 route to CE2s IP address 112.112.112.112? Let’s take a look at the routing tables.

root@R6-Junos1> show route 112.112.112.112

inet.0: 22 destinations, 22 routes (22 active, 0 holddown, 0 hidden)
@ = Routing Use Only, # = Forwarding Use Only
+ = Active Route, - = Last Active, * = Both

112.112.112.112/32 *[BGP/170] 00:05:00, MED 0, localpref 100, from 7.7.7.7
                      AS path: 102 I, validation-state: unverified
                    > to 192.168.46.4 via em0.0, Push 27

OK, so we see 112.112.112.112/32 in table inet.0 as expected, and it looks like label 27 is going to be pushed. Let’s take a look at this in more detail:

inet.0: 22 destinations, 22 routes (22 active, 0 holddown, 0 hidden)
112.112.112.112/32 (1 entry, 1 announced)
TSI:
KRT in-kernel 112.112.112.112/32 -> {indirect(131070)}
        *BGP    Preference: 170/-101
                Next hop type: Indirect
                Address: 0x9378ba4
                Next-hop reference count: 3
                Source: 7.7.7.7
                Next hop type: Router, Next hop index: 561
                Next hop: 192.168.46.4 via em0.0, selected
                Label operation: Push 27
                Label TTL action: prop-ttl
                Session Id: 0x1
                Protocol next hop: 192.168.102.1
                Indirect next hop: 93b8000 131070 INH Session ID: 0x2
                State: 
                Local AS:     1 Peer AS:     1
                Age: 5:41       Metric: 0       Metric2: 1
                Validation State: unverified
                Task: BGP_1.7.7.7.7+179
                Announcement bits (2): 0-KRT 6-Resolve tree 2
                AS path: 102 I
                Accepted
                Localpref: 100
                Router ID: 7.7.7.7
                Indirect next hops: 1
                        Protocol next hop: 192.168.102.1 Metric: 1
                        Indirect next hop: 93b8000 131070 INH Session ID: 0x2
                        Indirect path forwarding next hops: 1
                                Next hop type: Router
                                Next hop: 192.168.46.4 via em0.0
                                Session Id: 0x1
                        192.168.102.0/24 Originating RIB: inet.3
                          Metric: 1                       Node path count: 1
                          Forwarding nexthops: 1
                                Nexthop: 192.168.46.4 via em0.0

The key here is the protocol next hop – 192.168.102.1.

192.168.102.1 isn’t directly attached to Junos1 – it is CE2s address on the Junos2<->CE2 segment, Therefore BGP will recursively resolve this next hop via table inet.3 and inet.0. As the inet.3 LDP route has a lower preference compared to the inet.0 OSPF route, the inet.3 route will be chosen and traffic will be placed on the LSP automatically, pushing label 27 in this case.

root@R6-Junos1> show route 192.168.102.1

inet.0: 22 destinations, 22 routes (22 active, 0 holddown, 0 hidden)
@ = Routing Use Only, # = Forwarding Use Only
+ = Active Route, - = Last Active, * = Both

192.168.102.0/24   *[OSPF/10] 00:06:30, metric 5
                    > to 192.168.46.4 via em0.0

inet.3: 14 destinations, 14 routes (14 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.102.0/24   *[LDP/9] 00:06:30, metric 1
                    > to 192.168.46.4 via em0.0, Push 27

root@R6-Junos1> show route forwarding-table destination 112.112.112.112
Routing table: default.inet
Internet:
Destination        Type RtRef Next hop           Type Index NhRef Netif
112.112.112.112/32 user     0                    indr 131070     2
                              192.168.46.4      Push 27   561     2 em0.0

root@R6-Junos1> traceroute 112.112.112.112
traceroute to 112.112.112.112 (112.112.112.112), 30 hops max, 40 byte packets
 1  192.168.46.4 (192.168.46.4)  28.026 ms  27.884 ms  28.510 ms
     MPLS Label=27 CoS=0 TTL=1 S=1
 2  192.168.34.3 (192.168.34.3)  29.767 ms  25.848 ms  28.571 ms
     MPLS Label=28 CoS=0 TTL=1 S=1
 3  192.168.35.5 (192.168.35.5)  29.831 ms  26.455 ms  28.586 ms
     MPLS Label=27 CoS=0 TTL=1 S=1
 4  192.168.57.7 (192.168.57.7)  29.478 ms  25.518 ms  29.075 ms
 5  192.168.102.1 (192.168.102.1)  32.961 ms  31.147 ms  33.398 ms

Traffic is labelled!

But what about IGP traffic to the protocol next hop? Well that won’t follow the LSP of course because we don’t have “mpls traffic-engineering” configured.

root@R6-Junos1> show route forwarding-table destination 192.168.102.1
Routing table: default.inet
Internet:
Destination        Type RtRef Next hop           Type Index NhRef Netif
192.168.102.0/24   user     0 192.168.46.4       ucst   555    32 em0.0

Exactly as expected!

I’ve shown that BGP is using table inet.3 to resolve next hops, where as normal IGP routing is using inet.0.

Another thing to remember with BGP & inet.3… if inet.0 contains a better route (e.g. better preference) then BGP would use the inet.0 route and traffic would not be forwarded on the LSP.

In this case, as none of the P routers are running BGP, this would break the connectivity (the P routers don’t know how to get to 112.112.112.112 so would drop the traffic). Hence, the traffic has to follow the LSP for the traffic to reach CE2.

MTU settings on Junos & IOS (part 2) with OSPF

Part 1 of this series focussed on the interface MTU configuration, looking how different vendors implement the setting. Some include the layer 2 headers, some don’t. Part 3 will add MPLS MTU to the mix.

This post looks at Jumbo Frames, IP MTU and we’ll also introduce OSPF to the mix. OSPF exchanges IP MTU information in DBD packets when forming a neighbor adjacency and will detect any MTU mismatch, so if the MTU settings are wrong or mismatched, we won’t fully establish the adjacency. Of course, this feature can be turned off, but following the OSPF Version 2 specification (RFC 2328):

If the Interface MTU field in the Database Description packet
indicates an IP datagram size that is larger than the router can
accept on the receiving interface without fragmentation, the
Database Description packet is rejected.

Jumbo Frames

Why would anyone actually want to increase the MTU size beyond 1500? Well, back in the day, larger packets were desirable because they resulted in less overhead on the server – fewer CPU interrupts, fewer CPU cycles wasted etc. Today with Large Segment Offload (LSO) etc the performance increase might not be what you’d expect. Take a look here for more info. As always – implement something that is suitable for your environment, and test before you do.  Anyway, you are not here to debate the use of Jumbo frames or otherwise, so let’s crack on.

IP MTU

As with Ethernet frames, the protocol MTU can be changed for IP packets. The accepted “standard” payload for a Jumbo Frame is 9000 bytes (i.e. an IP MTU set to 9000).

In this post, the interface MTU will be increased to the maximum supported by the interface hardware, but for the purposes of this post and to demonstrate that the interface MTU and IP MTU can be different, we will set the IP MTU to a consistent value of 2000 bytes.

Topology

The same virtual topology will be used as Part 1.

IMG_0098

Software revisions are as follows

  • IOS (Cisco 7200 12.4(24)T)
  • IOS-XE (CSR1000V 15.4(1)S)
  • IOS-XR (IOS-XRv 5.1.1)
  • Junos (12.3R5.7)
  • Junos (Firefly 12.1X46)

IOS/IOS-XE

Our Interface config is below, the interface MTU has been changed to 9216, and the protocol MTU to 2000. From our tests earlier, we know what this means that the maximum IP payload is 2000 bytes, which would result in 2014 bytes being put on the wire including the L2 headers.

interface GigabitEthernet2
 mtu 9216
 ip address 192.168.1.4 255.255.255.0
 ip mtu 2000
 negotiation auto
 cdp enable
end

OK so let’s go ahead and enable OSPF, nothing much to see here

interface Loopback0
 ip address 4.4.4.4 255.255.255.255
 !
 router ospf 1
 network 4.4.4.4 0.0.0.0 area 0
 network 192.168.1.4 0.0.0.0 area 0
 !

I already enabled OSPF on another router, but didn’t change the MTU from 1500. We’re not going to get a full adjacency here but let’s troubleshoot

1000v#show ip ospf neighbor
 Neighbor ID Pri State Dead Time Address Interface
 192.168.1.3 1 INIT/DROTHER 00:00:35 192.168.1.3 GigabitEthernet2
1000v#debug ip ospf adj
 OSPF adjacency debugging is on
 1000v#
 *Apr 3 18:31:08.455: OSPF-1 ADJ Gi2: Neighbor change event
 *Apr 3 18:31:08.455: OSPF-1 ADJ Gi2: DR/BDR election
 *Apr 3 18:31:08.455: OSPF-1 ADJ Gi2: Elect BDR 192.168.1.3
 *Apr 3 18:31:08.455: OSPF-1 ADJ Gi2: Elect DR 192.168.1.4
 *Apr 3 18:31:08.455: OSPF-1 ADJ Gi2: DR: 192.168.1.4 (Id) BDR: 192.168.1.3 (Id)
 *Apr 3 18:31:08.455: OSPF-1 ADJ Gi2: Neighbor change event
 *Apr 3 18:31:08.455: OSPF-1 ADJ Gi2: DR/BDR election
 *Apr 3 18:31:08.455: OSPF-1 ADJ Gi2: Elect BDR 192.168.1.3
 *Apr 3 18:31:08.455: OSPF-1 ADJ Gi2: Elect DR 192.168.1.4
 *Apr 3 18:31:08.455: OSPF-1 ADJ Gi2: DR: 192.168.1.4 (Id) BDR: 192.168.1.3 (Id)
 *Apr 3 18:31:08.484: OSPF-1 ADJ Gi2: Send DBD to 192.168.1.3 seq 0x341 opt 0x52 flag 0x7 len 32
 *Apr 3 18:31:08.484: OSPF-1 ADJ Gi2: Retransmitting DBD to 192.168.1.3 [2]
 *Apr 3 18:31:09.102: OSPF-1 ADJ Gi2: Rcv DBD from 192.168.1.3 seq 0x26B1 opt 0x52 flag 0x7 len 32 mtu 1500 state EXSTART
 *Apr 3 18:31:09.102: OSPF-1 ADJ Gi2: Nbr 192.168.1.3 has smaller interface MTU

Well that’s pretty clear why there is a problem. Let’s move on to IOS-XR.

IOS-XR

Let’s see what IOS-XR has to say about OSPF & MTU

RP/0/0/CPU0:ios#show ospf neighbor
* Indicates MADJ interface

Neighbors for OSPF 1

Neighbor ID     Pri   State           Dead Time   Address         Interface
4.4.4.4         1     EXSTART/BDR     00:00:38    192.168.1.4     GigabitEthernet0/0/0/0
    Neighbor is up for 00:00:06

RP/0/0/CPU0:ios#show ospf trace errors
11   Apr  3 18:33:41.776* ospf_rcv_dbd: WARN nbr 4.4.4.4 larger MTU dbd_if_mtu 2000 oi_ip_mtu 1500

Pretty clear that there is an MTU problem. Below I’ve set the Interface MTU to 9000 – remember IOS-XR includes the L2 headers in the Interface MTU, so the maximum encapsulated data on this wire would be 8986.

UPDATE 23/12/15:  I’m setting the interface MTU to a different value to the IOS router’s MTU setting,  to show that it’s the IP MTU that is the important setting and must match for two OSPF routers to establish an adjacency.

The IP MTU is set to 2000 to match the other router.

interface Loopback0
 ipv4 address 3.3.3.3 255.255.255.255
!
interface GigabitEthernet0/0/0/0
 mtu 9000
 ipv4 mtu 2000
 ipv4 address 192.168.1.3 255.255.255.0
!
router ospf 1
 area 0
 interface Loopback0
 !
 interface GigabitEthernet0/0/0/0
 !
 !
!
end

OSPF has established with the 1000v now the IPv4 MTU has been changed to 2000, and we can see the route the 1000v’s loopback interface 4.4.4.4

Notice that for OSPF to be happy, it only matters that the IP MTU is the same on both routers, the physical MTU can be different.

RP/0/0/CPU0:ios#show ospf neighbor
Thu Apr 3 21:07:09.053 UTC
* Indicates MADJ interface
Neighbors for OSPF 1
Neighbor ID Pri State Dead Time Address Interface
4.4.4.4 1 FULL/DR 00:00:31 192.168.1.4 GigabitEthernet0/0/0/0
 Neighbor is up for 01:13:55
Total neighbor count: 1

RP/0/0/CPU0:ios#show route 4.4.4.4
Thu Apr 3 21:07:29.291 UTC
Routing entry for 4.4.4.4/32
 Known via "ospf 1", distance 110, metric 2, type intra area
 Installed Apr 3 19:53:14.416 for 01:14:14
 Routing Descriptor Blocks
 192.168.1.4, from 4.4.4.4, via GigabitEthernet0/0/0/0
 Route metric is 2
 No advertising protos.

RP/0/0/CPU0:ios#ping 4.4.4.4 donotfrag size 2000 co 1
Thu Apr 3 21:07:36.781 UTC
Type escape sequence to abort.
Sending 1, 2000-byte ICMP Echos to 4.4.4.4, timeout is 2 seconds:
!
Success rate is 100 percent (1/1), round-trip min/avg/max = 1/1/1 ms
RP/0/0/CPU0:ios#

Junos

Finally we’ll configure the Junos router and make sure that we are able to ping with 2000 bytes of protocol data to the 1000v and XRv.

The Junos configuration is as follows, I’ve not added the MTU settings yet.

interfaces {
    }
    ge-0/0/1 {
        unit 0 {
            family inet {
                address 192.168.1.5/24;
            }
        }
    }
    lo0 {
        unit 0 {
            family inet {
                address 5.5.5.5/32;
            }
        }
    }
}
protocols {
    ospf {
        area 0.0.0.0 {
            interface lo0.0 {
                passive;
            }
            interface ge-0/0/1.0;
        }
    }
}

The neigbor isn’t going to establish so let’s add some traceoptions to double check what the problem is.

root@firefly> show configuration protocols
ospf {
    traceoptions {
        file ospf-log;
        flag error;
    }
}
root@firefly> show log ospf-log
Apr 3 21:39:00 firefly clear-log[1261]: logfile cleared
Apr  3 21:39:01.701093 OSPF packet ignored: MTU mismatch from 192.168.1.3 on intf ge-0/0/1.0 area 0.0.0.0
Apr  3 21:39:04.794783 OSPF packet ignored: MTU mismatch from 192.168.1.4 on intf ge-0/0/1.0 area 0.0.0.0

OK, that’s pretty clear. Let’s fix the MTU and IP MTU.

root@firefly# show | compare
[edit interfaces ge-0/0/1]
+   mtu 9192;
[edit interfaces ge-0/0/1 unit 0 family inet]
+      mtu 2000;

Notice that the IP MTU is configured under family inet.

root@firefly> show interfaces ge-0/0/1 | match "ge-|MTU:"
Physical interface: ge-0/0/1, Enabled, Physical link is Up
  Link-level type: Ethernet, MTU: 9192, Link-mode: Full-duplex, Speed: 1000mbps,
  Logical interface ge-0/0/1.0 (Index 74) (SNMP ifIndex 519)
    Protocol inet, MTU: 2000
root@firefly> show ospf neighbor
Address          Interface              State     ID               Pri  Dead
192.168.1.4      ge-0/0/1.0             Full      4.4.4.4            1    38
192.168.1.3      ge-0/0/1.0             Full      3.3.3.3            1    37

root@firefly> show route protocol ospf

inet.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

3.3.3.3/32         *[OSPF/10] 00:02:15, metric 2
                    > to 192.168.1.3 via ge-0/0/1.0
4.4.4.4/32         *[OSPF/10] 00:02:15, metric 2
                    > to 192.168.1.4 via ge-0/0/1.0

All good, we can see routes to 3.3.3.3 and 4.4.4.4

Now for the ping, remember that the Junos ping size excludes the ICMP (8 bytes) and IP (20 bytes) headers , so we’ll be expecting the maximum working ping size to be 1972 bytes, for 2000 bytes of protocol data and 2014 bytes on the wire.

root@firefly> ping rapid count 1 do-not-fragment size 1972 3.3.3.3
PING 3.3.3.3 (3.3.3.3): 1972 data bytes
!
--- 3.3.3.3 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max/stddev = 3.293/3.293/3.293/0.000 ms

root@firefly> ping rapid count 1 do-not-fragment size 1972 4.4.4.4
PING 4.4.4.4 (4.4.4.4): 1972 data bytes
!
--- 4.4.4.4 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.961/0.961/0.961/0.000 ms

root@firefly> ping rapid count 1 do-not-fragment size 1973 3.3.3.3
PING 3.3.3.3 (3.3.3.3): 1973 data bytes
ping: sendto: Message too long
.
--- 3.3.3.3 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss

NX-OS

I don’t have a Nexus in my lab, but for completeness below is the config for updating the MTU on NX-OS.

On the 7k the system will be enabled for Jumbo by default, if you need to change this value it’s done like this

switch(config)#system jumbomtu 9216

Make sure you check the Interfaces and Vlan interfaces have the correct MTU. That’s done with the interface command “mtu X”. For Layer 2 interfaces, configure either the default MTU size (1500 bytes) or up to the system jumbo MTU size.

On the 5k it’s done a bit differently – in a QoS policy map! For NX-OS >4.1

switch(config)#policy-map type network-qos jumbo
switch(config-pmap-nq)#class type network-qos class-default
switch(config-pmap-c-nq)#mtu 9216
switch(config-pmap-c-nq)#exit
switch(config-pmap-nq)#exit
switch(config)#system qos
switch(config-sys-qos)#service-policy type network-qos jumbo

 Summary

In this post I’ve shown how the physical MTU can vary from the IP MTU, and how it’s important to have the same IP MTU when working with OSPF.

Also discussed were debugging steps to troubleshoot OSPF MTU issues on IOS and Junos.

Gotta love Junos for keeping things consistent!