I’m working on a Juniper rollout right now, and this network will need to interop with routers and switches from other vendors such as Cisco.
A mismatched MTU could result in something simple like an OSPF adjacency not forming, or cause layer-2 issues such as dropped frames. This post outlines the MTU configuration differences on Cisco IOS/XE/XR and Juniper Junos.
Parts 2 and 3 will cover Jumbo Frames/IP MTU and MPLS MTU/802.1q tagged interfaces.
Let’s remind ourselves of what an Ethernet frame looks like.
This is a standard “1500 byte” frame and consists of the following fields:
- Destination address: The 6 byte destination MAC address
- Source address: The 6 byte source MAC address
- Ethertype: A 2 byte field that identifies the upper layer protocol
- Data: a variable length field of length 46-1500 bytes
- FCS: The 4 byte checksum (frame check sequence / CRC)
You can see here that an Ethernet frame containing 1500 bytes of data actually places 1514 bytes on the wire, plus the CRC. So does that mean that the configured MTU is 1500 or 1514 or 1518? Is there a difference between the interface MTU and the protocol MTU? Let’s see!
Software revisions are as follows
- IOS (Cisco 7200 12.4(24)T)
- IOS-XE (CSR1000V 15.4(1)S)
- IOS-XR (IOS-XRv 5.1.1)
- Junos (12.3R5.7)
- Junos (Firefly 12.1X46)
OK, so lets start with IOS. The default physical MTU is set to 1500, but this does not include the 14-bytes of Layer 2 frame overhead. Note, the 4-byte FCS is never included in MTU calculations. The IP MTU is also set to 1500.
IOS#show interfaces GigabitEthernet 0/0 | i MTU MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec, IOS#show ip interface GigabitEthernet 0/0 | i MTU MTU is 1500 bytes
Let’s see what this router places on the wire.
IOS#ping 192.168.1.2 repeat 1 size 1500 Type escape sequence to abort. Sending 1, 1500-byte ICMP Echos to 192.168.1.2, timeout is 2 seconds: ! Success rate is 100 percent (1/1), round-trip min/avg/max = 16/16/16 ms
IOS includes the IP and ICMP headers in the size, as well as ICMP data. If we look at what’s been placed on the wire with a 1500 byte ping we should expect to see 1514 bytes:
- 1472 bytes data
- 8 bytes of ICMP header
- 20 bytes of IP header
- 14 bytes of Ethernet headers
Great – that’s exactly what we expected to see. We’ve confirmed that the MTU setting on IOS does not include the L2 headers and that setting the hardware MTU to 1500 on IOS enables 1500 bytes of protocol data.
As with IOS, the Interface MTU and IP MTU is showing as 1500
1000v#show interfaces GigabitEthernet 2 | i MTU MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec, 1000v#show ip interface GigabitEthernet 2 | i MTU MTU is 1500 bytes
Let’s see what this router places on the wire. Again, a ping of size 1500 so as with IOS we expect to see expect to see 1514 bytes on the wire, which we do
This is where things start to get a little different. The hardware MTU is configured including the L2 headers (not the FCS). Therefore if we want to allow 1500 bytes of protocol data, we must set the MTU to 1514.
RP/0/0/CPU0:ios#show interfaces GigabitEthernet 0/0/0/0 | i MTU Tue Apr 1 14:47:55.381 UTC MTU 1514 bytes, BW 1000000 Kbit (Max: 1000000 Kbit) RP/0/0/CPU0:ios#show ipv4 interface GigabitEthernet 0/0/0/0 | i MTU Tue Apr 1 14:48:00.090 UTC MTU is 1514 (1500 is available to IP)
In the second command, IOS-XR tells us that we have an interface MTU of 1514, with 1500 bytes available to IP.
Now for the ping
RP/0/0/CPU0:ios#ping 192.168.1.1 count 1 donotfrag size 1500 Type escape sequence to abort. Sending 1, 1500-byte ICMP Echos to 192.168.1.1, timeout is 2 seconds: ! Success rate is 100 percent (1/1), round-trip min/avg/max = 19/19/19 ms
The maximum ping achievable was with size 1500. We can derive from this that the ping “size” is not including the L2 headers, but does include the IP and ICMP headers. The packet capture should show 1514 bytes on the wire, which it does.
On Junos the MTU includes the Layer 2 headers, again without the FCS. Therefore if we want to allow 1500 bytes of protocol data, we must set the MTU to 1514.
The protocol MTU is derived from the interface MTU if it is not explicitely configured. Below we can see the interfaces MTU set to 1514 and the IP MTU set to 1500.
root@junos> show interfaces em0 | match MTU Type: Ethernet, Link-level type: Ethernet, MTU: 1514 Protocol inet, MTU: 1500
What happens with a ping size 1500?
root@junos> ping 192.168.1.1 rapid count 1 do-not-fragment size 1500 PING 192.168.1.1 (192.168.1.1): 1500 data bytes ping: sendto: Message too long . --- 192.168.1.1 ping statistics --- 1 packets transmitted, 0 packets received, 100% packet loss
Well that didn’t work. The 1500 on Junos is specifying the size of ICMP data only, not the data and headers. With that in mind, we’d expect a ping of size 1472 (1500 – 20 – 8) to be the maximum that will work. Let’s see.
root@junos> ping 192.168.1.1 rapid count 1 do-not-fragment size 1472 PING 192.168.1.1 (192.168.1.1): 1472 data bytes ! --- 192.168.1.1 ping statistics --- 1 packets transmitted, 1 packets received, 0% packet loss round-trip min/avg/max/stddev = 11.595/11.595/11.595/nan ms
root@junos> ping 192.168.1.1 rapid count 1 do-not-fragment size 1473 PING 192.168.1.1 (192.168.1.1): 1473 data bytes ping: sendto: Message too long . --- 192.168.1.1 ping statistics --- 1 packets transmitted, 0 packets received, 100% packet loss
Just as expected. To double check the interface MTU setting of 1514, let’s check how many bytes the 1472 ping put on the wire – it should be 1514, which it is.
We’d expect Firefly to be exactly the same – the Interface MTU should be including the L2 headers and therefore set to 1514 to allow 1500 bytes of IP data.
root@firefly> show interfaces ge-0/0/0 | match mtu Link-level type: Ethernet, MTU: 1514, Link-mode: Full-duplex, Speed: 1000mbps, Protocol inet, MTU: 1500
Yep, this is exactly as expected, now for the ping. Again, 1472 should be the max.
root@firefly> ping 192.168.1.1 count 1 rapid do-not-fragment size 1472 PING 192.168.1.1 (192.168.1.1): 1472 data bytes ! --- 192.168.1.1 ping statistics --- 1 packets transmitted, 1 packets received, 0% packet loss round-trip min/avg/max/stddev = 11.176/11.176/11.176/0.000 ms
In summary, on IOS and IOS-XE, the interface MTU does not include the 14 bytes of Layer-2 headers. The configured value represents the maximum size of the encapsulated data.
For IOS-XR and Junos, the Interface value must include both the data and 14 bytes of headers.
The 4 byte FCS is not included.