MTU and MSS are two very important terms we often forget about and not really know what they are. I myself learnt them many time and forgot them later. Knowledge of MTU and MSS comes in handy in different situations. I’ve seen two scenario when MTU and TCP-MSS size was the issue:
- spalsh page won’t come up for a wifi AP
- subdomain for booking site won’t come up
both the time MTU and TCP-MSS was a cause and it took a while to find this out. So here is some basic of MTU and TCP-MSS.
MTU
A maximum transmission unit (MTU) is the largest packet or frame size, specified in octets (eight-bit bytes) that can be sent in a packet- or frame-based network such as the internet. The internet’s transmission control protocol (TCP) uses the MTU to determine the maximum size of each packet in any transmission. MTU is usually associated with the Ethernet protocol, where a 1500-byte packet is the largest allowed in it (and hence over most of the internet).
One of the most common problems related to MTU is that sometimes higher-level protocols may create packets larger than a particular link supports, and you’ll need to make adjustments to make it work.
To get around this issue, IPv4 allows fragmentation which divides the datagram into pieces. Each piece is small enough to pass over the single link that it is being fragmented for, using the MTU parameter configured for that interface. This fragmentation process takes place at the IP layer (OSI layer 3) and marks the packets it fragments as such. This ensures the IP layer of the destination host knows it should reassemble the packets into the original datagram.
Fragmentation is sometimes not supported by applications, and is something we should avoid if possible. The best way to avoid fragmentation is to adjust the maximum segment size or TCP MSS so the segment will adjust its size before reaching the data link layer.
Before we look at TCP MSS, it helps to understand the build of the “unit” that’s being sent over the internet.
As mentioned, the common value of MTU in the internet is 1500 bytes.
As you can see in the figure above, the MTU is built from payload (also referred as data) and the TCP and the IP header, 20 bytes each. The total value of the IP and the TCP header is 40 bytes and mandatory for each packet, which leaves us 1460 bytes for our data.
Now, imagine that we are using the GRE protocol in our network, encapsulating the original packet and adding 24 bytes for the GRE header.
The total size of this kind of packet will be 1524 bytes, exceeding the 1500 bytes MTU value. The “data” size in this packet is 1460, but we can and should decrease it in order to make sure the total size will be 1500 bytes or less. And this is where TCP MSS comes into the picture.
TCP MSS, the maximum segment size, is a parameter of the options field of the TCP header that specifies the largest amount of data, specified in bytes, that a computer or communications device can receive in a single TCP segment. It does not include the TCP header or the IP header. This value will dictate the maximum size of the “data” part of the packet. In the following case for the GRE tunnel, we will set the tcp mss value to be 1436 or lower, while the default size is 1460.
The MSS announcement (often mistakenly called a negotiation) is sent during the three-way handshake by both sides, saying: “I can accept TCP segments up to size x”. The size (x) may be larger or smaller than the default. The MSS can be used completely independently in each direction of data flow.
Since the end device will not always know about high level protocols that will be added to this packet along the way, like GRE packets for example, it won’t usually adjust the TCP MSS value. As a result the network devices have the option to rewrite the value of TCP MSS packets that are processed through them. For example, in a Cisco Router the command “ip tcp mss-adjust 1436” in the interface level will rewrite the value of the TCP MSS of any SYN packet that will go via this interface.
Another likely case where a router in transit is carrying out additional encapsulation, that is, MPLS label swapping, this will add an additional label header as below:
This will eventually increase the size of the frame exiting a transiting router (in the case above it is 1508 bytes.) It will also create similar issue for IPSec and IPv6 in IPv4 tunnel etc.
The maximum MTU of an interface will depend on the hardware platform, but the IEEE 802.3 standards require a minimum MTU of 1500 bytes. Also if you notice the following CLI on a router, the maximum IP MTU is capped at the Ethernet MTU of 1500 bytes.
The problem is we cannot increase the IP MTU size on the router Ethernet interface because the MPLS label encapsulation frame size can potentially exceed the maximum MTU. If the source device creates a full size packet with a TCP MSS of 1460 bytes, it is likely this transiting router will drop/fragment the packet. This is bad for our network performance.
What is the best way to solve this?
Shrinking the IP MTU on the interface to 1448 bytes will create space for 12 additional bytes on the MPLS label header [if you consider a maximum of 3 MPLS label LDP + VPN + TE] plus a 20 byte TCP + 20 byte IP header within a 1500 byte Ethernet MTU.
Please remember, when we shrink the IP MTU on a transiting router, that will also shrink the maximum possible TCP MSS size to 1448 bytes from the source device. If the router does not signal the source and destination during the TCP handshake, the optimal TCP MSS could potentially create a dropping/fragmentation problem.
Here are the steps to solve this:
Transiting traffic (traffic going via this router):
Router1(config)#int ethernet 1/0
Router1(config-if)#ip mtu 1448
The above command will shrink the IP MTU on the interface to 1448 bytes, creating a 12 byte space for the MPLS label header.
Router1(config-if)#ip tcp adjust-mss ?
<500-1460>
Router1(config-if)#ip tcp adjust-mss 1448
The above command will signal the source and destination device during the three-way handshake to use the TCP MSS size of 1448 bytes so that if they create the full size packet there will still not be any drop/fragmentation on the router.
Terminating traffic (doing SSH, telnet to this router):
Router1(config)# ip tcp mss 1448?
<68-10000> MSS
https://www.incapsula.com/blog/mtu-mss-explained.html
IP MTU and TCP MSS Missmatch – an evil for network performance
Ranti's Blog 