EMSGSIZE networking network_error ai_generated partial

ICMP: Frag needed but DF set for packet size 1500 on vlan100

ID: networking/vlan-mtu-mismatch

Also available as: JSON · Markdown · 中文
80%Fix Rate
85%Confidence
1Evidence
2024-03-15First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
Linux kernel 5.10-6.8 active — — —
Cisco IOS XE 17.x active — — —
Juniper Junos 21.x-23.x active — — —
Nginx 1.24-1.26 active — — —
HAProxy 2.8-3.0 active — — —

Root Cause

A packet with the Don't Fragment (DF) flag set exceeds the MTU of a VLAN interface (e.g., 1400 due to QinQ or MPLS overhead), causing the router to send an ICMP Fragmentation Needed message, but the sender ignores it, leading to a black hole.

generic

中文

设置了不分片（DF）标志的数据包超过了VLAN接口的MTU（例如由于QinQ或MPLS开销导致MTU为1400），路由器发送ICMP需要分片消息，但发送方忽略该消息，导致通信黑洞。

Official Documentation

https://datatracker.ietf.org/doc/html/rfc1191

Workarounds

  1. 85% success Reduce the TCP MSS on the server to account for VLAN overhead: run `ip link set dev eth0 mtu 1400` on the server, then adjust MSS clamping via iptables: `iptables -t mangle -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu` 
    Reduce the TCP MSS on the server to account for VLAN overhead: run `ip link set dev eth0 mtu 1400` on the server, then adjust MSS clamping via iptables: `iptables -t mangle -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu`
  2. 80% success Configure MSS clamping on the router for the VLAN: on Cisco, `ip tcp adjust-mss 1360` on the VLAN interface; on Linux, use `ebtables` or `iptables` to rewrite MSS. 
    Configure MSS clamping on the router for the VLAN: on Cisco, `ip tcp adjust-mss 1360` on the VLAN interface; on Linux, use `ebtables` or `iptables` to rewrite MSS.
  3. 70% success Increase the MTU on the entire path (e.g., set jumbo frames on switches and routers) if the infrastructure supports it, but verify end-to-end consistency. 
    Increase the MTU on the entire path (e.g., set jumbo frames on switches and routers) if the infrastructure supports it, but verify end-to-end consistency.

中文步骤

  1. Reduce the TCP MSS on the server to account for VLAN overhead: run `ip link set dev eth0 mtu 1400` on the server, then adjust MSS clamping via iptables: `iptables -t mangle -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu`
  2. Configure MSS clamping on the router for the VLAN: on Cisco, `ip tcp adjust-mss 1360` on the VLAN interface; on Linux, use `ebtables` or `iptables` to rewrite MSS.
  3. Increase the MTU on the entire path (e.g., set jumbo frames on switches and routers) if the infrastructure supports it, but verify end-to-end consistency.

Dead Ends

Common approaches that don't work:

  1. Increasing MTU on the VLAN interface to 1500 without adjusting underlying physical links 85% fail

    The physical path (e.g., MPLS or QinQ) adds headers, so the actual MTU remains lower; increasing VLAN MTU alone causes fragmentation at lower layers or silent drops.

  2. Disabling DF flag on all TCP packets using iptables rules 70% fail

    Many applications (e.g., NFS, IPsec) require DF for PMTUD; disabling it can cause path MTU discovery failure and performance degradation.

  3. Ignoring the error and assuming it's a transient network issue 90% fail

    The error persists and causes intermittent timeouts for large-file transfers or streaming, leading to user complaints and retransmission storms.