ZNetLab › Published › Article

MTU black holes: why ping works and SSH hangs

ArticleTroubleshooting@admin2 min read

MTU black holes: why ping works and SSH hangs

Small packets get through and large ones vanish. The mechanism that is meant to fix this is ICMP, which is exactly what people block.

A tunnel goes up. Ping works. SSH connects, prints the banner, and hangs. Nothing is down, nothing is logged, and the ACLs look right.

This is an MTU black hole, and it is the same fault every time.

What is happening

Ping sends 64-byte packets. Your SSH session negotiates fine and then tries to send a 1500-byte one. Somewhere along the path there is a link that cannot carry 1500 bytes — a GRE tunnel takes 24, IPsec takes more, PPPoE takes 8 — and the packet has the Don't Fragment bit set, as nearly all TCP does.

The router on that link does the correct thing: it drops the packet and sends back ICMP type 3 code 4, "fragmentation needed and DF set", which carries the MTU it can take. The sender receives it, lowers its idea of the path MTU, and retransmits. That mechanism is Path MTU Discovery and it works.

Unless the ICMP never arrives.

Why the ICMP never arrives

Somebody blocked ICMP. Not maliciously — "ICMP is a security risk" is on every hardening checklist ever written, and the usual implementation is deny icmp any any somewhere upstream. Now the sender gets no feedback at all: its packet simply disappears, it retransmits the same oversized packet, and that disappears too. The session hangs rather than failing, because TCP is being patient.

Find it in two commands

C:\> ping 10.2.0.9 -f -l 1472
Packet needs to be fragmented but DF set.

1472 plus 28 bytes of IP and ICMP header is 1500. Walk the size down until it passes and you have the real path MTU. On IOS:

R1# ping 10.2.0.9 size 1500 df-bit

Fix it in one of three places

R1(config-if)# ip tcp adjust-mss 1360

MSS clamping only helps TCP. UDP over a tunnel has the same problem and no handshake to clamp, which is why a VPN can carry your SSH and break your file shares.

> The lesson is not "do not block ICMP". It is that ICMP is part of IP, > not a diagnostic add-on, and dropping all of it breaks a protocol you are > relying on without telling you which one.

mtupmtudtroubleshootingccnp

Join the discussion

Replies, likes and bookmarks live in the community half, which needs a free account. Writing here is free too, and everything is reviewed before it is published.

Open this in the communityEverything publishedHow this works