ICMPERR_HOP_LIMIT errors are generated for blacklisted packets

[toreanderson]

When a Jool node in stateless mode receives an IPv4 packet that is in its blacklist with TTL=1, it will steal the packet from its intended receipient and return an ICMPv4 error:

Catching IPv4 packet: 192.168.1.1->192.168.1.2
Translating the Packet.
Sending ICMPv4 error: ICMPERR_HOP_LIMIT, type: 11, code: 0.

Whether the destination address is in the implicit blacklist (due to being assigned to a local interface) or explicitly added with jool_siit --blacklist --add makes no difference.

The easiest way to reproduce this is to run ping -t 1 $joolnode from a machine in the same LAN segment. The moment you do modprobe jool_siit disabled=0 on «$joolnode», the ICMPv4 echo-replies turn into ttl-exceeded errors. After removing the module again, or running jool_siit --disable, the echo-replies come back.

For me, this is a blocker stopping me from putting Jool in production, because I need the Jool node to speak OSPF with its upstream router to dynamically advertise the various pools to the network. The upstream router sends OSPF hellos to the OSPF multicast address 224.0.0.5 with TTL=1, but since I cannot figure out how to prevent Jool from stealing these packets from the BIRD routing daemon, the OSPF adjacency cannot form, and the required routes cannot be advertised to the network.

[ydahhrk]

Here's a full list of everything I see that might go wrong, which should cause SIIT Jool to drop packets before realizing they weren't meant to be translated:

1. Header corruption (AFAIK this is fine, since the kernel would drop the packet anyway)
   • bad layer-4 length
   • ICMP inner IP header doesn't match the protocol of the external header
   • ICMP inner packet's IHL < 5
   • ICMP inner packet's total length < ICMP inner packet's IHL * 4
   • ICMP inner packet is not first fragment
   • Bad ICMP error checksum
   • ICMP error inside ICMP error
2. Fragmented pings
3. Fragmented packets with UDP checksum zero
4. Translated packet allocation failure
5. TTL runs out

The most reliable and natural way to solve this is by switching frameworks. This issue is another argument in favor of #140.

Alternatively, I can move the the address translation to somewhere earlier. I think the best tradeoff between fixing the problem and not changing the code too aggressively is to move it to the beginning of the translate submodule. This will only solve bullets 4 and 5.

[toreanderson]

Bullet 5 is the only one I really care about here due to the OSPF issue (RFC 2328 requires TTL=1, and I have found no way to override that in JUNOS). So it's not too much trouble to make a quick hack that solves bullet 5, that would be much appreciated.

[ydahhrk]

It should work now. Let's not close this issue until the problem gets a more elegant fix, though.

[toreanderson]

The kame is dancing, so indeed, that made my OSPF sessions come up. Thanks a lot! :-)

[ydahhrk]

:p

[ydahhrk]

The "more elegant fix" is iptables mode. Packets will need to match a rule, so there will be no risk of them not being intended for translation.

4.0.0 released; closing.