Skip to content

Latest commit

 

History

History
44 lines (26 loc) · 4.4 KB

checksum-hacks.md

File metadata and controls

44 lines (26 loc) · 4.4 KB

Checksum Hacks

The Linux kernel comes with support to checksum offload, which leaves calculations of RFC 1071 checksum in TCP, UDP and more (and also CRC for SCTP) to the NIC if possible, reducing CPU usage. This is done using three fields inside struct sk_buff:

  • ip_summed, if set to CSUM_PARTIAL, will make the skb do the offloading;
  • csum_start points to the start of the L4 packet to be summed; and
  • csum_offset points to the 16-bit checksum field where it should be written.

(Encapsulations, as well as RX checksum offload are not discussed here for the sake of simplicity. We only focus on TX checksum offload.)

This leaves us a problem: when Mimic transforms a UDP packet to fake TCP one, csum_start and csum_offset cannot be changed in eBPF alone; there is simply no such methods of doing that. This will make the NIC (or whatever that finishes the checksum) put the checksum in the wrong place and makes the packet malformed.

To solve this issue, we have to extend eBPF on the kernel side, and we also have to maintain compatibility across kernel versions. Mimic implements two kinds of checksum hacks implemented in its kernel module:

  • kfunc (default): using BPF kernel functions to extend eBPF. This is the current way of extending eBPF in-tree, and requires BPF JIT (CONFIG_BPF_JIT=y) and BTF (CONFIG_DEBUG_INFO_BTF=y) support.

  • kprobe: change existing BPF helpers' behaviour using kprobe. This is way hackier, but could be used when kernel BTF is not present (with BPF program's own BTF information also stripped). It also allows the kernel module to be optional, since not every case requires checksum hack. Pass CHECKSUM_HACK=kprobe to make to enable this behaviour (you would almost certainly need STRIP_BTF_EXT=1 too).

Normally with most Linux distros (Debian, Fedora, Arch, etc.), kfunc should be used. But in restricted environments like OpenWrt, kprobe might be the option.

When is Checksum Hack Not Necessary?

When the two conditions below both matches, checksum hack is not necessary.

  1. Driver does not use csum_offset, or checksum offload is maunally disabled.

    You can check if your NIC's driver source code contains csum_offset by simple searching either inside Linux kernel source code or out-of-tree somewhere. Realtek and Mediatek's Ethernet driver does not use it, while Intel and many others uses it.

  2. UDP socket in userspace.

    Kernel udptunnel{4,6}, used by many in-kernel tunnels such as WireGuard, and encapsulation protocols like FOU and GENEVE, is always CSUM_PARTIAL (see udp4 and udp6). Userspace implementations are not affected, like wireguard-go, OpenVPN and Hysteria. (However, since Hysteria uses (modified) QUIC, it is sometimes better to keep it as-is than using Mimic or other fake TCP solutions and turning it to something unknown to the middleboxes.)

Platform Support

kfunc hack requires CONFIG_BPF_JIT=y and CONFIG_DEBUG_INFO_BTF=y to be set. Below lists kernel support for BPF JIT and kfunc call per CPU architecture:

kprobe hack requires CONFIG_KRETPROBE=1, and any Linux version >= 6.1 will work.