Trailing junk on the end of packets?

[djs55]

Magnus writes

I've finally managed to set up an ARM system that can compile
successfully (based on the pre-built image with swap added and an NTP
client installed).  Compiling for unix now works perfectly fine,
albeit a bit slow.

The next step was to get mirage-www into a xen image.  I've
succesfully built and started the following two configurations:

    MODE=xen FS=crunch NET=direct DHCP=true PORT=80
    MODE=xen FS=crunch DEPLOY=true ADDR=192.168.0.21 MASK=255.255.255.0 GWS=192.168.0.1 PORT=80

In both cases I've modified www.xl to contain

    vif = [ 'bridge=xenbr0' ]

In both cases the VM comes up just fine, when using DCHP it gets an IP
without issues.  Then, when sitting there waiting for a connection I
get this on the console:

    RX exn File "lib/ipv4.ml", line 181, characters 4-10: Assertion failed

# When using DHCP

The DHCP setup works fine (I think):

    Sending DHCP broadcast (length 552)
    DHCP response:
    input ciaddr 0.0.0.0 yiaddr 192.168.0.21
    siaddr 192.168.0.1 giaddr 0.0.0.0
    chaddr 00163e2f813700000000000000000000 sname  file
    DHCP: offer received
                        IPv4: 192.168.0.21
                                          Netmask: 255.255.255.0
                                                                Gateways: [192.168.0.1]
     sg:true gso_tcpv4:true rx_copy:true rx_flip:false smart_poll:false
    ARP: sending gratuitous from 192.168.0.21
    DHCP offer received and bound to 192.168.0.21 nm 255.255.255.0 gw [192.168.0.1]
    Manager: configuration done
    RX exn File "lib/ipv4.ml", line 181, characters 4-10: Assertion failed
    RX exn File "lib/ipv4.ml", line 181, characters 4-10: Assertion failed
    RX exn File "lib/ipv4.ml", line 181, characters 4-10: Assertion failed
    ARP responding to: who-has 192.168.0.21?
    ARP: transmitting probe -> 192.168.0.11
    ARP: updating 192.168.0.11 -> 00:c2:c6:0f:72:dd

and then follows up with hex dumps:

    argh, it's all broken! len = 26 payload_len = 20

    45 00 00 28 2b af 40 00 40 06 8d af c0 a8 00 0b
    c0 a8 00 16 ea 30 00 50 d0 90 08 dc 1a fe 26 b0
    50 11 00 e5 27 e1 00 00 00 00 00 00 00 00
    argh, it's all broken! len = 26 payload_len = 20

(peering at the hex dump, with wikipedia in another window)

IP version = 4
Protocol = 6 = TCP
IHL = 5 words (ie no options)
Total length = 0x28

payload_len = 0x28 - 5 * 4 = 20 (looks ok)

It looks like the mirage-net-xen driver is supplying an extra 6 bytes for some reason.


    45 00 00 28 2b b0 40 00 40 06 8d ae c0 a8 00 0b
    c0 a8 00 16 ea 30 00 50 d0 90 08 dc 1a fe 26 b0
    50 11 00 e5 27 e1 00 00 00 00 00 00 00 00
    argh, it's all broken! len = 26 payload_len = 20

    45 00 00 28 2b b1 40 00 40 06 8d ad c0 a8 00 0b
    c0 a8 00 16 ea 30 00 50 d0 90 08 dc 1a fe 26 b0
    50 11 00 e5 27 e1 00 00 00 00 00 00 00 00
    argh, it's all broken! len = 26 payload_len = 20

    45 00 00 28 2b b2 40 00 40 06 8d ac c0 a8 00 0b
    c0 a8 00 16 ea 30 00 50 d0 90 08 dc 1a fe 26 b0
    50 11 00 e5 27 e1 00 00 00 00 00 00 00 00
    argh, it's all broken! len = 26 payload_len = 20

    45 00 00 28 2b b3 40 00 40 06 8d ab c0 a8 00 0b
    c0 a8 00 16 ea 30 00 50 d0 90 08 dc 1a fe 26 b0
    50 11 00 e5 27 e1 00 00 00 00 00 00 00 00
    argh, it's all broken! len = 26 payload_len = 20

    45 00 00 28 2b b4 40 00 40 06 8d aa c0 a8 00 0b
    c0 a8 00 16 ea 30 00 50 d0 90 08 dc 1a fe 26 b0
    50 11 00 e5 27 e1 00 00 00 00 00 00 00 00
    ARP responding to: who-has 192.168.0.22?
    argh, it's all broken! len = 26 payload_len = 8

    45 00 00 1c 00 00 00 00 01 02 19 36 c0 a8 00 01
    e0 00 00 01 11 0a ee f5 00 00 00 00 00 00 00 00
    00 00 00 00 00 00 00 00 00 00 00 00 00 00
    argh, it's all broken! len = 26 payload_len = 12

Even though this packet has a shorter total length (0x1c), the frame has the same length as the previous ones.

    45 00 00 20 00 00 00 00 01 02 19 32 c0 a8 00 01
    e0 00 00 01 11 0a ec 78 00 00 00 00 02 7d 00 00
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 

It looks like netif is returning trailing junk? Or possibly the peer is sending ethernet frames padded with junk?

[djs55]

This is easy to repro on a clean build. Interestingly the troublesome assert was added to tcpip recently:

mirage/mirage-tcpip@00ca583

was it always the case that mirage-net-xen read trailing junk on the end of ethernet frames? Is this gasp being sent by the backend? Say it isn't so

[hannesm]

Sorry, this seems to be my fault. I submitted mirage/mirage-tcpip#121 to allow trailing bytes for IP packets. I think those packets with trailing bytes simply occur in nowadays systems..

[koleini]

I use Channel interface (read_some function) to read bytes from a TCP socket in Mirage OpenFlow switch. I read those trailing bytes in an ACK packet as data, which confuses the software switch. Is there any specific reason to allow those trailing bytes? Suggestion to fix (in ipv4.ml):

if Cstruct.len data >= payload_len then begin
      let data = Cstruct.sub data 0 payload_len in

[samoht]

I agree this should be addressed properly. PR welcome.

[talex5]

I'm seeing this problem when making a TCP connection from my laptop to a unikernel on a cubieboard 2 (connecting from dom0 doesn't show the problem). The final ack in the syn,syn/ack,ack sequence is seen as having 6 trailing zeros. Mirage then acks the receipt of these phantom bytes, and things don't go well from then on...
Looking at the ack in Wireshark, the IP packet is 42 bytes long, so perhaps something is rounding the buffer up to a multipe of 8?

[hannesm]

while mirage/mirage-tcpip#145 is a hotfix for this issue, I'd hope someone would investigate whether the xen netif does the proper thing (or whether there is an issue in either xen or mirage-net-xen). closing it by the mirage ip layer doing some byte array cutting feels slightly hackish imho.

[samoht]

You are right, I didn't want to close that issue (which tracks the underlying bug). Anyway, rescaling the input buffer is always a good thing to do anyway.