fuzz testing

[jb55]

fuzz testing the linux kernel has cropped thousands of bugs, mostly in interfaces where you are encoding and decoding stuff. rust-lightning has been building fuzzers for lightning. It would be interesting to see how badly we can break c-lightning with these kinds of tests.

[niftynei]

If these count as 'fuzz' testing then we're doing some amount of this already, but could definitely improve.

https://github.com/ElementsProject/lightning/blob/master/tests/test_connection.py#L1494
https://github.com/ElementsProject/lightning/blob/master/wire/test/run-peer-wire.c#L904

[ZmnSCPxj]

What fuzz framework is easy to integrate with a networking application written in C?

[jb55]

The go-to appears to be afl (american fuzzy loop), I haven't tried it yet.

[ZmnSCPxj]

american fuzzy lop, not loop.
It seems, named after a dog breed, whatever a "dog" is, I am sure most humans find it delicious.

From what I understand, it requires that a program input a file.
Possibly we can compile the individual channel-level daemons with some way of forcing their inputs to arrive from a file rather than from "real" lightningd or other peers.
Though I admit I have not investigated much.

[ZmnSCPxj]

For the individual channel-level daemons, possibly we can add a --afl-fuzz flag that causes them to instead read messages from STDIN_FILENO and throw any output messages to /dev/null.

Possibly we can have messages from STDIN_FILENO mode to include whether the message is supposed to come from lightningd or from the network, then have an internal forwarder emulate the sockets. Not sure how to emulate forwarding of fds, maybe just close them when transferring out, and have a special code byte in STDIN_FILENO to feed a faked fd in.

onchaind is probably simplest as it only interacts with lightningd.

For the more complex daemons such as gossipd, connectd, and lightningd itself..................