test: Add Atheris fuzzing setup for cve-bin-tool

[terriko]

I'm working on some initial exploration into fuzzing cve-bin-tool using the Atheris fuzzer from Google. This looks like it handles instrumentation of python code and then hooks up libfuzzer.

Fuzzing (for those not familiar) involves sending potentially large amounts of "bad" data into a program and seeing how it responds. It's regularly used for security and validation to find crashes and potential security holes. Since we've never done it on cve-bin-tool before I expect we'll find a lot of crashes. Doing fuzzing can potentially damage data on your system, so I recommend you do this within a vm, docker container, or have some other way of sandboxing the tool. (Note that virtualenv won't work for this; it does not provide the protections needed.)

[codecov-commenter]

Codecov Report

Merging #1661 (7f14421) into main (23baf5e) will decrease coverage by 2.68%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #1661      +/-   ##
==========================================
- Coverage   80.93%   78.24%   -2.69%     
==========================================
  Files         291      291              
  Lines        6031     6032       +1     
  Branches      991      991              
==========================================
- Hits         4881     4720     -161     
- Misses        941     1095     +154     
- Partials      209      217       +8     

Flag	Coverage Δ
longtests	78.24% <ø> (-2.69%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
test/test_cvedb.py	58.53% <0.00%> (-34.15%)	⬇️
cve_bin_tool/checkers/xml2.py	61.36% <0.00%> (-27.28%)	⬇️
test/test_available_fix.py	76.47% <0.00%> (-23.53%)	⬇️
test/test_json.py	70.00% <0.00%> (-20.00%)	⬇️
cve_bin_tool/cvedb.py	71.85% <0.00%> (-13.67%)	⬇️
test/test_scanner.py	62.30% <0.00%> (-12.31%)	⬇️
test/test_cli.py	81.01% <0.00%> (-10.13%)	⬇️
cve_bin_tool/nvd_api.py	75.00% <0.00%> (-9.49%)	⬇️
cve_bin_tool/available_fix/redhat_cve_tracker.py	71.42% <0.00%> (-8.93%)	⬇️
cve_bin_tool/available_fix/debian_cve_tracker.py	77.55% <0.00%> (-6.13%)	⬇️
... and 4 more

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

[anthonyharrison]

Looks good. I note the comment about the need to create a separate environment to do the fuzzing (VM, docker container, etc). Whilst it isn't the purpose of this repo to provide the environment, should we define the contents of such an environment as part of a 'How to Fuzz' guide to help interested users/developers?

[terriko]

ugh. You're not wrong, but telling people how to set up VMs is not my favourite thing. I mostly just wanted to make sure that people knew virtualenv wasn't going to provide protections (and now I see I did that in the comment but not the readme, so I'll go fix that).

Can we point at someone else's instructions for vm/container setup and maybe give a setup script? I have a bash setup script partially written as part of my internal cloud setup, and nothing about it should be proprietary. Let me see about adding that in here...

[terriko]

Okay, updated the docs part with some links. I also added in a (commented out) line to get it running with a valid filename. I'm not sure if that's actually helping, since my run with that addition had to be stopped early for unrelated reasons.

I'm unlikely to get back to this before Tuesday, if anyone else wants to have a go please feel free. I'm going to go ahead and merge what I have so far to make it easier for anyone else who wants to have a go.