Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add fuzzer scripts for botan #150

Merged
merged 1 commit into from
Dec 8, 2016
Merged

Conversation

randombit
Copy link
Contributor

Have tested several of the fuzzers and everything seems to be working fine locally.

@randombit randombit mentioned this pull request Dec 8, 2016
RUN git clone --depth 1 https://github.com/randombit/botan.git botan
WORKDIR botan
COPY build.sh $SRC/
RUN ls -R $SRC/
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will remove this debug line "RUN ls -R $SRC/" :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops thank you!

@inferno-chromium inferno-chromium merged commit 278a542 into google:master Dec 8, 2016
@kcc
Copy link
Contributor

kcc commented Dec 8, 2016

This looks great as is, thanks for sending this.
Some comments for future improvements.

Will these targets benefit from having a seed corpus?
(Most targets will, but not all)
If so, please consider extending build.sh to supply the corpus.
https://github.com/google/oss-fuzz/blob/master/docs/new_project_guide.md#seed-corpus

The build.sh uses -DUSE_LLVM_FUZZER to modify the behavior of src/extra_tests/fuzzers/jigs/driver.h.
But looks like you don't make any assumption about the actual fuzzing engine being used -- just implement the libFuzzer's interface.
We are using the same interface for other engines as well, including AFL.

[maybe the least important]
You set different max_len parameters to different targets.
That totally makes sense (assuming that the targets can't process inputs longer than their max_len).
But perhaps you want to move this logic to your repo.
E.g. have files ecc_p521.max_len adjacent to ecc_p521.cpp.

Another strategy is to have if (len > MaxLen) return in your "jigs".
This may work better with AFL, since AFL does not respect max_len argument.
With libFuzzer this will cause it to generate some number of mutations longer than MaxLen,
but the overhead of that is low.

@inferno-chromium
Copy link
Collaborator

Looks like Botan builds are failing. One of your fuzzers seems to crash easily on startup, can you please fix that first.
https://oss-fuzz-build-logs.storage.googleapis.com/build_logs/botan/latest.txt

ref_rejected = 1 at 59:/src/botan/src/extra_tests/fuzzers/jigs/pkcs1.cpp
==74== ERROR: libFuzzer: deadly signal
#0 0x4e1243 in __sanitizer_print_stack_trace /src/llvm/projects/compiler-rt/lib/asan/asan_stack.cc:38
#1 0x59f6dd in fuzzer::Fuzzer::CrashCallback() /src/libfuzzer/FuzzerLoop.cpp:270:5
#2 0x59f5e1 in fuzzer::Fuzzer::StaticCrashSignalCallback() /src/libfuzzer/FuzzerLoop.cpp:259:6
#3 0x646d37 in fuzzer::CrashHandler(int, siginfo_t*, void*) /src/libfuzzer/FuzzerUtilPosix.cpp:38:3
#4 0x7f3ddf8a238f (/lib/x86_64-linux-gnu/libpthread.so.0+0x1138f)
#5 0x7f3ddeedb427 in gsignal (/lib/x86_64-linux-gnu/libc.so.6+0x35427)
#6 0x7f3ddeedd029 in abort (/lib/x86_64-linux-gnu/libc.so.6+0x37029)
#7 0x5121e9 in fuzz(unsigned char const*, unsigned long) /src/botan/src/extra_tests/fuzzers/jigs/pkcs1.cpp:59:4
#8 0x510fd1 in LLVMFuzzerTestOneInput /src/botan/src/extra_tests/fuzzers/jigs/driver.h:34:4
#9 0x5a58dd in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/libfuzzer/FuzzerLoop.cpp:536:13
#10 0x5a4b0d in fuzzer::Fuzzer::ShuffleAndMinimize(std::__1::vector<std::__1::vector<unsigned char, std::__1::allocator >, std::__1::allocator<std::__1::vector<unsigned char, std::__1::allocator > > >) /src/libfuzzer/FuzzerLoop.cpp:464:3
#11 0x5424d6 in fuzzer::FuzzerDriver(int
, char***, int ()(unsigned char const, unsigned long)) /src/libfuzzer/FuzzerDriver.cpp:523:6
#12 0x536f08 in main /src/libfuzzer/FuzzerMain.cpp:20:10
#13 0x7f3ddeec682f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
#14 0x41b908 in _start (/out/pkcs1+0x41b908)

NOTE: libFuzzer has rudimentary signal handlers.
Combine libFuzzer with AddressSanitizer or similar for better crash reports.
SUMMARY: libFuzzer: deadly signal
MS: 0 ; base unit: 0000000000000000000000000000000000000000

artifact_prefix='./'; Test unit written to ./crash-da39a3ee5e6b4b0d3255bfef95601890afd80709

@inferno-chromium
Copy link
Collaborator

Ok that error was fixed, now there is a new failure.

  • clang++ -g -fsanitize=address -fsanitize-coverage=edge,indirect-calls,8bit-counters -stdlib=libc++ -DUSE_LLVM_FUZZER -std=c++11 -I/src/botan/build/include -o /out/ber_decode /src/botan/src/extra_tests/fuzzers/jigs/ber_decode.cpp -L/src/botan -lbotan-1.11 -lFuzzingEngine
    In file included from /src/botan/src/extra_tests/fuzzers/jigs/ber_decode.cpp:6:
    /src/botan/src/extra_tests/fuzzers/jigs/driver.h:100:17: error: no type named 'ChaCha' in namespace 'Botan'
    Botan::ChaCha m_chacha;
    ~~~~~~~^
    /src/botan/src/extra_tests/fuzzers/jigs/driver.h:104:11: error: non-const lvalue reference to type 'Botan::RandomNumberGenerator' cannot bind to a value of unrelated type 'ChaCha20_RNG'
    return rng;
    ^~~
    2 errors generated.

@randombit
Copy link
Contributor Author

@kcc Sorry for delayed reply just saw this. Yes I need to work out seed corpus for at least cert, CRL, and TLS client/server, hopefully within next week or so. I will probably also take corpus generated from ClusterFuzz for some/all? of the others.

As to your other comments, good suggestions all. I will change it so our fuzzers natively export the LLVM interface, and then the AFL adaptor main function (for local AFL use) can just call the LLVM fuzzer functions. And returning if the input is too long is clearer in fuzzer code and simplifies build on OSS-Fuzz side.

randombit added a commit to randombit/oss-fuzz that referenced this pull request Dec 17, 2016
Botan's fuzzers now have builtin maximums where necessary, instead of relying on
max len option. So the code to generate the .options files goes. Suggested by
@kcc in google#150

Use corpus for the common formats, seems to help with initial coverage at least.

Disables Botan's pool allocator at build time, since it hides things from ASan.
The fuzzer driver already tries to disable it at runtime in LLVMFuzzerInitialize,
but the Clusterfuzz coverage report indicates that this init function is not
ever called, and the pool allocator ends up being used.
inferno-chromium pushed a commit that referenced this pull request Dec 17, 2016
Botan's fuzzers now have builtin maximums where necessary, instead of relying on
max len option. So the code to generate the .options files goes. Suggested by
@kcc in #150

Use corpus for the common formats, seems to help with initial coverage at least.

Disables Botan's pool allocator at build time, since it hides things from ASan.
The fuzzer driver already tries to disable it at runtime in LLVMFuzzerInitialize,
but the Clusterfuzz coverage report indicates that this init function is not
ever called, and the pool allocator ends up being used.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants