Fuzzer Framework #5940
Fuzzer Framework #5940
Conversation
| void FuzzObject::commit(SharedRealm shared_realm, FuzzLog& log) | ||
| { | ||
| log << "FuzzObject::commit();\n"; | ||
| log << "FuzzObject::commit() - shared_realm->is_in_transaction();\n"; |
There was a problem hiding this comment.
logs like this cannot be replayed as code, isn't the point of the log file to copy-paste into a test case to reproduce and debug it?
There was a problem hiding this comment.
I am not sure if I understood the comment, the point of the log is to help us to debug the issue. The crash report will serve as input to reproduce the issue.
There was a problem hiding this comment.
The crash report is a good first step to reproduce the issue, but it is not human readable. In the previous AFL based fuzz tests, the purpose of the log file was to be able to copy and paste the output into a core unit test so that we could debug the issue further. The logs generated by this code are not compilable code without being modified, this would be fine for a small amount of code, but in my experience, the logs generated are many hundreds of lines long which would be very tedious to transform by hand into a unit test. However, this is more of a nice to have feature so it is not blocking for me.
There was a problem hiding this comment.
I can try to reimplement this feature, but I would like to get this PR into master first.
Co-authored-by: James Stone <james.stone@mongodb.com>
test/object-store/CMakeLists.txt
Outdated
| @@ -126,36 +126,23 @@ target_include_directories(ObjectStoreTests PRIVATE | |||
| # on Apple platforms we use the built-in CFRunLoop | |||
| # everywhere else it's libuv, except UWP where it doesn't build | |||
| if(NOT APPLE AND NOT WINDOWS_STORE) | |||
| option(REALM_FETCH_MISSING_DEPENDENCIES "Download missing dependencies with CMake's FetchContent where possible" ON) | |||
| if(REALM_FETCH_MISSING_DEPENDENCIES) | |||
There was a problem hiding this comment.
Still not sure why we would need to change this file. I can build without the change and I would prefer not to be forced to download and build libuv.
|
I am not sure how you are compiling, when I did not export the libUV symbols all the builders were failing. However it was working for me locally as well, since I had installed libuv via shell command and symbols were visible in my system. #5940 (comment) |
What, How & Why?
We want to have fuzz testing run consistently on evergreen every night, in order to try to find out possible bugs before they actually get filled as a result of crashes in production.
LibFuzzer seems a better option since it easily integrates with Clang. AFL++ is still supported, but eventually the support for it should be dropped.
The final goals are
This PR is just the first seed, we need to move all under the realm-fuzzer umbrella all the small fuzzers we already have. Improve the column types we are testing. Start fuzzing parts of the code we don't fuzz right now (e.g the query subsystem)