go-fuzz-build with '.' #3
Comments
|
Yeah, I have no idea how to fix it. Go-fuzz-build is hacky enough already. |
|
Maybe put a readme note if there isn't one already. I may have missed it. |
|
Local import paths always start with ".", right? I think it's better to provide readable diagnostic in go-fuzz-build if package name starts with ".". What do you think? |
|
Sounds good to me. I am up and running with my own fuzzing now btw. |
|
Great! |
Is there a way to see which code the profiler is actually running? something like go tool cover -html=prof.html but for the current "best" mutation, perhaps via a status http server. |
|
My corpus is currently not good. I have code which converts files into a tree data structure based on features of the data, I want to see if the coverage guided mutation can trigger the case where the maximum tree depth is met. |
|
Most of the bugs that I've found triggered by relatively simple inputs. I've not yet seen fuzzer generating a tricky malicious input. Eager to hear about your experience. Leave it at least for a night. |
restarts seems to be OK, the test is not crashing too frequently |
No, there is no anything like this atm. |
|
Ah, I understand. That is a good solution. Viewing just gives an idea of what sorts of code paths are being taken. In my current case, my function is already randomly tested with full coverage, so I don't think it will find any bugs. My problem was that in order to get full test coverage, I had to artificially modify probabilities to make certain things more likely. I was curious if using this will allow me to remove the artificial special cases from my test code. |
|
btw: |
|
I think once knowledge of this tool spreads and the tool is easy enough to use (easy after those few fixes), hobbyists will start security fuzzing any code they can get their hands on. |
|
top looks reasonable
I would suggest to delete delete any of you current code. Generators usually can generate different things that fuzzers can, so they are complementary. I bet that go-fuzz won't be able to generate everything your generator can. That's based on my experience with GoSmith (random Go program generator) and Trinity (linux kernel syscall fuzzer). Btw, I have an idea to combine fuzzing and generation in symbiosis. What do you think? |
|
I like that idea. One example that comes to mind is that most compilers have really bad test suites checking it rejects bad input, instead they check for correct compilation only. I can see how something like that could detect missing validation checks of mostly correct inputs. Another idea would be to generate a correct program, mutate it slightly, and then compare gccgo with gc to make sure both either accept, or reject the mutated program to ensure consensus and no crashes. This case doesn't rely on coverage info though. |
|
Although with regard to fuzz testing the Go compilers, it is less useful unless it is a security flaw, and not just an edge case. The bugs you found in the image packages could potentially stop denial of service attacks on servers so are really valuable imo. |
* preliminary Go modules support (#2) #### Summary The heart of the change is very small -- we stop overwriting the user's value for the `GO111MODULE` env var in two places. This means we end up respecting whatever the user has set for `GO111MODULE`, and relying on `cmd/go` to interpret the meaning of an unset `GO111MODULE` (which defaults to `auto` for Go 1.11-1.13, and likely will default to `on` at some point during Go 1.14 development). In addition, there are some comments added, as well as an explicit check for `GOFLAGS=-mod=vendor`, which is not supported. The tests exercises 'replace' directives, v2+ modules, GO111MODULE set to on|off|auto, running inside GOPATH, running outside GOPATH, the mechanisms by which go-fuzz/go-fuzz-dep is found, as well as vendoring (which is not supported, but it tests that a sensible error is emitted). #### Additional Details Historically, go-fuzz has set up a clean GOPATH environment in a temp directory, instrumented the code for fuzzing coverage, and built the instrumented code in the temp directory. The current approach is to keep doing that (and not set up a full-blown modules-based environment for building), but use modules to find the code to instrument. v2+ modules are potentially problematic due to Semantic Import Versioning (with the major version in the import path). However, it turns out to not be too bad to handle it. For a v2+ module created following the 'major branch' approach, the /vN is materialized into a physical directory. For example, the github.com/russross/blackfriday/v2 module ends up in the /tmp/blah/gopath/src/github.com/russross/blackfriday/v2 directory. This should be a valid layout, and people use this layout today (including when manually switching to a 'major subdirectory' approach for a v2+ module, which involves creating a /v2 or /vN directory so that pre-modules toolchains follow a /v2 or /vN import path and find an on-disk layout that they expect without any knowledge of modules). The approach does not copy go.mod files at all. (Before starting, I was thinking I would need to copy go.mod files to trigger "minimal module awareness", which is a transitional mechanism within cmd/go, but after some experimentation / thinking, I concluded it was not needed based on materializing the /v2). For tests, I am using the script-driven test facility that the core Go team created for testing cmd/go, which is well tuned to testing the various permutations of modules, including it lets you relatively easily set up go.mod files and source files, run commands and check results. (Some more details of the capability here: https://github.com/golang/go/blob/master/src/cmd/go/testdata/script/README). * modules support: adjust wording based on first-pass review, and update vendor test (#3) * travis: attempt to install cmd/testscript * testing: initial check of 'go-fuzz-build -h' output to see if any testscripts work * testing: change name of first test script * travis: check if the 'testscript' cmd seems to function at all. * testing: use the exec command to invoke 'go-fuzz-build -h' * testing: create initially failing test case for v2 modules * travis: run testscript for v2 modules * testing: remember to use 'exec' to invoke go-fuzz-build * testing: adjust comments in v2 module testscript * go-fuzz-build: preliminary module support * testing: flip from an expected failure to an expected pass for a module test * go-fuzz-build: update comments for preliminary modules support * go-fuzz: preliminary modules support for go-fuzz itself * travis: go-fuzz-defs sanity check * travis: additional temp sanity check * travis: add more comments, some additional sanity checks in case we need to debug in future * travis: s/go-fuzz-deps/go-fuzz-dep/ * testing: validate behavior of v2 modules * testing: renamed fuzz_help.txt * testing: update travis to use the renamed testscripts * travis: conditionally set GO111MODULE=auto, and test Go 1.13 rather than 1.11 * testing: validate modules outside GOPATH * testing: validate modules inside GOPATH * testing: invoke timeout properly * testing: preliminary test for vendored modules (not yet supported) * testing: validate how go-fuzz-dep and go-fuzz-defs are found * travis: run the latest testscripts for modules testing * testing: fix typo in mod_outside_gopath.txt file name * travis: fix typo in file name * go-fuzz-build: detect -mod=vendor * testing: validate we get a proper error for -mod=vendor * readme: describe preliminary modules support * readme: correct typo in modules support * readme: adjust modules support section * readme: tweak wording based on PR review * go-fuzz: tweak comment wording by removing "preliminary modules support" * go-fuzz-build: tweak comment wording by removing "preliminary modules support" * mod_vendor.txt: avoid triggering Go 1.14 auto detection of a vendor directory * readme: update wording based on review * readme: remove a blank line to re-trigger travis
I don't mind if you don't care to support this.
The text was updated successfully, but these errors were encountered: