forked from dvyukov/go-fuzz
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
preliminary Go modules support (dvyukov#274)
* preliminary Go modules support (#2) #### Summary The heart of the change is very small -- we stop overwriting the user's value for the `GO111MODULE` env var in two places. This means we end up respecting whatever the user has set for `GO111MODULE`, and relying on `cmd/go` to interpret the meaning of an unset `GO111MODULE` (which defaults to `auto` for Go 1.11-1.13, and likely will default to `on` at some point during Go 1.14 development). In addition, there are some comments added, as well as an explicit check for `GOFLAGS=-mod=vendor`, which is not supported. The tests exercises 'replace' directives, v2+ modules, GO111MODULE set to on|off|auto, running inside GOPATH, running outside GOPATH, the mechanisms by which go-fuzz/go-fuzz-dep is found, as well as vendoring (which is not supported, but it tests that a sensible error is emitted). #### Additional Details Historically, go-fuzz has set up a clean GOPATH environment in a temp directory, instrumented the code for fuzzing coverage, and built the instrumented code in the temp directory. The current approach is to keep doing that (and not set up a full-blown modules-based environment for building), but use modules to find the code to instrument. v2+ modules are potentially problematic due to Semantic Import Versioning (with the major version in the import path). However, it turns out to not be too bad to handle it. For a v2+ module created following the 'major branch' approach, the /vN is materialized into a physical directory. For example, the github.com/russross/blackfriday/v2 module ends up in the /tmp/blah/gopath/src/github.com/russross/blackfriday/v2 directory. This should be a valid layout, and people use this layout today (including when manually switching to a 'major subdirectory' approach for a v2+ module, which involves creating a /v2 or /vN directory so that pre-modules toolchains follow a /v2 or /vN import path and find an on-disk layout that they expect without any knowledge of modules). The approach does not copy go.mod files at all. (Before starting, I was thinking I would need to copy go.mod files to trigger "minimal module awareness", which is a transitional mechanism within cmd/go, but after some experimentation / thinking, I concluded it was not needed based on materializing the /v2). For tests, I am using the script-driven test facility that the core Go team created for testing cmd/go, which is well tuned to testing the various permutations of modules, including it lets you relatively easily set up go.mod files and source files, run commands and check results. (Some more details of the capability here: https://github.com/golang/go/blob/master/src/cmd/go/testdata/script/README). * modules support: adjust wording based on first-pass review, and update vendor test (#3) * travis: attempt to install cmd/testscript * testing: initial check of 'go-fuzz-build -h' output to see if any testscripts work * testing: change name of first test script * travis: check if the 'testscript' cmd seems to function at all. * testing: use the exec command to invoke 'go-fuzz-build -h' * testing: create initially failing test case for v2 modules * travis: run testscript for v2 modules * testing: remember to use 'exec' to invoke go-fuzz-build * testing: adjust comments in v2 module testscript * go-fuzz-build: preliminary module support * testing: flip from an expected failure to an expected pass for a module test * go-fuzz-build: update comments for preliminary modules support * go-fuzz: preliminary modules support for go-fuzz itself * travis: go-fuzz-defs sanity check * travis: additional temp sanity check * travis: add more comments, some additional sanity checks in case we need to debug in future * travis: s/go-fuzz-deps/go-fuzz-dep/ * testing: validate behavior of v2 modules * testing: renamed fuzz_help.txt * testing: update travis to use the renamed testscripts * travis: conditionally set GO111MODULE=auto, and test Go 1.13 rather than 1.11 * testing: validate modules outside GOPATH * testing: validate modules inside GOPATH * testing: invoke timeout properly * testing: preliminary test for vendored modules (not yet supported) * testing: validate how go-fuzz-dep and go-fuzz-defs are found * travis: run the latest testscripts for modules testing * testing: fix typo in mod_outside_gopath.txt file name * travis: fix typo in file name * go-fuzz-build: detect -mod=vendor * testing: validate we get a proper error for -mod=vendor * readme: describe preliminary modules support * readme: correct typo in modules support * readme: adjust modules support section * readme: tweak wording based on PR review * go-fuzz: tweak comment wording by removing "preliminary modules support" * go-fuzz-build: tweak comment wording by removing "preliminary modules support" * mod_vendor.txt: avoid triggering Go 1.14 auto detection of a vendor directory * readme: update wording based on review * readme: remove a blank line to re-trigger travis
- Loading branch information
Showing
10 changed files
with
593 additions
and
21 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| # Start by seeing if anything works at all. | ||
| # This is intended to excercise travis and the testscript cmd invoking go-fuzz. | ||
| # If this fails, likely everything else will fail too, but if so might be easier to | ||
| # start by troubleshooting this. | ||
|
|
||
| # Ask go-fuzz-build to emit its help message, which should be a failing status code. | ||
| ! exec go-fuzz-build -h | ||
| stderr 'Usage of.*go-fuzz-build' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| # These steps validate that go-fuzz/go-fuzz-dep and go-fuzz/go-fuzz-defs are found, | ||
| # along with validating how they are currently found, and where they end up being located. | ||
|
|
||
| # Enter a simple module with a fuzz function. | ||
| # Note that we are outside of GOPATH, so the presence of the 'go.mod' here will | ||
| # enable module-module for cmd/go (which is true in Go 1.11-1.13, and likely will be true in 1.14 as well). | ||
| cd testmod | ||
|
|
||
| # Sanity check the module seems well formed. | ||
| exec go list -m all | ||
| stdout '^example.com/testmod$' | ||
| exec go build | ||
|
|
||
| # Sanity check go-fuzz is not in our go.mod, nor reported by 'go list -m all'. | ||
| ! grep 'github.com/dvyukov/go-fuzz' go.mod | ||
| exec go list -m all | ||
| ! stdout 'github.com/dvyukov/go-fuzz' | ||
|
|
||
| # Ask go-fuzz-build to build, including specifying the fuzz function for mod. | ||
| exec go-fuzz-build -func=FuzzMod | ||
| exists testmod-fuzz.zip | ||
|
|
||
| # Sanity check github.com/dvyukov/go-fuzz was added by go-fuzz-build | ||
| # to our go.mod and is now visible via 'go list -m all'. | ||
| # This is not necessarily a requirement for all time, | ||
| # but this is the way the current modules approach for go-fuzz works. | ||
| # This is important to make sure the go-fuzz-dep source code is | ||
| # findable by go-fuzz-build (when it invokes 'go list'). | ||
| grep -count=1 '^require github.com/dvyukov/go-fuzz v[^ ]+ // indirect$' go.mod | ||
| exec go list -m all | ||
| stdout '^example.com/testmod$' | ||
| stdout '^github.com/dvyukov/go-fuzz v[^ ]+$' | ||
|
|
||
| # Also output directories for go-fuzz-defs and go-fuzz-dep | ||
| # in case we need to debug this at some point in the future, | ||
| # or in case cmd/go or go-fuzz change in the future | ||
| # in some way that moves these out of $GOPATH/pkg/testmod or | ||
| # otherwise alters where these are located. | ||
| # The exact location might not be critical, but we should be aware of what it is, | ||
| # so capture the location here in this test so we know if it changes later. | ||
| exec go list -f {{.Dir}} github.com/dvyukov/go-fuzz/go-fuzz-defs | ||
| stdout 'gopath.pkg.mod.github.com.dvyukov.go-fuzz@v[^ ]+.go-fuzz-defs$' | ||
| exec go list -f {{.Dir}} github.com/dvyukov/go-fuzz/go-fuzz-dep | ||
| stdout 'gopath.pkg.mod.github.com.dvyukov.go-fuzz@v[^ ]+.go-fuzz-dep$' | ||
|
|
||
| # Define a simple module 'mod' that has a fuzz function. | ||
|
|
||
| -- testmod/go.mod -- | ||
| module example.com/testmod | ||
|
|
||
| -- testmod/fuzz.go -- | ||
| package testmod | ||
|
|
||
| func FuzzMod(data []byte) int { | ||
| return 0 | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,99 @@ | ||
| # These steps validate we can fuzz inside of GOPATH | ||
| # with GO111MODULE on and off. | ||
|
|
||
| # Here, we purposefully do not test with GO111MODULE auto and unset, | ||
| # because the meaning of auto within GOPATH has changed with | ||
| # Go 1.13, and the meaning of unset will likely change in Go 1.14 | ||
| # (and some chance auto could change meaning again in Go 1.14). | ||
| # However, the Go 1.13 and 1.14 changes ultimately translate | ||
| # to what external conditions enable or disable module mode | ||
| # for cmd/go. Testing here with GO111MODULE on and off allows | ||
| # us to explicitly test with cmd/go's module mode enabled and disabled in GOPATH, | ||
| # which is our current goal here. (These are not cmd/go tests after all). | ||
|
|
||
| # The foo module being fuzzed depends on a 'replace' | ||
| # in its go.mod to find one of its dependencies, and that dependency bar | ||
| # is located *outside* of GOPATH, which is a way to check that | ||
| # module-mode is required to compile successfully. | ||
| # (Other go-fuzz tests validate that non-module targets work in GOPATH with | ||
| # cmd/go's module mode disabled; that has been the status quo). | ||
|
|
||
| # Enter a simple module with a fuzz function. | ||
| # This is inside GOPATH. | ||
| cd gopath/src/example.com/foo | ||
|
|
||
| # Copy a pristine go.mod file. | ||
| cp go.mod_PRISTINE go.mod | ||
|
|
||
| # First, we test with GO111MODULE=on, which will likely be the default in Go 1.14. | ||
| env GO111MODULE=on | ||
|
|
||
| # Sanity check the module seems well formed. | ||
| exec go list -m all | ||
| stdout '^example.com/foo$' | ||
| exec go build | ||
|
|
||
| # Ask go-fuzz-build to build, including specifying the fuzz function for mod. | ||
| exec go-fuzz-build -func=FuzzMod | ||
| exists foo-fuzz.zip | ||
|
|
||
| # Validate we can start fuzzing. | ||
| # Note that 'timeout(1)' will error here, so we preface the invocation with '!'. | ||
| # For travis on Windows, we install 'timeout(1)' as part of our travis setup steps. | ||
| # To test this locally on Windows, you might need to change 'timeout' to '\cygwin64\bin\timeout' or similar. | ||
| ! exec timeout 5 go-fuzz -procs=1 -func=FuzzMod | ||
| stderr 'workers: \d+, corpus: ' | ||
|
|
||
| # Clean up. | ||
| cp go.mod_PRISTINE go.mod | ||
| rm foo-fuzz.zip | ||
|
|
||
| # Second, we test with GO111MODULE=off. | ||
| # The meaning of this is unlikely to change in Go 1.14, | ||
| # altough in some (distant?) future, GO111MODULE=off might | ||
| # no longer be supported. | ||
| env GO111MODULE=off | ||
|
|
||
| # Confirm 'go list -m' and 'go build' fail. | ||
| ! exec go list -m all | ||
| ! exec go build | ||
|
|
||
| # Confirm go-fuzz-build fails. | ||
| ! exec go-fuzz-build -func=FuzzMod | ||
| ! exists foo-fuzz.zip | ||
|
|
||
| # Clean up (mainly in case we later add another test below). | ||
| cp go.mod_PRISTINE go.mod | ||
| rm foo-fuzz.zip | ||
|
|
||
| # Define two modules. | ||
| # example.com/foo has a fuzz function, and depends on example.com/bar. | ||
| # foo is inside GOPATH, and bar is outside of GOPATH. | ||
|
|
||
| -- gopath/src/example.com/foo/go.mod_PRISTINE -- | ||
| module example.com/foo | ||
|
|
||
| require example.com/bar v0.0.0 | ||
|
|
||
| replace example.com/bar => ../../../../bar | ||
|
|
||
| -- gopath/src/example.com/foo/fuzz.go -- | ||
| package foo | ||
|
|
||
| import "example.com/bar" | ||
|
|
||
| func FuzzMod(data []byte) int { | ||
| bar.Bar() | ||
| return 0 | ||
| } | ||
|
|
||
| -- bar/go.mod -- | ||
| module example.com/bar | ||
|
|
||
| -- bar/bar.go -- | ||
| package bar | ||
|
|
||
| func Bar() string { | ||
| return "hello from bar" | ||
| } | ||
|
|
Oops, something went wrong.