cmd/go: ignore 'go.mod' files in 'testdata' subdirectories when computing module boundaries

[bcmills]

Some existing tests of Go tools make use of testdata directories containing a self-contained GOPATH.¹

For those tests to work as expected in module mode, the testdata directories may need to contain go.mod files, but in order for the tests to pass when run from within other modules, those go.mod files should be included in the same module as the test — they should not actually define separate modules.

(There's just one caveat that I know of: at the moment, adding a go.mod file is the easiest way to prune out files whose paths are invalid in the module zip format (see #26672).)

CC: @rsc @myitcv @thepudds

¹ e.g., x/tools/cmd/fiximports/testdata

[myitcv]

but in order for the tests to pass when run from within other modules, those go.mod files should be included in the same module as the test — they should not actually define separate modules.

This bit I don't quite understand, specifically "should be included in the same module as the test". Could you clarify?

That said, with tests that do require a "full setup" (i.e. a go.mod etc) is it not just as easy to create a temporary directory and programmatically create a go.mod within it? Seems a bit easier/cleaner than offering an exemption to the rule?

Or just use _testdata? (I realise this might be contrary to suggested best practice, but again maybe easier to offer a variation on this best practice with the introduction of modules?)

[bcmills]

This bit I don't quite understand, specifically "should be included in the same module as the test". Could you clarify?

If I add a require golang.org/x/tools to my go.mod file and then run go test golang.org/x/tools/cmd/fiximports, the test should pass. That implies that the source directory within the module cache must include the entire testdata directory.

That said, with tests that do require a "full setup" (i.e. a go.mod etc) is it not just as easy to create a temporary directory and programmatically create a go.mod within it?

That is a straightforward workaround, but it splits the test input across two places (the testdata directory and the code to generate the go.mod files), and makes it so that you can't just cd testdata and export GOPATH=$(pwd) to investigate the (accurate) contents of the test directory.

[gopherbot]

Change https://golang.org/cl/137815 mentions this issue: all: set GO111MODULE=off for tests that use GOPATHs in testdata.

[bcmills]

Note from @rsc: if we do this, we should also reject module paths that include a testdata component.

[rsc]

I don't know that we should introduce a special case here. We have a very simple rule at the moment. Maybe we should figure out how to work within that rule.

[bcmills]

Maybe we should figure out how to work within that rule.

The stack of changes starting at https://golang.org/cl/163209 works within that rule, but it's tedious (because we have to explicitly copy the modules in each test) and doesn't interoperate well with IDEs (because the import paths used by the files in testdata don't match the import paths that will be implied by the injected go.mod file).

Plus, the tests end up a lot less efficient because we have to copy the entire contents of the directory: os.Symlink is not portable to plan9, and module testdata directories are often read-only (see also #28387).

[bcmills]

Plus, the tests end up a lot less efficient

I spoke too soon: it's easy enough to fall back to copying if os.Symlink is not supported. So we end up needing to copy the branching structure of the directory tree and make some symlinks, but not copy individual files.

[dmitshur]

I don't know that we should introduce a special case here.

cmd/go documentation says the following about "testdata":

The go tool will ignore a directory named "testdata", making it available to hold ancillary data needed by the tests.

Directory and file names that begin with "." or "_" are ignored by the go tool, as are directories named "testdata".

If creating a go.mod file inside a "testdata" directory defines a new module and removes files from the original module, isn't that a case of the go tool not ignoring the "testdata" directory fully?