Upgrade golang to v1.16

[mctofu]

This allows use of the new embed package added in 1.16.

There's also a few notable updates for modules:

The go mod vendor and go mod tidy subcommands now accept the -e flag, which instructs them to proceed despite errors in resolving missing packages. (noted here)

Module-aware mode is enabled by default, regardless of whether a go.mod file is present in the current working directory or a parent directory. More precisely, the GO111MODULE environment variable now defaults to on.

We could omit GO111MODULE=on from the helper scripts.

[xlgmokha]

#3235 should help fix the failing tests.

[mctofu]

The tests are showing a behavior change when there is a reference to a package that has been renamed. In the test we are expecting that go mod tidy would complain about this but we are now seeing it fail prior to that at the go get -d step.

$ go get -d
github.com/dependabot/vgotest imports
        github.com/googleapis/gnostic/OpenAPIv2: cannot find module providing package github.com/googleapis/gnostic/OpenAPIv2

[thepwagner]

🎉
Aside: https://github.com/thepwagner/action-update-dockerurl supports the Dockerfile syntax we're using for this dependency, and https://github.com/thepwagner/action-update-brewformula/blob/main/brew/golang.go is gnarly, but handles parsing https://golang.org/dl/?mode=json - we could automate these bumps!

I'd dig some tests that cover the new retract directive too, e.g. to ensure Dependabot won't propose an update to a retracted version. As noted, there are a lot of module goodies in 1.16.

[mctofu]

I pushed an update to fix the failing test that is worth re-reviewing: f94051a.

The issue is that go get -d now fails if the code references a package that can't be found so I've reverted to our previous behavior of raising DependencyFileNotResolvable (#2627). However, since the new error message doesn't differentiate between the module not existing vs existing but not providing the package I also had to make some adjustments to prevent the error from raising GitDependenciesNotReachable instead. I found that running go list -m -versions #{repo_path} was a workable way to test that a module's repo was reachable and was potentially lighter weight than using go get since it doesn't download the module's dependencies.

[mctofu]

I'd dig some tests that cover the new retract directive too, e.g. to ensure Dependabot won't propose an update to a retracted version.

@thepwagner: I added a test that we won't strip the retract directive from a go.mod. Avoiding retracted versions isn't working yet. I think we need to update the extracted go module code to get the latest behavior updates there.

[thepwagner]

I think we need to update the extracted go module code to get the latest behavior updates there.

I assumed since 0.4.1 included golang/mod@c0d644d#diff-320fe6a1bc7a6321d706e6c7fc95d457cd5ea1b13425b320d56897c78074b350 , we'd have the necessary bits to update the helpers.

[mctofu]

I assumed since 0.4.1 included golang/mod@c0d644d#diff-320fe6a1bc7a6321d706e6c7fc95d457cd5ea1b13425b320d56897c78074b350 , we'd have the necessary bits to update the helpers.

Aha! I had been hoping that modfetch.Lookup("direct", args.Dependency.Name) would have already filtered retracted versions in 1.16. That doesn't seem to be the case so we will need to filter on our end.