cmd/go: document that go mod download -json uses exit code 1 when at least one JSON entry contains non-zero Error

[dmitshur]

go mod download -json documentation says:

[...] The -json flag causes download to print a sequence of JSON objects to standard output, describing each downloaded module (or failure), corresponding to this Go struct:

type Module struct {
	...
	Error string // error loading module
	...
}

What did you do?

I tried to execute go mod download -json and interpret its results programmatically.

What did you expect to see?

I expected non-zero exit code to mean a fatal problem occurred and that I should not expect valid JSON output.

What did you see instead?

There are 3 possible scenarios:

1. When there are no errors, go mod download -json emits exit code 0 and prints JSON.

2. When there are errors downloading some modules, go mod download -json emits exit code 1 and prints JSON that has some entries with non-empty Error fields.

   Example

   
   

   An example of such an error is when GOPROXY is set to off; it results in "module lookup disabled by GOPROXY=off" errors:

   $ cat go.mod
   module example.com/m
   require golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a
   $ echo $GOPROXY
   off
   $ go mod download -json
   {
   	"Path": "golang.org/x/sys",
   	"Version": "v0.0.0-20190215142949-d0b11bdaac8a",
   	"Error": "module lookup disabled by GOPROXY=off"
   }
   $ echo $?
   1
   

3. When there are some other type of errors, go mod download -json emits exit code 1 and does not print JSON.

   Example

   
   

   Examples of other errors include a syntax error in the current go.mod file, or a replace directive pointing to a directory without a go.mod file.

   $ cat go.mod
   bad
   $ go mod download -json
   go: errors parsing go.mod:
   /tmp/issue35380/go.mod:1: unknown directive: bad
   $ echo $?
   1
   

In two of the scenarios the exit code is 1. One of them emits JSON. That is challenging for a tool that needs to parse the output; it's hard to be sure it's safe to expect/parse JSON on stdout when exit code is non-zero.

/cc @jayconrod @bcmills @marwan-at-work

[dmitshur]

I checked scenario 3 just now, and the error it emits goes to stderr. Its stdout is empty, which is a valid sequence of JSON objects (and so it will not produce an error when trying to JSON decode it).

[dmitshur]

I guess the remaining usability problem is that it's still necessary for caller to parse as JSON even when exit code is 1. And exit code 1 may mean "there are JSON objects to be read and processed", or it could mean "there was a fatal error that prevented go mod download from starting", and it's not easy to tell those apart.

[heschi]

Related: #34485.

[bcmills]

@dmitshur, the easy way to tell those two cases apart should be to check whether the output to stderr was non-empty. (If there is any data written to stdout, then it should be valid, but we do not guarantee to write anything on error.)

[dmitshur]

I wanted to follow up here with an update. After reading the relevant source code very closely:

go/src/cmd/go/internal/modcmd/download.go

Lines 160 to 171 in d2de9bd

	if *downloadJSON {
	for _, m := range mods {
	b, err := json.MarshalIndent(m, "", "\t")
	if err != nil {
	base.Fatalf("%v", err)
	}
	os.Stdout.Write(append(b, '\n'))
	if m.Error != "" {
	base.SetExitStatus(1)
	}
	}
	} else {

I was able to actually simplify the code that consumed output from go mod download -json. I made use of the fact that if exit code is 0, then I know the Error field will always be empty, and if it's non-zero, then there will be at least one module with a non-zero Error field.

While I still think a different behavior when -json flag may have slightly preferable properties, I doubt that the benefits would outweigh the cost of making a change. Perhaps this issue should be about documenting the current behavior, so that people can rely on it without having to read the source code.

[perillo]

I think returning exit status 1 is the correct thing to do, since there was an error.
The problem is that go list and go mod download do different things and it can be confusing.

The reason it is the right thing to do is that, in case of exit status 1, the caller can simply abort the program ignoring the returned data.

A counter example is golang.org/x/tools/go/packages, where after calling Load, the caller needs to check (usually):

1. If err != nil, in case of catastrophic error
2. Call PrintErrors to report the errors to the user (in case of a cli tool), and check if it returns a number > 0

Another counter example is my wrapper of go mod download
https://github.com/perillo/gocmd/blob/master/modfetch/modfetch.go
where, to keep my api consistent (if there are errors, they are in the error value), I need to iterate over the returned modules.