Cannot get latest version: module contains a go.mod file, so module path should be github.com/sensu/sensu-go/v5

[KateGo520]

Background

The github.com/sensu/sensu-go uses Go modules and the current release version is v5. And it’s module path is "github.com/sensu/sensu-go", instead of "github.com/sensu/sensu-go/v5". It must comply with the specification of "Releasing Modules for v2 or higher" available in the Modules documentation. Quoting the specification:

A package that has opted in to modules must include the major version in the import path to import any v2+ modules
To preserve import compatibility, the go command requires that modules with major version v2 or later use a module path with that major version as the final element. For example, version v2.0.0 of example.com/m must instead use module path example.com/m/v2.
https://github.com/golang/go/wiki/Modules#releasing-modules-v2-or-higher

Steps to Reproduce

GO111MODULE=on, run go get targeting any version >= v5.12.0 of the sensu/sensu-go:

$ go get github.com/sensu/sensu-go@v5.20.0
go: finding github.com/sensu/sensu-go v5.20.0
go: finding github.com/sensu/sensu-go v5.20.0
go get github.com/sensu/sensu-go@v5.20.0: github.com/sensu/sensu-go@v5.20.0: invalid version: module contains a go.mod file, so major version must be compatible: should be v0 or v1, not v5

run go get github.com/sensu/sensu-go, the version will stuck in v5.10.1:

$go get github.com/sensu/sensu-go
go: downloading github.com/sensu/sensu-go v5.10.1
go: github.com/sensu/sensu-go upgrade => v5.10.1

SO anyone using Go modules will not be able to easily use any newer version of sensu/sensu-go.

Solution

1. Kill the go.mod files, rolling back to GOPATH.

This would push them back to not being managed by Go modules (instead of incorrectly using Go modules).
Ensure compatibility for downstream module-aware projects and module-unaware projects projects

I see these dependencies in your go.mod file, which need modle awareness. So you'd better not use third-party tools(such as: Dep, glide, govendor…).

github.com/go-resty/resty/v2 v2.1.0 
github.com/robfig/cron/v3 v3.0.1 

You also need to update the import path to:

import github.com/go-resty/resty/…
import github.com/robfig/cron/…

2. Fix module path to strictly follow SIV rules.

Patch the go.mod file to declare the module path as github.com/sensu/sensu-go/v5 as per the specs. And adjust all internal imports.
The downstream projects might be negatively affected in their building if they are module-unaware (Go versions older than 1.9.7 and 1.10.3; Or use third-party dependency management tools, such as: Dep, glide,govendor…).

[*] You can see who will be affected here: [1 module-unaware user, i.e., portertech/sensu-graphite-handler]
https://github.com/search?q=%22github.com%2Fsensu%2Fsensu-go%22+filename%3AGodeps.json+filename%3Avendor.conf+filename%3Avendor.json+filename%3Aglide.toml+filename%3AGodep.toml&type=Code

If you don't want to break the above repos. This method can provides better backwards-compatibility.
Release a v2 or higher module through the major subdirectory strategy: Create a new v5 subdirectory (github.com/sensu/sensu-go/v5) and place a new go.mod file in that subdirectory. The module path must end with /v5. Copy or move the code into the v5 subdirectory. Update import statements within the module to also use /v5 (import "github.com/sensu/sensu-go/v5/…"). Tag the release with v5.x.y.

3. Suggest your downstream module users use hash instead of a version tag.

If the standard rule of go modules conflicts with your development mode. Or not intended to be used as a library and does not make any guarantees about the API. So you can’t comply with the specification of "Releasing Modules for v2 or higher" available in the Modules documentation.
Regardless, since it's against one of the design choices of Go, it'll be a bit of a hack. Instead of go get github.com/sensu/sensu-go@version-tag, module users need to use this following way to get the sensu/sensu-go:
(1) Search for the tag you want (in browser)
(2) Get the commit hash for the tag you want
(3) Run go get github.com/sensu/sensu-go@commit-hash
(4) Edit the go.mod file to put a comment about which version you actually used
This will make it difficult for module users to get and upgrade sensu/sensu-go.

[*] You can see who will be affected here: [89 module users, e.g., nixwiz/sensu-go-flowdock-handler, owenbattye/atlassian-checks, agm650/sensu-entities-status]
https://github.com/search?q=%22github.com%2Fsensu%2Fsensu-go%22+filename%3Ago.mod&type=Code

Summary

You can make a choice to fix DM issues by balancing your own development schedules/mode against the affects on the downstream projects.

For this issue, Solution 2 can maximize your benefits and with minimal impacts to your downstream projects the ecosystem.

References

• https://github.com/golang/go/wiki/Modules#semantic-import-versioning
• https://golang.org/cmd/go/#hdr-Module_compatibility_and_semantic_versioning
• https://github.com/golang/go/wiki/Modules#releasing-modules-v2-or-higher

[echlebek]

Thanks @KateGo520! We are aware of this issue, but the way you have summarized it, including the paths forward, is very valuable.

Since sensu-go is primarily an application, there has not been a huge driver to make it importable as a module, despite the benefits of doing so. It's been on my bucket list since the advent of Go modules.

Unfortunately before modules existed, we made some choices about our semver and our package paths that conflict with the design of Go modules.

There are also parts of the code base that we would like to be able to make breaking code changes in more frequently, without doing major version bumps. (The CLI tools, for instance)

We've discussed the idea of separating out the API components of Sensu so that users who want to use concepts from the system have an easier time. To my mind, this would include things like the "core/v2" API, the store interfaces, and any HTTP client interfaces that might exist. It would not include things like the scheduler, bits and pieces of the CLI, or parts of the sensu-agent that really aren't meant for public consumption.

This story would have been simpler had we made use of internal packages, but the project has a fairly scrappy history, and such considerations were not made.

Your issue will make it easier to discuss where we want to take things in the future, so thanks again, and stay tuned for updates.

[echlebek]

Today I published two modules in sensu-go. They are nested in the repository and so we were able to create them with the nested tagging scheme that Go modules supports.

They are github.com/sensu/sensu-go/api/core/v2@v2.0.0 and github.com/sensu/sensu-go/types@v0.1.0.

These changes were mainly done to support work being done in sensu-sdk, which needs to use Sensu's core types.

I believe we can solve this issue by adopting similar techniques throughout the project for its other packages. If we made modules for each top level package, then we could delete the go.mod and go.sum files at the repository root.

Q: What if you just deleted the go.mod and go.sum at the repository root without creating new modules elsewhere in the project?
A: We can't simply delete those files without representing their dependencies elsewhere, as it would break build reproducibility.

Q: Will you consider publishing a v5 or v6 module for sensu-go?
A: Probably not. We don't wish to support the entire code base as a module, especially a v2+ module. We wish to retain implementation flexibility in non-API parts of the system, and break things from time to time without publishing new modules.

We don't consider sensu-go's semantic version to pertain to source code APIs, but rather things like HTTP APIs and storage layouts. The v5.x.x semantic version is for our users, not developers. Therefore it makes sense to avoid publishing a module at the repository root, and instead publish modules for each top level package in the project. This should hopefully be a pretty sensible approach, as the repository root does not even have any Go source files. However, we'll need to investigate how these module versions will be maintained and released, so it will probably take some time.

[KateGo520]

@echlebek Thank very much for your reply. So you want to create sub-module. But maintaining sub-modules is a big job. Maybe this issue can help you:
gomodule/redigo#440 (comment)

Having the go.mod files in subdirectories means that they are effectively separate modules from the original github.com/gomodule/redigo, are independently versioned, and require separate version tags (like redisx/v1.0.0 and redis/v1.0.0 respectively).

gomodule/redigo#440 (comment)

to be fair, the Wiki recommends against multi-module repos, and https://blog.golang.org/migrating-to-go-modules explicitly says “If you have existing version tags, you should increment the minor version.”