x/vgo: look up branches (not just tags) in version fixer

[markbates]

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.10 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/markbates/Library/Caches/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/markbates/Dropbox/development/gocode"
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/6m/vw2ck7mj32z5f63wpgfw5qk80000gn/T/go-build653177088=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I wanted to use a branch with vgo, but couldn't.

What did you expect to see?

I expected to be able to do something like

module "github.com/gobuffalo/updater"

require (
	"github.com/gobuffalo/buffalo" development
	"github.com/gobuffalo/envy" v1.4.0
	"github.com/markbates/inflect" v0.0.0-20180217223538-80fa1ea41b0d
	"github.com/markbates/pop" v0.0.0-20180220131018-2a6ad5481507
	"github.com/sirupsen/logrus" v1.0.4
	"golang.org/x/crypto" v0.0.0-20180219163459-432090b8f568
)

What did you see instead?

vgo: errors parsing go.mod:
/Users/markbates/Dropbox/development/gocode/src/github.com/gobuffalo/updater/go.mod:4: invalid module version "development": unexpected status (https://api.github.com/repos/gobuffalo/buffalo/git/refs/tags/development): 404 Not Found

Why is this useful?

During the course of development it is common to work on branches while developing features. When developing against these branches it would be nice to be able to continue using vgo against a branch.

As an example, https://github.com/gobuffalo/buffalo uses a development between releases so as not to pollute the master branch with breaking changes and other code not ready for "release". Since go get pulls master this could be a very unwelcome experience.

In the case of Buffalo the development branch is a long lived, however, there are plenty of times when you are developing a feature, bug fix, etc... on a branch and need to deploy an app to test it production, or other environment. If vgo doesn't support branches then it becomes difficult to work against these branches.

[kardianos]

I can't speak for anyone else, but I don't think this will fly.

I think what you would want to use is a replace directive in go.mod:
From https://research.swtch.com/vgo-tour

replace "rsc.io/quote" v1.5.2 => "../quote"

You can then develop locally. This brings up a different point where to enable this development locally, but not affect what gets pushed to git, it may be useful to have go go.mod files. One for the specific user, and one for the repo with git ignoring the hypothetical go-user.mod file where you have the development replace statement.

[huguesb]

@markbates Can't you simply manually specify a commit inside the development branch? It's admittedly awkward to have to manually form the proper time-hash string but that should solve your testing use case.

In your above case, were you expecting that vgo would pick up the latest commit on the development and update go.mod accordingly (probably doable, although vgo get would arguably be a better place for such a corner case), or were you expecting vgo to always track the HEAD commit of that branch (clearly against vgo's Minimal Version Selection approach)?

[markbates]

@kardianos Pointing to a folder on disk, while it has usefulness, it wouldn't work on a CI server. If I'm going to copy one or more packages into my app so I can point to a folder, I would be better putting them into the vendor folder. But now I'm managing the vendor folder directly.

@huguesb You could point to a particular commit, but there's a constant maintenance around that.

I feel like this is a very common use case as every package manager I've used does it. Pointing at forks, branches, etc... is fairly common when writing complex applications.

[kardianos]

It would work, you would need to git clone in a sibling folder first.

[markbates]

@kardianos why does that seem a better solution than allowing branches? I'm not trying to be stubborn or hard headed, I just don't get why this seems so foreign when it's a common thing to do and is supported by almost every current package manager.

I get that vgo is taking a different approach to the problem, but we can't throw out common development practices because it's not perfect. The community really needs vgo, but it has to support how people actually work.

[huguesb]

@markbates Is there really? Your stated use case was deploying an app for testing. In such a case it should be pretty easy to script the update to go.mod (either vgo get will support that or you can write your own one-liner bash script) as part of the build/deploy process.

[huguesb]

The community really needs vgo, but it has to support how people actually work.

It doesn't have to support every single workflow. In fact it's arguably good and not unusual for toolchains to enforce sane defaults. As rsc explains in his blog posts, Minimal Version Selection matters for two very important reasons:

• it avoids all kinds of weird corner cases and conflicts in real-world dependency graphs (crucially if a package builds on day 1 it will keep building regardless of upstream changes, which isn't possible if you unconditionally track an upstream branch)
• it makes it possible to efficiently solve dependency constraints (afaik tool speed has always been an important metric for the go team)

[rsc]

Hi @markbates,

One of the key points about go.mod is that it serves as both manifest and lock file. Because we want builds to be reproducible by default (and on top of that high-fidelity), the revisions named in go.mod need to be fixed identifiers, not branch names, whose job is to change meaning. (I know that semver tags can be force pushed to change meaning, but that's strongly discouraged; not so with branch names, whose whole point is to change meaning from day to day.)

That said, I understand your point about needing an easy way to say "I want to build against the current development branch, whatever that means." Right now if you had a tag named development, it would work to say:

vgo get github.com/gobuffalo/buffalo@development

or to put

"github.com/gobuffalo/buffalo" development

into your go.mod file, and vgo would look up that tag and replace it in the go.mod with the corresponding pseudo-hash for the commit that development means right now. If "development" moves, you could run the "vgo get" command again and vgo would once again update to the new commit. But go.mod would always say a specific commit. Of course, you don't have a tag named development; you have a branch named development. And vgo doesn't look for branch names when it's resolving those kinds of identifiers to pseudo-versions. But I think it probably should.

If we made it the case that you could always run

vgo get github.com/gobuffalo/buffalo@development

to mean "use the current development branch HEAD", and you just had to repeat that command when you wanted to try a newer commit, would that address your workflow?

Thanks.

[snowzach]

Researching a similar use case. Running vgo get with the current branch name makes sense to me.

[rsc]

We should definitely do the branch lookup same as tag lookup.

[markbates]

@rsc that makes complete sense, definitely. that would definitely solve this use case. That would be great! Thanks.

[gopherbot]

Change https://golang.org/cl/105216 mentions this issue: cmd/go/internal/modfetch: allow branch name as rev identifier