Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/vgo: go-import mod meta tag is not backwards compatible with old go #24751

Closed
myitcv opened this issue Apr 7, 2018 · 5 comments
Closed

x/vgo: go-import mod meta tag is not backwards compatible with old go #24751

myitcv opened this issue Apr 7, 2018 · 5 comments
Milestone

Comments

@myitcv
Copy link
Member

myitcv commented Apr 7, 2018

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.10.1 linux/amd64 vgo:2018-02-20.1

Specifically vgo is as of 5db81b5cd6ab5ee1dd1b1b0ee0843f194ab7b908

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/myitcv/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/myitcv/.mountpoints/x"
GORACE=""
GOROOT="/home/myitcv/gos"
GOTMPDIR=""
GOTOOLDIR="/home/myitcv/gos/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build749051986=/tmp/go-build -gno-record-gcc-switches"
VGOMODROOT=""

What did you do?

Everything works just fine with vgo:

cd `mktemp -d`
export GOPATH=$PWD
mkdir blah
cd blah/
echo 'module "rubbish.com/blah"' >> go.mod
echo 'package blah
import _ "myitcv.io/blah2"
' >> blah.go
vgo build

works fine and gives the output:

vgo: resolving import "myitcv.io/blah2"
vgo: finding myitcv.io/blah2 (latest)
vgo: adding myitcv.io/blah2 v0.0.0-20180407140204-f591dcc2d6822ecafa384190ef45f5c333f4f82d
vgo: finding myitcv.io/blah2 v0.0.0-20180407140204-f591dcc2d6822ecafa384190ef45f5c333f4f82d
vgo: downloading myitcv.io/blah2 v0.0.0-20180407140204-f591dcc2d6822ecafa384190ef45f5c333f4f82d

But with old go get:

cd `mktemp -d`
export GOPATH=$PWD
go get myitcv.io/blah2

What did you expect to see?

Success.

What did you see instead?

package myitcv.io/blah2: unrecognized import path "myitcv.io/blah2" (parse https://myitcv.io/blah2?go-get=1: multiple meta tags match import path "myitcv.io/blah2")

It was my understanding that the mod VCS meta go-import should be able to co-exist with the current git (or equivalent) ones in order that a package can be go get-able as well as vgo-compatible. E.g.:

$ curl https://myitcv.io/blah2?go-get=1

<!DOCTYPE html>
<html>
<head>
        <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
        <meta name="go-import" content="myitcv.io git https://github.com/myitcv/x">
        <meta name="go-import" content="myitcv.io/blah2 mod https://raw.githubusercontent.com/myitcv/pubx/master">
        <meta name="go-source" content="myitcv.io https://github.com/myitcv/x/wiki https://github.com/myitcv/x/tree/master{/dir} https://github.com/myitcv/x/blob/master{/dir}/{file}#L{line}">
        <meta http-equiv="refresh" content="0; url=https://godoc.org/myitcv.io/blah2">
</head>
<body>
Redirecting to docs at <a href="https://godoc.org/myitcv.io/blah2">godoc.org/myitcv.io/blah2</a>...
</body>
</html>

But old go get doesn't ignore unknown VCS's, and so the two meta tags "clash".

Or have I got this wrong?

If I have got this wrong, how can a repo support both?

As a side note, the corresponding test for matchGoImport looks a bit suspicious; it doesn't actually assert that the actual error (when there is one) matches the error we want; there are a number of mismatches in the expectations vs the actual result. And there is also no expectation for this scenario of multiple matches.

@gopherbot gopherbot added this to the vgo milestone Apr 7, 2018
@myitcv
Copy link
Member Author

myitcv commented Apr 7, 2018

Since raising this issue (that report a problem with myitcv.io/gogenerate) I've had to revert that package into a working state (so that I don't break things).

I've therefore updated this issue to use a new dummy package (myitcv.io/blah2), specific to this issue, so that the problem remains static.

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/109340 mentions this issue: cmd/go: add minimal module-awareness for legacy operation

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/114500 mentions this issue: [release-branch.go1.10] cmd/go: add minimal module-awareness for legacy operation

gopherbot pushed a commit that referenced this issue May 30, 2018
…cy operation

We want authors to be able to publish code that works with both
the current standard go command and the planned new go command
support for modules. If authors have tagged their code v2 or later,
semantic import versioning means the import paths must include a
v2 path element after the path prefix naming the module.
One option for making this convention compatible with original go get
is to move code into a v2 subdirectory of the root.
That makes sense for some authors, but many authors would prefer
not to move all the code into a v2 subdirectory for a transition and
then move it back up once we everyone has a module-aware go command.

Instead, this CL teaches the old (non-module-aware) go command
a tiny amount about modules and their import paths, to expand
the options for authors who want to publish compatible packages.
If an author has a v2 of a package, say my/thing/v2/sub/pkg,
in the my/thing repo's sub/pkg subdirectory (no v2 in the file system path),
then old go get continues to import that package as my/thing/sub/pkg.
But when go get is processing code in any module (code in a tree with
a go.mod file) and encounters a path like my/thing/v2/sub/pkg,
it will check to see if my/thing/go.mod says "module my/thing/v2".
If so, the go command will read the import my/thing/v2/sub/pkg
as if it said my/thing/sub/pkg, which is the correct "old" import path
for the package in question.

This CL will be back-ported to Go 1.10 and Go 1.9 as well.

Once users have updated to the latest Go point releases containing
this new logic, authors will be able to update to using modules
within their own repos, including using semantic import paths
with vN path elements, and old go get will still be able to consume
those repositories.

This CL also makes "go get" ignore meta go-import lines using
the new "mod" VCS type. This allows a package to specify both
a "mod" type and a "git" type, to present more efficient module
access to module-aware go but still present a Git repo to the old
"go get".

Fixes #24751.
Fixes #25069.

Backport fixes #25139.

Change-Id: I378955613a0d63834d4f50f121f4db7e4d87dc0a
Reviewed-on: https://go-review.googlesource.com/114500
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
@gopherbot
Copy link
Contributor

Change https://golang.org/cl/115298 mentions this issue: [release-branch.go1.9] cmd/go: add minimal module-awareness for legacy operation

gopherbot pushed a commit that referenced this issue Jun 4, 2018
…y operation

We want authors to be able to publish code that works with both
the current standard go command and the planned new go command
support for modules. If authors have tagged their code v2 or later,
semantic import versioning means the import paths must include a
v2 path element after the path prefix naming the module.
One option for making this convention compatible with original go get
is to move code into a v2 subdirectory of the root.
That makes sense for some authors, but many authors would prefer
not to move all the code into a v2 subdirectory for a transition and
then move it back up once we everyone has a module-aware go command.

Instead, this CL teaches the old (non-module-aware) go command
a tiny amount about modules and their import paths, to expand
the options for authors who want to publish compatible packages.
If an author has a v2 of a package, say my/thing/v2/sub/pkg,
in the my/thing repo's sub/pkg subdirectory (no v2 in the file system path),
then old go get continues to import that package as my/thing/sub/pkg.
But when go get is processing code in any module (code in a tree with
a go.mod file) and encounters a path like my/thing/v2/sub/pkg,
it will check to see if my/thing/go.mod says "module my/thing/v2".
If so, the go command will read the import my/thing/v2/sub/pkg
as if it said my/thing/sub/pkg, which is the correct "old" import path
for the package in question.

This CL will be back-ported to Go 1.10 and Go 1.9 as well.

Once users have updated to the latest Go point releases containing
this new logic, authors will be able to update to using modules
within their own repos, including using semantic import paths
with vN path elements, and old go get will still be able to consume
those repositories.

This CL also makes "go get" ignore meta go-import lines using
the new "mod" VCS type. This allows a package to specify both
a "mod" type and a "git" type, to present more efficient module
access to module-aware go but still present a Git repo to the old
"go get".

Fixes #24751.
Fixes #25069.

This backport to Go 1.9 also had to pick up p.Internal.RawImports
from CL 74750 and CL 74356 and use it to prepare an updated
set of -importmap arguments for the compiler. (The old code only
understood vendor-related rewriting of import paths.)

Backport fixes #25140.

Change-Id: I378955613a0d63834d4f50f121f4db7e4d87dc0a
Reviewed-on: https://go-review.googlesource.com/115298
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Andrew Bonventre <andybons@golang.org>
@gopherbot
Copy link
Contributor

Change https://golang.org/cl/175219 mentions this issue: go/vcs: ignore "mod" VCS type

gopherbot pushed a commit to golang/tools that referenced this issue May 6, 2019
golang.org/x/tools/go/vcs is significantly behind the upstream
cmd/go/internal/get code, and has no support for modules. It continues
to implement mechanics for GOPATH mode only.

This change is a minimal fix to get it to continue to work
in the presence of the module mode-only "mod" VCS type
(documented at https://golang.org/cmd/go/#hdr-Remote_import_paths)
by effectively implementing IgnoreMod ModuleMode behavior.

It is similar to issue golang/go#24751 and a small subset of CL 109340
that fixed it.

This helps with module adoption by reducing the harm of adding the
"mod" VCS type for vanity import paths, something that was meant to
be backwards compatible.

While here, also backport CL 14482 (the Token to RawToken change).

Fixes golang/go#31845
Updates golang/go#24751

Change-Id: I0852f52cb9bda56879f923337c7f361df8412845
Reviewed-on: https://go-review.googlesource.com/c/tools/+/175219
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
@golang golang locked and limited conversation to collaborators May 4, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

2 participants