Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New submodules: upgrade before 2024-03-01 to avoid ambiguous import errors with google.golang.org/genproto #1015

Open
quartzmo opened this issue May 26, 2023 · 16 comments · Fixed by fortio/fortiotel#257
Labels
type: process A process-related concern. May include testing, release, or the like.

Comments

@quartzmo
Copy link
Member

quartzmo commented May 26, 2023

TL;DR

Upgrade to a version of google.golang.org/genproto at or above version v0.0.0-20230526161137-0005af68ea54 with the following command:

go get google.golang.org/genproto@latest
go mod tidy

Doing this before the 9-month deadline of 2024-03-01 will avoid "ambiguous import" errors.

Background

Many of the packages in the google.golang.org/genproto module represent API surfaces belonging to Google Cloud. Since approximately the v0.0.0-20220921223823-23cae91e6737 release of google.golang.org/genproto, a change in #901 introduced dependencies on virtually every submodule in cloud.google.com/go. This change moved type definitions for Google Cloud client libraries to cloud.google.com/go, greatly improving usability of the latter. To maintain compatibility with existing code, aliases were introduced into google.golang.org/genproto that pointed to the new sources of truth in cloud.google.com/go. This added the dependencies on cloud modules.

Even though most packages in google.golang.org/genproto are Google Cloud, a few packages provide simple “common” types used in projects unrelated to Google Cloud. For users of the simpler packages, this commingled existence resulted in a large number of extra cloud modules appearing in the dependency graph. This can trigger alarms in automatic dependency checks, false positives in vulnerability checks, and adds constraints to minimum version selection when resolving acceptable versions of other dependencies.

New submodules in this repo

To improve usability for consumers that do not depend on Google Cloud modules, packages providing widely used "common" types will be published as submodules. The following packages do not depend on cloud types and are now isolated in new non-cloud submodules:

  • google.golang.org/genproto/googleapis/api (Note: this does not include 4 subpackages referencing cloud types, listed below)
  • google.golang.org/genproto/googleapis/bytestream
  • google.golang.org/genproto/googleapis/rpc

The following subpackages of google.golang.org/genproto/googleapis/api reference cloud types and are now isolated from the api package as new cloud submodules:

  • google.golang.org/genproto/googleapis/api/apikeys
  • google.golang.org/genproto/googleapis/api/servicecontrol
  • google.golang.org/genproto/googleapis/api/servicemanagement
  • google.golang.org/genproto/googleapis/api/serviceusage

To ease migration, these submodules initially have a “back-ref” dependency on the google.golang.org/genproto parent. After a 9-month period, this "back-ref" dependency will be removed, making it possible to once again depend on the simpler packages of this project without adding cloud modules to the dependency graph.

Avoiding ambiguous imports for submodules

The root package docs for google.golang.org/genproto state:

IMPORTANT This repository is currently experimental. The structure of the contained packages is subject to change. Please see the original source repositories (listed below) to find out the status of each protocol buffer's associated service.

In addition to this warning, the pseudo version numbering of the module:

Signals that the module is still in development and unstable. This release carries no backward compatibility or stability guarantees.

However, despite these warnings, we know that critical infrastructure depends on many of the packages in google.golang.org/genproto. To avoid "ambiguous import" errors for consumers, we are adding a “back-ref” dependency on the google.golang.org/genproto parent to each new submodule for 9 months to allow a seamless transition.

After the 9-month deadline of 2024-03-01 has passed, the “back-ref” dependency on the google.golang.org/genproto parent will be removed from each new submodule. Client code still using a version of the google.golang.org/genproto parent earlier than v0.0.0-20230525234044-86246a831fb4 may then encounter "ambiguous import" errors when importing packages that have been moved to submodules. Those errors can be resolved by updating to a version at or above version v0.0.0-20230526161137-0005af68ea54 with the following command:

go get google.golang.org/genproto@latest
go mod tidy

References

@quartzmo quartzmo pinned this issue May 26, 2023
@quartzmo quartzmo changed the title google.golang.org/genproto has new submodules: upgrade before 2024-03-01 to avoid ambiguous import errors New submodules: upgrade before 2024-03-01 to avoid ambiguous import errors with google.golang.org/genproto May 26, 2023
@codyoss codyoss added the type: process A process-related concern. May include testing, release, or the like. label May 26, 2023
codeboten pushed a commit to codeboten/opentelemetry-go-contrib that referenced this issue Jun 9, 2023
This is to avoid issues caused by googleapis/go-genproto#1015

Signed-off-by: Alex Boten <aboten@lightstep.com>
codeboten pushed a commit to codeboten/opentelemetry-go-contrib that referenced this issue Jun 12, 2023
This is to avoid issues caused by googleapis/go-genproto#1015

Signed-off-by: Alex Boten <aboten@lightstep.com>
MrAlias pushed a commit to open-telemetry/opentelemetry-go-contrib that referenced this issue Jun 12, 2023
* [chore] updating google.golang.org/genproto

This is to avoid issues caused by googleapis/go-genproto#1015

Signed-off-by: Alex Boten <aboten@lightstep.com>

* update example

Signed-off-by: Alex Boten <aboten@lightstep.com>

* update metadata module

Signed-off-by: Alex Boten <aboten@lightstep.com>

---------

Signed-off-by: Alex Boten <aboten@lightstep.com>
abhinav added a commit to pulumi/pulumi that referenced this issue Jul 26, 2023
Updates to the latest version of google.golang.org/genproto
in all submodules in the repository.

This is necessary because in a recent change,
genproto split out some of its subpackages into independent submodules.
(googleapis/go-genproto#1015)

As a result of this, some users may see the error:

```
google.golang.org/genproto/googleapis/rpc/status: ambiguous import: found package google.golang.org/genproto/googleapis/rpc/status in multiple modules:
    google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 (/home/runner/go/pkg/mod/google.golang.org/genproto@v0.0.0-20230410155749-daa745c078e1/googleapis/rpc/status)
    google.golang.org/genproto/googleapis/rpc v0.0.0-20230725213213-b022f6e96895
```

Because pu/pu is using 20230410155749,
which has googleapis/rpc as a subpackage,
but another dependency references the independent submodule (20230725213213),
so the system doesn't know which module to use for the import path,
google.golang.org/genproto/googleapis/rpc/status.

This is specifically a problem for ProgramTest-based tests
for Pulumi Go programs that do not have a go.mod in the test directory.
This issue was encountered by @thomas11 while attempting to upgrade
dependencies in pulumi-docker (pulumi/pulumi-docker#700).

---

The contents of this commit were generated with the following command:

```
find . -name go.mod -exec dirname '{}' ';' | while read R; do
  (echo "--- $R" && cd "$R" && go get google.golang.org/genproto@latest && go mod tidy) || break
done
```

This failed for tests/integration/go/go-build-target
which had an old `go 1.17` directive. This was changed to `go 1.20`.
abhinav added a commit to pulumi/pulumi that referenced this issue Jul 26, 2023
Updates to the latest version of google.golang.org/genproto
in all submodules in the repository.

This is necessary because in a recent change,
genproto split out some of its subpackages into independent submodules.
(googleapis/go-genproto#1015)

As a result of this, some users may see the error:

```
google.golang.org/genproto/googleapis/rpc/status: ambiguous import: found package google.golang.org/genproto/googleapis/rpc/status in multiple modules:
    google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 (/home/runner/go/pkg/mod/google.golang.org/genproto@v0.0.0-20230410155749-daa745c078e1/googleapis/rpc/status)
    google.golang.org/genproto/googleapis/rpc v0.0.0-20230725213213-b022f6e96895
```

Because pu/pu is using 20230410155749,
which has googleapis/rpc as a subpackage,
but another dependency references the independent submodule (20230725213213),
so the system doesn't know which module to use for the import path,
google.golang.org/genproto/googleapis/rpc/status.

This is specifically a problem for ProgramTest-based tests
for Pulumi Go programs that do not have a go.mod in the test directory.
This issue was encountered by @thomas11 while attempting to upgrade
dependencies in pulumi-docker (pulumi/pulumi-docker#700).

---

The contents of this commit were generated with the following command:

```
find . -name go.mod -exec dirname '{}' ';' | while read R; do
  (echo "--- $R" && cd "$R" && go get google.golang.org/genproto@latest && go mod tidy) || break
done
```

This failed for tests/integration/go/go-build-target
which had an old `go 1.17` directive. This was changed to `go 1.20`.

This also accidentally tidied up go/regress-13301 which was reverted.
A follow-up is incoming to prevent issues like that in the future.
bors bot added a commit to pulumi/pulumi that referenced this issue Jul 26, 2023
13593: deps: Upgrade google.golang.org/genproto r=abhinav a=abhinav

Updates to the latest version of google.golang.org/genproto
in all submodules in the repository.

This is necessary because in a recent change,
genproto split out some of its subpackages into independent submodules.
(googleapis/go-genproto#1015)

As a result of this, some users may see the error:

```
google.golang.org/genproto/googleapis/rpc/status: ambiguous import: found package google.golang.org/genproto/googleapis/rpc/status in multiple modules:
    google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 (/home/runner/go/pkg/mod/google.golang.org/genproto@v0.0.0-20230410155749-daa745c078e1/googleapis/rpc/status)
    google.golang.org/genproto/googleapis/rpc v0.0.0-20230725213213-b022f6e96895
```

Because pu/pu is using 20230410155749,
which has googleapis/rpc as a subpackage,
but another dependency references the independent submodule (20230725213213),
so the system doesn't know which module to use for the import path,
google.golang.org/genproto/googleapis/rpc/status.

This is specifically a problem for ProgramTest-based tests
for Pulumi Go programs that do not have a go.mod in the test directory.
This issue was encountered by `@thomas11` while attempting to upgrade
dependencies in pulumi-docker (pulumi/pulumi-docker#700).

---

The contents of this commit were generated with the following command:

```
find . -name go.mod -exec dirname '{}' ';' | while read R; do
  (echo "--- $R" && cd "$R" && go get google.golang.org/genproto@latest && go mod tidy) || break
done
```

This failed for tests/integration/go/go-build-target
which had an old `go 1.17` directive. This was changed to `go 1.20`.

This also accidentally tidied up go/regress-13301 which was reverted.
PR #13596 aims to prevent mistakes like this in the future.

Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
bors bot added a commit to pulumi/pulumi that referenced this issue Jul 26, 2023
13593: deps: Upgrade google.golang.org/genproto r=abhinav a=abhinav

Updates to the latest version of google.golang.org/genproto
in all submodules in the repository.

This is necessary because in a recent change,
genproto split out some of its subpackages into independent submodules.
(googleapis/go-genproto#1015)

As a result of this, some users may see the error:

```
google.golang.org/genproto/googleapis/rpc/status: ambiguous import: found package google.golang.org/genproto/googleapis/rpc/status in multiple modules:
    google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 (/home/runner/go/pkg/mod/google.golang.org/genproto@v0.0.0-20230410155749-daa745c078e1/googleapis/rpc/status)
    google.golang.org/genproto/googleapis/rpc v0.0.0-20230725213213-b022f6e96895
```

Because pu/pu is using 20230410155749,
which has googleapis/rpc as a subpackage,
but another dependency references the independent submodule (20230725213213),
so the system doesn't know which module to use for the import path,
google.golang.org/genproto/googleapis/rpc/status.

This is specifically a problem for ProgramTest-based tests
for Pulumi Go programs that do not have a go.mod in the test directory.
This issue was encountered by `@thomas11` while attempting to upgrade
dependencies in pulumi-docker (pulumi/pulumi-docker#700).

---

The contents of this commit were generated with the following command:

```
find . -name go.mod -exec dirname '{}' ';' | while read R; do
  (echo "--- $R" && cd "$R" && go get google.golang.org/genproto@latest && go mod tidy) || break
done
```

This failed for tests/integration/go/go-build-target
which had an old `go 1.17` directive. This was changed to `go 1.20`.

This also accidentally tidied up go/regress-13301 which was reverted.
PR #13596 aims to prevent mistakes like this in the future.

13598: Changelog and go.mod updates for v3.76.1 r=pulumi-bot a=pulumi-bot

bors merge

Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
Co-authored-by: github-actions <github-actions@github.com>
abhinav added a commit to pulumi/pulumi that referenced this issue Jul 26, 2023
Updates to the latest version of google.golang.org/genproto
in all submodules in the repository.

This is necessary because in a recent change,
genproto split out some of its subpackages into independent submodules.
(googleapis/go-genproto#1015)

As a result of this, some users may see the error:

```
google.golang.org/genproto/googleapis/rpc/status: ambiguous import: found package google.golang.org/genproto/googleapis/rpc/status in multiple modules:
    google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 (/home/runner/go/pkg/mod/google.golang.org/genproto@v0.0.0-20230410155749-daa745c078e1/googleapis/rpc/status)
    google.golang.org/genproto/googleapis/rpc v0.0.0-20230725213213-b022f6e96895
```

Because pu/pu is using 20230410155749,
which has googleapis/rpc as a subpackage,
but another dependency references the independent submodule (20230725213213),
so the system doesn't know which module to use for the import path,
google.golang.org/genproto/googleapis/rpc/status.

This is specifically a problem for ProgramTest-based tests
for Pulumi Go programs that do not have a go.mod in the test directory.
This issue was encountered by @thomas11 while attempting to upgrade
dependencies in pulumi-docker (pulumi/pulumi-docker#700).

---

The contents of this commit were generated with the following command:

```
find . -name go.mod -exec dirname '{}' ';' | while read R; do
  (echo "--- $R" && cd "$R" && go get google.golang.org/genproto@latest && go mod tidy) || break
done
```

This failed for tests/integration/go/go-build-target
which had an old `go 1.17` directive. This was changed to `go 1.20`.

This also accidentally tidied up go/regress-13301 which was reverted.
A follow-up is incoming to prevent issues like that in the future.
jchappelow pushed a commit to kwilteam/kwil-db that referenced this issue Jul 26, 2023
See googleapis/go-genproto#1015

Several new submodules were carved out of the large genproto module,
This is causing an ambiguous import. The resolution here is to use the
newly carved out genproto/googleapis/api submodule.

Also run go mod tidy.

"go build -v ./..." succeeds now.
bors bot added a commit to pulumi/pulumi that referenced this issue Jul 26, 2023
13593: deps: Upgrade google.golang.org/genproto r=abhinav a=abhinav

Updates to the latest version of google.golang.org/genproto
in all submodules in the repository.

This is necessary because in a recent change,
genproto split out some of its subpackages into independent submodules.
(googleapis/go-genproto#1015)

As a result of this, some users may see the error:

```
google.golang.org/genproto/googleapis/rpc/status: ambiguous import: found package google.golang.org/genproto/googleapis/rpc/status in multiple modules:
    google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 (/home/runner/go/pkg/mod/google.golang.org/genproto@v0.0.0-20230410155749-daa745c078e1/googleapis/rpc/status)
    google.golang.org/genproto/googleapis/rpc v0.0.0-20230725213213-b022f6e96895
```

Because pu/pu is using 20230410155749,
which has googleapis/rpc as a subpackage,
but another dependency references the independent submodule (20230725213213),
so the system doesn't know which module to use for the import path,
google.golang.org/genproto/googleapis/rpc/status.

This is specifically a problem for ProgramTest-based tests
for Pulumi Go programs that do not have a go.mod in the test directory.
This issue was encountered by `@thomas11` while attempting to upgrade
dependencies in pulumi-docker (pulumi/pulumi-docker#700).

---

The contents of this commit were generated with the following command:

```
find . -name go.mod -exec dirname '{}' ';' | while read R; do
  (echo "--- $R" && cd "$R" && go get google.golang.org/genproto@latest && go mod tidy) || break
done
```

This failed for tests/integration/go/go-build-target
which had an old `go 1.17` directive. This was changed to `go 1.20`.

This also accidentally tidied up go/regress-13301 which was reverted.
PR #13596 aims to prevent mistakes like this in the future.

13596: test(regress-13301): Avoid accidental tidying r=abhinav a=abhinav

The regression test for #13301 needs an intentionally bad go.mod file.
This file was excluded from `make tidy`, allowing it to remain invalid,
but this doesn't protect it from bulk commands like the following
used in #13593

```bash
find . -name go.mod -exec dirname '{}' ';' | while read R; do
  (cd "$R" && ... && go mod tidy)
done
```

In fact, #13593 accidentally tidied this go.mod file
(removing the extraneous dependencies critical to the regression test)
and failed in CI.

To prevent issues like this, rename the go.mod to go.mod.bad,
and rename it back to go.mod in the test environment at test time.

This also lets us revert the `make tidy` exclusion support in tidy.sh.


Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
bors bot added a commit to pulumi/pulumi that referenced this issue Jul 27, 2023
13593: deps: Upgrade google.golang.org/genproto r=abhinav a=abhinav

Updates to the latest version of google.golang.org/genproto
in all submodules in the repository.

This is necessary because in a recent change,
genproto split out some of its subpackages into independent submodules.
(googleapis/go-genproto#1015)

As a result of this, some users may see the error:

```
google.golang.org/genproto/googleapis/rpc/status: ambiguous import: found package google.golang.org/genproto/googleapis/rpc/status in multiple modules:
    google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 (/home/runner/go/pkg/mod/google.golang.org/genproto@v0.0.0-20230410155749-daa745c078e1/googleapis/rpc/status)
    google.golang.org/genproto/googleapis/rpc v0.0.0-20230725213213-b022f6e96895
```

Because pu/pu is using 20230410155749,
which has googleapis/rpc as a subpackage,
but another dependency references the independent submodule (20230725213213),
so the system doesn't know which module to use for the import path,
google.golang.org/genproto/googleapis/rpc/status.

This is specifically a problem for ProgramTest-based tests
for Pulumi Go programs that do not have a go.mod in the test directory.
This issue was encountered by `@thomas11` while attempting to upgrade
dependencies in pulumi-docker (pulumi/pulumi-docker#700).

---

The contents of this commit were generated with the following command:

```
find . -name go.mod -exec dirname '{}' ';' | while read R; do
  (echo "--- $R" && cd "$R" && go get google.golang.org/genproto@latest && go mod tidy) || break
done
```

This failed for tests/integration/go/go-build-target
which had an old `go 1.17` directive. This was changed to `go 1.20`.

This also accidentally tidied up go/regress-13301 which was reverted.
PR #13596 aims to prevent mistakes like this in the future.

Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
bors bot added a commit to pulumi/pulumi that referenced this issue Jul 27, 2023
13593: deps: Upgrade google.golang.org/genproto r=abhinav a=abhinav

Updates to the latest version of google.golang.org/genproto
in all submodules in the repository.

This is necessary because in a recent change,
genproto split out some of its subpackages into independent submodules.
(googleapis/go-genproto#1015)

As a result of this, some users may see the error:

```
google.golang.org/genproto/googleapis/rpc/status: ambiguous import: found package google.golang.org/genproto/googleapis/rpc/status in multiple modules:
    google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 (/home/runner/go/pkg/mod/google.golang.org/genproto@v0.0.0-20230410155749-daa745c078e1/googleapis/rpc/status)
    google.golang.org/genproto/googleapis/rpc v0.0.0-20230725213213-b022f6e96895
```

Because pu/pu is using 20230410155749,
which has googleapis/rpc as a subpackage,
but another dependency references the independent submodule (20230725213213),
so the system doesn't know which module to use for the import path,
google.golang.org/genproto/googleapis/rpc/status.

This is specifically a problem for ProgramTest-based tests
for Pulumi Go programs that do not have a go.mod in the test directory.
This issue was encountered by `@thomas11` while attempting to upgrade
dependencies in pulumi-docker (pulumi/pulumi-docker#700).

---

The contents of this commit were generated with the following command:

```
find . -name go.mod -exec dirname '{}' ';' | while read R; do
  (echo "--- $R" && cd "$R" && go get google.golang.org/genproto@latest && go mod tidy) || break
done
```

This failed for tests/integration/go/go-build-target
which had an old `go 1.17` directive. This was changed to `go 1.20`.

This also accidentally tidied up go/regress-13301 which was reverted.
PR #13596 aims to prevent mistakes like this in the future.

13596: test(regress-13301): Avoid accidental tidying r=abhinav a=abhinav

The regression test for #13301 needs an intentionally bad go.mod file.
This file was excluded from `make tidy`, allowing it to remain invalid,
but this doesn't protect it from bulk commands like the following
used in #13593

```bash
find . -name go.mod -exec dirname '{}' ';' | while read R; do
  (cd "$R" && ... && go mod tidy)
done
```

In fact, #13593 accidentally tidied this go.mod file
(removing the extraneous dependencies critical to the regression test)
and failed in CI.

To prevent issues like this, rename the go.mod to go.mod.bad,
and rename it back to go.mod in the test environment at test time.

This also lets us revert the `make tidy` exclusion support in tidy.sh.


Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
bors bot added a commit to pulumi/pulumi that referenced this issue Jul 27, 2023
13593: deps: Upgrade google.golang.org/genproto r=abhinav a=abhinav

Updates to the latest version of google.golang.org/genproto
in all submodules in the repository.

This is necessary because in a recent change,
genproto split out some of its subpackages into independent submodules.
(googleapis/go-genproto#1015)

As a result of this, some users may see the error:

```
google.golang.org/genproto/googleapis/rpc/status: ambiguous import: found package google.golang.org/genproto/googleapis/rpc/status in multiple modules:
    google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 (/home/runner/go/pkg/mod/google.golang.org/genproto@v0.0.0-20230410155749-daa745c078e1/googleapis/rpc/status)
    google.golang.org/genproto/googleapis/rpc v0.0.0-20230725213213-b022f6e96895
```

Because pu/pu is using 20230410155749,
which has googleapis/rpc as a subpackage,
but another dependency references the independent submodule (20230725213213),
so the system doesn't know which module to use for the import path,
google.golang.org/genproto/googleapis/rpc/status.

This is specifically a problem for ProgramTest-based tests
for Pulumi Go programs that do not have a go.mod in the test directory.
This issue was encountered by `@thomas11` while attempting to upgrade
dependencies in pulumi-docker (pulumi/pulumi-docker#700).

---

The contents of this commit were generated with the following command:

```
find . -name go.mod -exec dirname '{}' ';' | while read R; do
  (echo "--- $R" && cd "$R" && go get google.golang.org/genproto@latest && go mod tidy) || break
done
```

This failed for tests/integration/go/go-build-target
which had an old `go 1.17` directive. This was changed to `go 1.20`.

This also accidentally tidied up go/regress-13301 which was reverted.
PR #13596 aims to prevent mistakes like this in the future.

Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
abhinav added a commit to pulumi/pulumi that referenced this issue Jul 27, 2023
Updates to the latest versions of
google.golang.org/genproto and google.golang.org/grpc
in all submodules in the repository.

This is necessary because in a recent change,
genproto split out some of its subpackages into independent submodules.
(googleapis/go-genproto#1015)

As a result of this, some users may see the error:

```
google.golang.org/genproto/googleapis/rpc/status: ambiguous import: found package google.golang.org/genproto/googleapis/rpc/status in multiple modules:
    google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 (/home/runner/go/pkg/mod/google.golang.org/genproto@v0.0.0-20230410155749-daa745c078e1/googleapis/rpc/status)
    google.golang.org/genproto/googleapis/rpc v0.0.0-20230725213213-b022f6e96895
```

Because pu/pu is using 20230410155749,
which has googleapis/rpc as a subpackage,
but another dependency references the independent submodule (20230725213213),
so the system doesn't know which module to use for the import path,
google.golang.org/genproto/googleapis/rpc/status.

This is a problem for codegen tests and ProgramTest-based tests
for Pulumi Go programs that do not have a go.mod in the test directory.
This issue was encountered by @thomas11 while attempting to upgrade
dependencies in pulumi-docker (pulumi/pulumi-docker#700).

The grpc upgrade is necessary because the current version of grpc
also pulls the outdated version of genproto.
@ldemailly
Copy link

ldemailly commented Oct 4, 2024

@ldemailly can you give a small reproducer of this problem with a go.mod. Likely the solution is to bump some other out of date dependency.

Thanks! yes:

git clone https://github.com/fortio/fortiotel
cd fortiotel
go build . # all good/fine
go mod tidy
git diff
go build .  # error as below and this original ticket

diff is

13:57:13 main fortiotel$ go mod tidy
13:57:18 main fortiotel$ git diff
diff --git a/go.mod b/go.mod
index d54dab6..3be40ca 100644
--- a/go.mod
+++ b/go.mod
@@ -46,7 +46,6 @@ require (
        golang.org/x/net v0.29.0 // indirect
        golang.org/x/sys v0.25.0 // indirect
        golang.org/x/text v0.18.0 // indirect
-       google.golang.org/genproto v0.0.0-20240924160255-9d4c2d233b61 // indirect
        google.golang.org/genproto/googleapis/api v0.0.0-20240903143218-8af14fe29dc1 // indirect
        google.golang.org/genproto/googleapis/rpc v0.0.0-20240903143218-8af14fe29dc1 // indirect
        google.golang.org/grpc v1.67.1 // indirect

error after tidy is

$ go build .
../../../go/pkg/mod/google.golang.org/grpc@v1.67.1/status/status.go:35:2: ambiguous import: found package google.golang.org/genproto/googleapis/rpc/status in multiple modules:
	google.golang.org/genproto v0.0.0-20230216225411-c8e22ba71e44 (/Users/dl/go/pkg/mod/google.golang.org/genproto@v0.0.0-20230216225411-c8e22ba71e44/googleapis/rpc/status)
	google.golang.org/genproto/googleapis/rpc v0.0.0-20240903143218-8af14fe29dc1 (/Users/dl/go/pkg/mod/google.golang.org/genproto/googleapis/rpc@v0.0.0-20240903143218-8af14fe29dc1/status)
../../../go/pkg/mod/github.com/grpc-ecosystem/grpc-gateway/v2@v2.22.0/runtime/handler.go:13:2: ambiguous import: found package google.golang.org/genproto/googleapis/api/httpbody in multiple modules:
	google.golang.org/genproto v0.0.0-20230216225411-c8e22ba71e44 (/Users/dl/go/pkg/mod/google.golang.org/genproto@v0.0.0-20230216225411-c8e22ba71e44/googleapis/api/httpbody)
	google.golang.org/genproto/googleapis/api v0.0.0-20240903143218-8af14fe29dc1 (/Users/dl/go/pkg/mod/google.golang.org/genproto/googleapis/api@v0.0.0-20240903143218-8af14fe29dc1/httpbody)
../../../go/pkg/mod/go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc@v1.30.0/client.go:12:2: ambiguous import: found package google.golang.org/genproto/googleapis/rpc/errdetails in multiple modules:
	google.golang.org/genproto v0.0.0-20230216225411-c8e22ba71e44 (/Users/dl/go/pkg/mod/google.golang.org/genproto@v0.0.0-20230216225411-c8e22ba71e44/googleapis/rpc/errdetails)
	google.golang.org/genproto/googleapis/rpc v0.0.0-20240903143218-8af14fe29dc1 (/Users/dl/go/pkg/mod/google.golang.org/genproto/googleapis/rpc@v0.0.0-20240903143218-8af14fe29dc1/errdetails)

@codyoss
Copy link
Member

codyoss commented Oct 4, 2024

I have filed envoyproxy/go-control-plane#1014. In the meantime another workaround would be to underscore import genproto in your project so that you can control the version of the dependency and there is no ambiguity.

@ldemailly
Copy link

ldemailly commented Oct 4, 2024

turns out... it's the go.work or something related to the simple/ sub directory
(btw trying to _ import didn't work because there is nothing at the top and even trying to read say

	protobuf "google.golang.org/genproto/protobuf/api"

 protobuf.File_google_protobuf_api_proto

also somehow left it in indirect

@ldemailly
Copy link

So in short, sorry for the noise yet maybe that oddity that a sub project go.mod can cause that issue might help someone else

@ldemailly
Copy link

well it's actually not fixed... go mod tidy in simple if they all use the same toolchain does remove it... thankfully that one isn't done by dependabot and rots but... something fishy is going on

yuhan6665 added a commit to yuhan6665/AndroidLibXrayLite that referenced this issue Oct 6, 2024
2dust pushed a commit to 2dust/AndroidLibXrayLite that referenced this issue Oct 7, 2024
skylenet added a commit to ethpandaops/dora that referenced this issue Oct 16, 2024
gmalouf added a commit to gmalouf/indexer that referenced this issue Oct 29, 2024
gmalouf added a commit to algorand/indexer that referenced this issue Oct 29, 2024
@bmulder-innoseis
Copy link

well it's actually not fixed... go mod tidy in simple if they all use the same toolchain does remove it... thankfully that one isn't done by dependabot and rots but... something fishy is going on

@ldemailly I don't understand what you mean and I have the same issue. What did you do to work around it?

@ldemailly
Copy link

@ldemailly I don't understand what you mean and I have the same issue. What did you do to work around it?

the workaround I have is to have a subdirectory with its own go.mod with the added indirect require - having that allows the real go.mod to work and survive go mod tidy. it's a horrible workaround and I don't understand why/how it works (see fortio/fortiotel repo)

@bmulder-innoseis
Copy link

bmulder-innoseis commented Nov 1, 2024

the workaround I have is to have a subdirectory with its own go.mod with the added indirect require

Oh dear...
Anyway, thanks I guess

Maybe someone can reopen this in the meantime? Or is there another ticket for this?

@quartzmo
Copy link
Member Author

quartzmo commented Nov 8, 2024

tbg added a commit to tbg/cockroach that referenced this issue Nov 12, 2024
…f68ea54

See googleapis/go-genproto#1015.

Sadly grpc-gateway is incompatible with this version of `genproto`:

```
ERROR: no such package '@@org_golang_google_genproto//googleapis/api/httpbody': BUILD file not found in directory 'googleapis/api/httpbody' of external repository @@org_golang_google_genproto. Add a BUILD file to a directory to mark it as a package.
ERROR: /private/var/tmp/_bazel_tbg/b1346cddcc70d57afdaa90f7f09f9b2c/external/com_github_grpc_ecosystem_grpc_gateway/runtime/BUILD.bazel:5:11: no such package '@@org_golang_google_genproto//googleapis/api/httpbody': BUILD file not found in directory 'googleapis/api/httpbody' of external repository @@org_golang_google_genproto. Add a BUILD file to a directory to mark it as a package. and referenced by '@@com_github_grpc_ecosystem_grpc_gateway//runtime:go_default_library'
```

Since we are on the final `grpc-gateway/v1` version already[^1], we'll
have to make the leap to v2 to fix this.

[^1]: https://github.com/grpc-ecosystem/grpc-gateway/releases/tag/v1.16.0
tbg added a commit to tbg/cockroach that referenced this issue Nov 13, 2024
…f68ea54

See googleapis/go-genproto#1015.

Sadly grpc-gateway is incompatible with this version of `genproto`:

```
ERROR: no such package '@@org_golang_google_genproto//googleapis/api/httpbody': BUILD file not found in directory 'googleapis/api/httpbody' of external repository @@org_golang_google_genproto. Add a BUILD file to a directory to mark it as a package.
ERROR: /private/var/tmp/_bazel_tbg/b1346cddcc70d57afdaa90f7f09f9b2c/external/com_github_grpc_ecosystem_grpc_gateway/runtime/BUILD.bazel:5:11: no such package '@@org_golang_google_genproto//googleapis/api/httpbody': BUILD file not found in directory 'googleapis/api/httpbody' of external repository @@org_golang_google_genproto. Add a BUILD file to a directory to mark it as a package. and referenced by '@@com_github_grpc_ecosystem_grpc_gateway//runtime:go_default_library'
```

Since we are on the final `grpc-gateway/v1` version already[^1], we'll
have to make the leap to v2 to fix this.

[^1]: https://github.com/grpc-ecosystem/grpc-gateway/releases/tag/v1.16.0
tbg added a commit to tbg/cockroach that referenced this issue Nov 27, 2024
…f68ea54

See googleapis/go-genproto#1015.

Sadly grpc-gateway is incompatible with this version of `genproto`:

```
ERROR: no such package '@@org_golang_google_genproto//googleapis/api/httpbody': BUILD file not found in directory 'googleapis/api/httpbody' of external repository @@org_golang_google_genproto. Add a BUILD file to a directory to mark it as a package.
ERROR: /private/var/tmp/_bazel_tbg/b1346cddcc70d57afdaa90f7f09f9b2c/external/com_github_grpc_ecosystem_grpc_gateway/runtime/BUILD.bazel:5:11: no such package '@@org_golang_google_genproto//googleapis/api/httpbody': BUILD file not found in directory 'googleapis/api/httpbody' of external repository @@org_golang_google_genproto. Add a BUILD file to a directory to mark it as a package. and referenced by '@@com_github_grpc_ecosystem_grpc_gateway//runtime:go_default_library'
```

Since we are on the final `grpc-gateway/v1` version already[^1], we'll
have to make the leap to v2 to fix this.

[^1]: https://github.com/grpc-ecosystem/grpc-gateway/releases/tag/v1.16.0
tbg added a commit to tbg/cockroach that referenced this issue Nov 27, 2024
…f68ea54

See googleapis/go-genproto#1015.

Sadly grpc-gateway is incompatible with this version of `genproto`:

```
ERROR: no such package '@@org_golang_google_genproto//googleapis/api/httpbody': BUILD file not found in directory 'googleapis/api/httpbody' of external repository @@org_golang_google_genproto. Add a BUILD file to a directory to mark it as a package.
ERROR: /private/var/tmp/_bazel_tbg/b1346cddcc70d57afdaa90f7f09f9b2c/external/com_github_grpc_ecosystem_grpc_gateway/runtime/BUILD.bazel:5:11: no such package '@@org_golang_google_genproto//googleapis/api/httpbody': BUILD file not found in directory 'googleapis/api/httpbody' of external repository @@org_golang_google_genproto. Add a BUILD file to a directory to mark it as a package. and referenced by '@@com_github_grpc_ecosystem_grpc_gateway//runtime:go_default_library'
```

Since we are on the final `grpc-gateway/v1` version already[^1], we'll
have to make the leap to v2 to fix this.

[^1]: https://github.com/grpc-ecosystem/grpc-gateway/releases/tag/v1.16.0
tbg added a commit to tbg/cockroach that referenced this issue Nov 29, 2024
…f68ea54

See googleapis/go-genproto#1015.

Sadly grpc-gateway is incompatible with this version of `genproto`:

```
ERROR: no such package '@@org_golang_google_genproto//googleapis/api/httpbody': BUILD file not found in directory 'googleapis/api/httpbody' of external repository @@org_golang_google_genproto. Add a BUILD file to a directory to mark it as a package.
ERROR: /private/var/tmp/_bazel_tbg/b1346cddcc70d57afdaa90f7f09f9b2c/external/com_github_grpc_ecosystem_grpc_gateway/runtime/BUILD.bazel:5:11: no such package '@@org_golang_google_genproto//googleapis/api/httpbody': BUILD file not found in directory 'googleapis/api/httpbody' of external repository @@org_golang_google_genproto. Add a BUILD file to a directory to mark it as a package. and referenced by '@@com_github_grpc_ecosystem_grpc_gateway//runtime:go_default_library'
```

Since we are on the final `grpc-gateway/v1` version already[^1], we'll
have to make the leap to v2 to fix this.

[^1]: https://github.com/grpc-ecosystem/grpc-gateway/releases/tag/v1.16.0
tbg added a commit to tbg/cockroach that referenced this issue Dec 2, 2024
…f68ea54

See googleapis/go-genproto#1015.

Sadly grpc-gateway is incompatible with this version of `genproto`:

```
ERROR: no such package '@@org_golang_google_genproto//googleapis/api/httpbody': BUILD file not found in directory 'googleapis/api/httpbody' of external repository @@org_golang_google_genproto. Add a BUILD file to a directory to mark it as a package.
ERROR: /private/var/tmp/_bazel_tbg/b1346cddcc70d57afdaa90f7f09f9b2c/external/com_github_grpc_ecosystem_grpc_gateway/runtime/BUILD.bazel:5:11: no such package '@@org_golang_google_genproto//googleapis/api/httpbody': BUILD file not found in directory 'googleapis/api/httpbody' of external repository @@org_golang_google_genproto. Add a BUILD file to a directory to mark it as a package. and referenced by '@@com_github_grpc_ecosystem_grpc_gateway//runtime:go_default_library'
```

Since we are on the final `grpc-gateway/v1` version already[^1], we'll
have to make the leap to v2 to fix this.

[^1]: https://github.com/grpc-ecosystem/grpc-gateway/releases/tag/v1.16.0
tbg added a commit to tbg/cockroach that referenced this issue Dec 4, 2024
…f68ea54

See googleapis/go-genproto#1015.

Sadly grpc-gateway is incompatible with this version of `genproto`:

```
ERROR: no such package '@@org_golang_google_genproto//googleapis/api/httpbody': BUILD file not found in directory 'googleapis/api/httpbody' of external repository @@org_golang_google_genproto. Add a BUILD file to a directory to mark it as a package.
ERROR: /private/var/tmp/_bazel_tbg/b1346cddcc70d57afdaa90f7f09f9b2c/external/com_github_grpc_ecosystem_grpc_gateway/runtime/BUILD.bazel:5:11: no such package '@@org_golang_google_genproto//googleapis/api/httpbody': BUILD file not found in directory 'googleapis/api/httpbody' of external repository @@org_golang_google_genproto. Add a BUILD file to a directory to mark it as a package. and referenced by '@@com_github_grpc_ecosystem_grpc_gateway//runtime:go_default_library'
```

Since we are on the final `grpc-gateway/v1` version already[^1], we'll
have to make the leap to v2 to fix this.

[^1]: https://github.com/grpc-ecosystem/grpc-gateway/releases/tag/v1.16.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: process A process-related concern. May include testing, release, or the like.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants