Multiple go versions, toolchains from go_sdk

[Chronojam]

What version of rules_go are you using?

0.19.5, but this can be reproduced with the latest

What version of gazelle are you using?

0.18.2

What version of Bazel are you using?

2.2.0

Does this issue reproduce with the latest releases of all the above?

Appears too yes, although i didnt test with gazelle, only rules_go

What operating system and processor architecture are you using?

linux/amd64

Any other potentially useful information about your toolchain?

What did you do?

# WORKSPACE

go_download_sdk(
        name = "go_sdk_1.14.2",
        #version = "1.14.2",
        sdks = {
            "darwin_amd64": ("go1.14.2.darwin-amd64.tar.gz", "e0db81f890bb253552b3fd783fccbc2cdda02552295cb305e75984eef1c1e2b9"),
            "freebsd_386": ("go1.14.2.freebsd-386.tar.gz", "7b8eccc27bf0a6a57999c1d30447ab61be19509a244ac3617bc4d1797eecb555"),
            "freebsd_amd64": ("go1.14.2.freebsd-amd64.tar.gz", "b9a26f5e2443898b54d7778d6861df70e8342d33a74bd7e02cf1105c4144ec05"),
            "linux_386": ("go1.14.2.linux-386.tar.gz", "cab5f51e6ffb616c6ee963c3d0650ca4e3c4108307c44f2baf233fcb8ff098f6"),
            "linux_amd64": ("go1.14.2.linux-amd64.tar.gz", "6272d6e940ecb71ea5636ddb5fab3933e087c1356173c61f4a803895e947ebb3"),
            "linux_arm64": ("go1.14.2.linux-arm64.tar.gz", "bb6d22fe5806352c3d0826676654e09b6e41eb1af52e8d506d3fa85adf7f8d88"),
            "linux_arm": ("go1.14.2.linux-armv6l.tar.gz", "eb4550ba741506c2a4057ea4d3a5ad7ed5a887de67c7232f1e4795464361c83c"),
            "linux_ppc64le": ("go1.14.2.linux-ppc64le.tar.gz", "48c22268c81ced9084a43bbe2c1596d3e636b5560b30a32434a7f15e561de160"),
            "linux_s390x": ("go1.14.2.linux-s390x.tar.gz", "501cc919648c9d85b901963303c5061ea6814c80f0d35fda9e62980d3ff58cf4"),
            "windows_386": ("go1.14.2.windows-386.zip", "76beccad98b6d3f8de935e5c9dbd69934cca1baa45b880965cdcf2779df56524"),
            "windows_amd64": ("go1.14.2.windows-amd64.zip", "1b5a60b3bbaa81106d5ee03499b5734ec093c6a255abf9a6a067f0f497a57916"),
        }
    )

# BUILD

go_binary(
    name = "app.wasm",
    embed = [":go_default_library"],
    visibility = ["//visibility:public"],
    goarch = "wasm",
    goos = "js",
    toolchains = [
        "@go_sdk_1.14.2//:go_linux_amd64",
        "@io_bazel_rules_go//go:toolchain",
    ]
)

Seperately, I did also attempt the following:

go_toolchain(
    name = "go_1.14.2",
    goos = "linux",
    goarch = "amd64",
    sdk = "@go_sdk_1.14.2//:go_sdk",
    builder = "@go_sdk_1.14.2//:builder"
)

toolchain(
    name = "my_toolchain",
    toolchain_type = "@io_bazel_rules_go//go:toolchain",
    exec_compatible_with = [
        "@io_bazel_rules_go//go/toolchain:linux",
        "@io_bazel_rules_go//go/toolchain:amd64",
    ],
    target_compatible_with = [],
    toolchain = ":go_1.14.2",
    visibility = ["//visibility:public"]
)

go_binary(
    name = "app.wasm",
    embed = [":go_default_library"],
    visibility = ["//visibility:public"],
    goarch = "wasm",
    goos = "js",
    toolchains = [
      ":my_toolchain",
    ]
)

Unfortunately, that also resulted in the same error :(

What did you expect to see?

Id like to have multiple versions of go_sdk in my project, (we've got some targets that need 1.14, and some that use 1.13)

Id expect using a toolchain provided by a go_sdk rule to work with a go_binary rule.

What did you see instead?

Spooky errors, even when using the default toolchain

'@go_sdk_1.14.2//:go_linux_amd64' does not have mandatory providers: 'TemplateVariableInfo'

'@io_bazel_rules_go//go:toolchain' does not have mandatory providers: 'TemplateVariableInfo'

Appreciate this could also be a misunderstanding from my part, this is my first real foray into toolchains, but given the documentation I could find; my approach seems ok?

Thanks!

[jayconrod]

You can have multiple go_download_sdk rules in WORKSPACE. Each one can download and register toolchains for a specific Go version. However, Bazel will only pick one version. Bazel selects the first registered toolchain that satisfies all the platform constraints, and there is currently no constraint setting for the Go version.

Even if that constraint setting were defined, I don't think it would solve your problem. You'd only be able to build for one Go version at a time. It might be possible to build for multiple platforms selecting multiple toolchains, but it would be difficult to arrange, and I'm not sure that Bazel supports it very well yet.

What is blocking you from building all targets with Go 1.14? That would be a much, much easier approach.

[Chronojam]

Thanks for your response!

I did try to get everything building with 1.14, but unfortunately I wasnt able to get rules_go to select that version by default, I tried the following configurations (using rules_go version 0.22.3):

### go_sdk name
go_download_sdk(
        name = "go_sdk",
        #version = "1.14.2",
        sdks = {
            "darwin_amd64": ("go1.14.2.darwin-amd64.tar.gz", "e0db81f890bb253552b3fd783fccbc2cdda02552295cb305e75984eef1c1e2b9"),
            "freebsd_386": ("go1.14.2.freebsd-386.tar.gz", "7b8eccc27bf0a6a57999c1d30447ab61be19509a244ac3617bc4d1797eecb555"),
            "freebsd_amd64": ("go1.14.2.freebsd-amd64.tar.gz", "b9a26f5e2443898b54d7778d6861df70e8342d33a74bd7e02cf1105c4144ec05"),
            "linux_386": ("go1.14.2.linux-386.tar.gz", "cab5f51e6ffb616c6ee963c3d0650ca4e3c4108307c44f2baf233fcb8ff098f6"),
            "linux_amd64": ("go1.14.2.linux-amd64.tar.gz", "6272d6e940ecb71ea5636ddb5fab3933e087c1356173c61f4a803895e947ebb3"),
            "linux_arm64": ("go1.14.2.linux-arm64.tar.gz", "bb6d22fe5806352c3d0826676654e09b6e41eb1af52e8d506d3fa85adf7f8d88"),
            "linux_arm": ("go1.14.2.linux-armv6l.tar.gz", "eb4550ba741506c2a4057ea4d3a5ad7ed5a887de67c7232f1e4795464361c83c"),
            "linux_ppc64le": ("go1.14.2.linux-ppc64le.tar.gz", "48c22268c81ced9084a43bbe2c1596d3e636b5560b30a32434a7f15e561de160"),
            "linux_s390x": ("go1.14.2.linux-s390x.tar.gz", "501cc919648c9d85b901963303c5061ea6814c80f0d35fda9e62980d3ff58cf4"),
            "windows_386": ("go1.14.2.windows-386.zip", "76beccad98b6d3f8de935e5c9dbd69934cca1baa45b880965cdcf2779df56524"),
            "windows_amd64": ("go1.14.2.windows-amd64.zip", "1b5a60b3bbaa81106d5ee03499b5734ec093c6a255abf9a6a067f0f497a57916"),
        }
    )

 go_rules_dependencies()
 go_register_toolchains()

This configuration causes 1.13.x to be loaded (from what i can tell it should check for the existence of a go_sdk rule and use that instead of the default)

$ bazel-drogo/external/go_sdk/bin/go version
go version go1.13 linux/amd64

also

# go_sdk rule deleted above
go_register_toolchains(
  go_version = "1.14.2"
)

this sadly errors with
go_version set after go sdk rule declared (go_sdk)

[jayconrod]

You should be able to download a 1.14 release without setting the sdks attribute. sdks is mainly useful for downloading an unsupported release (most often, a beta or release candidate).

go_download_sdk(
    name = "go_sdk",
    version = "1.14.2",
)

That said, setting sdks as you did should work. I tried that with rules_go v0.22.3 but couldn't reproduce the problem.

For go_register_toolchains, there's no need to pass the go_version argument in this case. go_register_toolchains will declare a go_download_sdk rule if you haven't explicitly declared one already. go_version can be used to choose the version that will be downloaded (by default, it's the highest supported release).

[Chronojam]

Hiya, thanks for confirming the behaviour - I guess its something funky with our setup.
Im going to close this as i dont want to take up more of your time

Thanks again 👍

[bartle-stripe]

One case where this would be useful is in large monorepos, where it's risky to upgrade all services at once. I think instead of modeling different versions of go as different toolchains, I think you'd want "version" as a configuration to a single toolchain? If I understand correctly, you could then use a transition to have a single go_binary and its dependencies use the same toolchain version?

[jayconrod]

@bartle-stripe I'm not sure what you mean here.

A toolchain in Bazel is pretty abstract: it's something that provides an object that rules requesting a toolchain of that type know how to use. Generally it provides dependencies (toolchain binaries, standard library) and functions for creating actions. Toolchains are automatically selected based on compatibility constraints they declare and constraints of the target platform.

In theory, you could define a toolchain which is the union of several Go distributions. You could define a new build setting for the Go version, and make the toolchain functions provide different sets of files based on that setting. Is this what you mean? I think this would be very difficult to implement.

[bartle-stripe]

Yeah, that's what I was thinking.