doc: document installation from package managers

[rakyll]

The officially documented ways to install Go are to download the release archives from golang.org/dl, install it via msi or pkg installater or installing Go from the source. Even though we only support these options, a large portion of Go users are already depending on package managers to install Go and even more to desire that they can depend on package manager installations [*].

Currently, we have community-drive approach to maintain and release Go packages and this model seems to be working for some and widely used to install Go (32% of the installers already use brew and Linux package managers already).

To improve this experience, I propose the following:

• Keep package maintainers in sync during the releases and make sure that they are unblocked.
• Document these packages and instructions to use them on the golang.org website.
• Reevaluate the material on the wiki, such as https://github.com/golang/go/wiki/Ubuntu, to make it consistent with the official documentation.

/cc @bradfitz @mwhudson @rsc

[rsc]

The root problem is that Go is too hard to install by following the instructions at golang.org/doc/install (compare with https://www.rust-lang.org/en-US/install.html; I believe @broady and @campoy are working on a similar curl | sh for Go on Unix).

Another is that even if it's easy, people also want familiar, meaning that if their favorite binary distribution is available (apt-get, brew, yum), they might prefer that. There's an open question of how involved we want to be, how much we want to support those.

Another is that it's too difficult to upgrade once you already have Go and a new version comes out. #18136 is about making that easier for betas, and it would be nice to do that for true releases too.

None of this addresses Windows, which also has popular package managers that I don't know about (Chocolatey? 0install?).

At the moment, no one owns any of this. There are open questions about what we should do and no one to do it. You mention that the current approach is community-driven. What extra steps should we take to "keep package maintainers in sync"?

[mwhudson]

It's part of my job to maintain Go in Debian and Ubuntu and for the case of getting new Go onto new Debian and Ubuntu I think things are in pretty good shape, I'm not sure if we really need anything from the Go team (predictable release dates are nice!). There are other (perhaps slightly less active) maintainers in Debian (hi @tianon) and I'm pretty Canonical would find someone else to maintain Go in Ubuntu if I left.

This leaves out the case of people who want new Go on older (i.e. LTS) Ubuntu. I maintain a PPA that can be used for that (https://code.launchpad.net/~gophers/+archive/ubuntu/archive). It might be nice to link to that from somewhere more official (certainly the wiki page could link to it). This is a bit more of a personal thing, I'm not sure if someone would/could step up to maintain it if I couldn't do it any more for whatever reason (it mostly runs itself these days but does use a complicated git repo & associated shell script). Maybe we can recruit some backup and make it more official?

Another option would be to ship golang as a snap (as in http://snapcraft.io/), which would work on many distributions.

[tianon]

For installing official binary releases to $HOME via pure sh, I collaborated with the Travis folks to create https://github.com/travis-ci/gimme (specifically https://github.com/travis-ci/gimme/blob/master/gimme; cc @meatballhat). 😇

I'd love to see / collaborate on creating official Debian or Ubuntu PPAs though (and I'd imagine @jessfraz would be interested too -- she's expressed interest in creating more "official" packaging over in #10965). 👍

[kardianos]

@mwhudson I tend to install go directly from golang.org/dl/ exactly to ensure freshness. I also have been playing with snaps recently and I've been very pleased with them; I think snaps would be ideally suited to package Go and would keep it up to date cleanly.

For Windows I think the existing msi is very consistent with most users expectations on that platform.

[rakyll]

There many potential solutions to improve installing/updating Go. But, this proposal is only about documenting the currently used/preferred ways. It is not about fixing the installation/updating experience in Go.

You mention that the current approach is community-driven. What extra steps should we take to "keep package maintainers in sync"?

• Give the maintainers release dates.
• Track their progress on the issue tracker. For brew, we can file a cross issue on https://github.com/Homebrew/homebrew-core/.
• Make sure popular package managers have the latest version before announcing the release.
• Offer help if needed.

Maybe we can recruit some backup and make it more official?

I assume we should keep this convo on #10965?

FWIW, these packages don't have be owned/maintained by the Go project to be documented. We still can have them unofficial and add a notice that are them so. The goal here is to have a proper point to document such widely used packages.

[rsc]

• Give the maintainers release dates.

How? Mailing go-announce seems too spammy for go-announce. Is there something else we should do?

• Track their progress on the issue tracker. For brew, we can file a cross issue on https://github.com/Homebrew/homebrew-core/.

Do you mean file issues for each popular package manager? On their tracker or ours?

• Make sure popular package managers have the latest version before announcing the release.

You mean cut the release, publish binaries, and sit on it until package managers have it? That seems a bit odd. People will want the announcement as soon as the golang.org/dl binaries are ready.

• Offer help if needed.

I hope people understand that offer is always there in general (golang.org/help), but if there's something more you think we should do, please be specific.

---

Maybe stepping back, do we know what the preferred methods of contact are for each package maintainer? If it's lightweight we could probably add "notify these projects" somewhere in our release calendar, where notify is whatever method they prefer. @spf13 is interested in helping here, if this is what we decide.

[jessfraz]

Maybe stepping back, do we know what the preferred methods of contact are for each package maintainer? If it's lightweight we could probably add "notify these projects" somewhere in our release calendar, where notify is whatever method they prefer.

As someone who maintains packages for other things, and has been on the side of notifying "downstreams", I think the usual protocol here is to have a mailing list group with the necessary people on it. Then you all can just email go-downstreams or something

[spf13]

@rakyll I've reached out to the maintainers of the chocolatey and brew packages as those seemed the most straight forward.

If they are good with a list as you and @jessfraz suggested then we can start there.

The Linux packages are a bit more challenging. They often have much longer delays for various reasons and I think it's critical that we minimize the amount of time between the release announcement and the availability of the packages through any of the "official" or "recommended" installation methods.

Brew and Chocolatey seem like a good start while we explore the linux packages and the right solution there.

[broady]

I think the usual protocol here is to have a mailing list group with the necessary people on it.

Isn't this just go-announce? We only announce new releases on there (and pre-announce security releases).

[spf13]

It's not just go-announce. It's more like go-preannounce. A message should be sent to this list early in the release process to give heads up that the release is about to happen and give the package maintainers as much notice as possible to be able to prepare their packages/recipes to be released at the same time as the announcement.

[rakyll]

@spf13 As far as I understand brew packages are maintained by the overall community, not by a single person. We need to reach out to them by filing an issue. The notification and coordination by mailing works for the rest of the package manager maintainers.

Do you mean file issues for each popular package manager? On their tracker or ours?

The way to get a new version into brew is to open a CL against brew or let someone do it for us. If we file an issue on brew, they will resolve it. If the Go project wants to track the progress, we can have a tracking issue on the Go issue tracker.

People will want the announcement as soon as the golang.org/dl binaries are ready.

A common way to install Go is via the package manager. In practice, a part of the Go users don't get the latest version until the package is updated. The entire proposal exists to suggest that package manager users should be able to get the new releases via their desired way. If you have any concerns that the release might be delayed, we need a broader strategy where we own the packages and control their lifecycle.

I hope people understand that offer is always there in general (golang.org/help), but if there's something more you think we should do, please be specific.

The communication- and coordination-related help is extra work for the project, I added the line to acknowledge it has to be handled, ideally by the person who are cutting the releases.

[spf13]

As far as I understand brew packages are maintained by the overall community, not by a single person. We need to reach out to them by filing an issue. The notification and coordination by mailing works for the rest of the package manager maintainers.

This is true, but there's a set of people who have primarily been doing it for Go. I reached out to them to understand their process better and they provided a lot of good feedback.

I think there's a few efforts needed here and perhaps they can happen in parallel.
1a. Update the installation instructions to include vetted package managers.
2a. Understand the constraints of the different package distribution pipelines.
2b. Shorten the delay between Go release and the release of the various packages.

[spf13]

This was discussed during the proposal review meeting today.

Acknowledging that there will always be a delay between when a release is published and when a package distribution is able to distribute that version, it was determined that the ideal path forward would be to query the current version being distributed by each of the package managers and display this with a note along with the directions... something like, "The current version of Go is 1.8.0, brew currently provides 1.7.5" followed by the directions to obtain that version. @rsc mentioned that as long as we have a url we can check and a regex we can use to obtain the current version we should be able to do this.

We agreed to add these in piece meal starting with homebrew with a hope of having at least those instructions in by the 1.8 release as there's an easy way to determine the current version being shipped via url https://raw.githubusercontent.com/Homebrew/homebrew-core/master/Formula/go.rb (stable do >> url or stable do >> version)

cc @broady to confirm this approach is doable and sane.

[minux]

Unless we make the check dynamic, displaying stale information is worse than not showing any information at all. Why do we need to show the versions provided by each and every package managers? What could the user do if she sees that her favorite package manager doesn't contain the latest version? Either she accepts the older version or she install from our binary distribution, which makes further update of Go harder as it's not tracked by the package manager. Neither solution is ideal. Why couldn't we distribute deb/rpm packages that could be installed and tracked by those package managers instead? We can't help with the outdated Go packages case, but we can make the users' other choice less painful. (We don't need to go to great length to make our deb/rpm package conforms to the standard set by respectively packager, we just need to wrap our regular .tar.gz distribution in a deb/rpm container with enough metadata to make the package manager able to install/remove/update our packages.)

[rsc]

@minux, the point is to allow people to use their package managers without putting us in the business of maintaining those packages. Creating our own deb/rpms defeats the point. Windows and OS X are just as important as Linux anyway.

And yes, the plan is to make the check dynamic or at least high enough frequency (once every hour or even five minutes might be fine) that the page is basically always up to date, without us being involved.

[broady]

cc @broady to confirm this approach is doable and sane.

Yes, I like the approach of displaying a small note if the version in the PM is out of date.
Happy to implement this.

What could the user do if she sees that her favorite package manager
doesn't contain the latest version?

I've seen impressively fast update times from Homebrew, at least (updated within hours).
If a package manager is not keeping up to date, then we would consider removing it from the download/installation page.

[spf13]

Just following up a bit on this.

I've talked to the people who have maintained the homebrew & chocolatey Go packages. Brew takes between a few hours and a few days to merge in a new recipe. Chocolatey averages around a week.

@broady and I talked over the design for the proposed update and determined it should happen post 1.8 release as it requires some new wiring that we'd prefer to defer to after the release.

[rsc]

This is a good idea and as a proposal I think it can be accepted. Will leave to @spf13, @broady, and @rakyll to work out any further details. Thanks very much for proposing this. I think it will be a big help.

[nathany]

Since Rust was mentioned as inspiration, I'll mention a minor issue I ran into with Rust installations.

Over the course of time, I managed to install Rust in two different ways. With Homebrew, which I regularly update, as well as with rustup via the shell script. The Rustup installer revised my PATH automatically, in a .profile file that I rarely use (preferring .zshenv). It came in the PATH before the Homebrew version, so I was using an older version of rustc without realizing it.

I'm not sure if there is a way to avoid such mixups of multiple installation methods. It may be something worth thinking about though, especially for casual users who may use Go on and off as they are learning it.

At least Go is straightforward in that there is a single go tool for anyone who checks the version they have. With Rust, the CLI tools are rustc, cargo, rustdoc and so on. So it's not entirely clear if the Rust version and the rustc compiler version should be the same (they should be). In some languages, like Erlang, the version is for the runtime, which makes it all very confusing for the newcomer (which version do I have?).

Overall, I'm happy to see this proposal approved. In the case of Homebrew, it makes it easier to install additional tools like delve, which historically hasn't been the easiest thing to install on a Mac.