proposal: a version numbering scheme and release process for Go projects

[davecheney]

Preface

Go projects do not have version numbers in the way it is commonly understood by our counterparts in other languages communities. This is because there is no formalised notion of releasing a Go project. There is no process of taking an arbitrary vcs commit hash and assigning it a version number that is both meaningful for humans and machines.

Additionally, Operating System distributors such as Debian and Ubuntu strongly prefer to package released versions of a project, and are currently reduced to doing things like this.

To put it another way,

Go projects are commonly tracked in vcs repositories and derive their import prefix from their vcs location. Version control systems assign revision identifiers or hashes to various copies of the source they track over time. These vcs hashes are ideal for identifying a particular copy, or revision, of a Go project. However vcs hashes are less useful at answering other types of question like:

• "Am I using the latest copy of this project ?"
• "Are there any bug fixes for version 1.5 available ?"
• "Which version of the postgres driver are compatible with this library ?"

The aim of this proposal is to establish a single recommended procedure for releasing Go projects, in the same way that gofmt defines a single recommended way to format Go source code.

Proposal

This proposal seeks to establish a recommended version numbering scheme and minimal release process for Go projects.

A Go project is a collection of one or more Go packages whose source is tracked in a vcs repository.

A repository root, as defined by cmd/go, identifies both the unique import path prefix for the Go project, and the vcs repository that holds the project's source.

This proposal describes a release process for Go projects by tagging the repository which hold the project's code.

This process is intended to be light weight and will facilitate the creation of tools that automate the creation and consumption of released versions of Go projects.

Version numbering scheme

This proposal recommends that Go projects adopt the semantic versioning 2.0 standard for their version numbering scheme.

This recommendation is informed by the broad support for semantic versioning across our contemporaries like node.js (npm), rust (cargo), javascript (bower), and ruby (rubygems). Adherence to a commonly accepted ideal of what constitutes a major, minor, or patch release will allow Go programmers to benefit from the experiences of these other communities' dependency management ecosystems.

Tag format

Furthermore, this proposal recommends that Go projects adopt a process of releasing their software by applying a tag to their vcs repositories. The format of this tag is defined as

v<semver>

That is, the character v, U+0075, followed directly by a string which is compliant with the SemVer 2.0 standard. Tags which do not fit this format should be ignored for the purpose of determining which versions of a Go project are released.

Out of scope

The following items are out of scope, but would be addressed in a later proposals:

• How Go projects can declare the version numbers or ranges for projects they depend on.
• How go get may be changed to consume this version information.

Additionally, this proposal not seek to change the release process, or version numbering scheme for the Go (https://golang.org) distribution itself.

Thank you for your time.

[crawshaw]

It would be easier to judge this proposal if it included examples from a specific extant tool that would work better with a standard versioning scheme. As it stands, the proposal's preface justifies versions, but not consistent versions.

[kardianos]

To be explicit, this proposal ties requires versions be assigned at the repository level, it cannot be assigned at the package level. Agree with @crawshaw observation.

[davecheney]

To be explicit, this proposal ties requires versions be assigned at the repository level, it cannot be assigned at the package level. Agree with @crawshaw observation.

@kardianos this is correct. Go packages belong to projects which live inside vcs repositories. This is a proposal to release these projects by tagging those repositories with semver tags.

[davecheney]

It would be easier to judge this proposal if it included examples from a specific extant tool that would work better with a standard versioning scheme.

@crawshaw thank you for your comments. To give two concrete examples:

\1. For my project, gb I provide a helper plugin, gb-vendor, which assists users in maintaining the contents of the code they have vendored. gb-vendor also provides an update command, but at the moment the best we can do is update from the currently vendored revision to the latest on the branch, if users want something else they have to manually update to a different revision.

This is because, given an arbitrary branch and revision, it is not possible to say "give me the latest stable version". Sure, if you knew the latest "blessed" revision, you could do that, which is effectively what gb-vendor users have to do now, but then again, if they already knew the revision they wanted to switch to, they wouldn't ask questions like "give me the latest stable version".

\2. For my day job, both Debian and Ubuntu have a strong preference to only package released versions of software. Ubuntu want to make their distribution an excellent platform for Go developers by packaging up all the commonly used dependencies, as we do for other popular languages like Ruby and Python. At the moment Debian maintainers are forced to do things like this, which are of no use to anyone because it's a made up un released version number assigned by the packager, not the owner of the source code.

Now, that is not to say that all projects do not have usable versions, docker, kubernetes, and coreos, the three biggest projects in Go, use the form I proposed above and that benefits all the downstream consumers of those projects.

As it stands, the proposal's preface justifies versions, but not consistent versions.

If this proposal removed the use of semver, and made the version string opaque, that is to say v<x.y.z> where x, y and z were not specified, would you be more favourable to this propsal ?

[tianon]

As a Debian Developer, Go user, and Docker project maintainer, I'm definitely very +1 to explicit versioning. I'm not personally attached to any particular scheme (semver is nice), but having explicit points in time where a package maintainer has said "I feel good enough about my library/program at this particular point in time to attach some extra meaning to it" is really useful, even if they're simple monotonic increases like what systemd uses.

As a simplistic example, it's easy for users to say "I'm on version 34 and hitting bug XYZ" and the maintainer can respond with something simple for users to understand like "that was fixed in version 36".

Semver specifically is nice when used properly, but it does apply a light layer of "social contract" between package authors and consumers such that (for example) version 1.0.0 and version 1.0.1 are "compatible" (for varying values of compatible, I suppose).

The fake VCS-based version numbers used in Debian can be useful (0.0~git20150526.1.5478c06-1 for example), but IMO should only be necessary for snapshots between releases (such as 1.2.3~git20150526.1.5478c06-1).

[adg]

So what does the implementation of this look like? Where would the convention be documented? Would it be enforced by the Go tool chain somewhere? Is this analogous to the vendor-spec proposal?

[davecheney]

So what does the implementation of this look like?

I imagine the implementation would be a design document in the proposal repo, like the security policy.

Where would the convention be documented?

In the document above.

Would it be enforced by the Go tool chain somewhere?

Not at the moment. Releasing code is outside the purview of the go tool at the moment and go get has no capabilities to choose which tag to check out at the moment. However, there are many other tools, both in the Go sphere, and outside, that consume Go projects and will benefit from being able to identify released versions of Go projects.

Is this analogous to the vendor-spec proposal?

Not directly. The vendor spec is about recording what is vendored into the local copy. This proposal is about providing a way for Go project owners to indicate which revisions of their repository they want to be considered "released", in the same way that we develop Go on master, but tag a specific revision when it's ready to be released.

[adg]

I imagine the implementation would be a design document in the proposal repo, like the security policy.

The security policy will be documented somewhere outside the proposal repo. Either the wiki or on the web site (probably the former). I would expect the same outcome both for this proposal and vendor-spec.

I understand how this is distinct from vendor-spec. My question about vendor-spec was asking if this were basically "version-spec". I think that vendor-spec might be influenced by this proposal, though.

[davecheney]

The security policy will be documented somewhere outside the proposal repo. Either the wiki or on the web site (probably the former). I would expect the same outcome both for this proposal and vendor-spec.

I am sorry, this was my misunderstanding, I'm still getting used to the proposal process.

I agree that the website, maybe in one of the 'how to write Go code' pages would be a good place for this information, or possibly on the wiki. For me, the location is not as important as establishing a single recommended way to tag a Go project in their repos.

I understand how this is distinct from vendor-spec. My question about vendor-spec was asking if this were basically "version-spec".

I'm sorry for misunderstanding. I guess this is a version-spec if you want to think of it that way.

I should add that in this initial proposal I have err'd on the side of brevity to focus on the outcomes, not the procedure. I fully expect that should this proposal move forward, the next step would be a design document as you outlined in the proposal process.

I think that vendor-spec might be influenced by this proposal, though.

Possibly a bit, but I don't think either proposal should be dependant on one another. The former, this one, is about a format for a release tag on repositories, while the vendor-spec caters for tags, branches, and revisions equally, and it is hard to see this proposal designing a tag format that is impossible to express in the vendor-spec.

Thanks

Dave

[kardianos]

I've stated this elsewhere, but if a version was specified to be consistent I would prefer a spec that allowed sub-paths to be separately versioned. "websocket-v2.2.3" or "context-v1.0.0" in addition to the repo wide option specified above.

[crawshaw]

I don't have any problem with the specifics of this proposal, I'm just a little confused by the scope. The go tool has opinions about Go packages and the layout of the repositories they exist in, but has no concept of a project. Introducing project-level versioning requires introducing the notion of a project.

The kinds of questions I might ask about projects:

• Can a project span repositories?
• Can a single repository contain multiple projects?
• How many main packages can be in a project?
• What names a project?

[kardianos]

@crawshaw Dave has stated elsewhere that the framing is one version per VCS repository. So the repository becomes the "project". This is a stronger notion than even what the "vendor" folder implies as it defines a single root.

[crawshaw]

That's fine, it's just that project needs to be defined before project versioning can be defined. That probably needs to happen as part of the proposal.

[gravis]

This looks like the early days of ruby on rails. Before having bundler, we had to put dependencies in /vendor/plugins. It lasted for some years, then rubygems became the standard. The dependencies were defined in the code, and a rake task was used to install the deps: rake gems:install (<= search for issues with that on google...)
It introduced a lot of issues, because the gem (ruby packages) were installed using the application itself. Of course, sometimes, the app couldn't load enough code to install the gems, and the user was stuck.
Then, bundler was designed, an independent tool, reading one config file (Gemfile). The locked versions are saved to Gemfile.lock, and installed most of the time in a dedicated "gemset" (a GOPATH).
I writing all of this, because it seems go is heading the same way, and will spend months/years to adopt what the other languages figured out in the last years ;)
What go could do better, is the namespace of packages, because the source of the package is obviously explicit. There's no need for a central registry, VCS tags are more than enough.

[davecheney]

@crawshaw

I don't have any problem with the specifics of this proposal, I'm just a little confused by the scope. The go tool has opinions about Go packages and the layout of the repositories they exist in, but has no concept of a project. Introducing project-level versioning requires introducing the notion of a project.

This is correct, the go tool does not have a concept of how the packages on your GOPATH came to be there, but go get does. Before fetching, go get must canonicalise the import path, find some prefix of the path you provided you gave it to the location of a remote repository, which it fetches in its entirety. It is this import prefix, or what the Go tool calls, the repository root, that is a project.

With that said, any changes to the go tool are outside the scope of this proposal.

I've updated the original proposal with a definition of a Go project.

The kinds of questions I might ask about projects:

• Can a project span repositories?

No. Each repository is a versioned and released individually.

• Can a single repository contain multiple projects?

No. The contents of a repository are released together as a whole, golang.org/x/tools is a great example of that.

• How many main packages can be in a project?

Unlimited. There is an unfortunate coincidence with the gb definition of a project. If you think it would help I can reword this proposal to remove all references to Project, and replace it with Remote Repository.

• What names a project?

Go projects are named by the import prefix that is defined by their repository root. Import prefix and repository root are defined by the documentation on the go tool.

[davecheney]

@gravis thanks for your comments and your perspective.

I agree that Go is lucky that we have always had a package namespace (although noting that Go packages are not hierarchical) so we can lean on this heavily.

For example, what defines a project, and who gets to name a project ? Luckily go get already defines the rules for what is a remote package and how it is fetched, so questions of ownership can be punted to a 3rd party.

Who owns the github.com/docker/docker project, and thus the github.com/docker/docker/... namespace ? It's who controls the docker repository on the docker account at github.

Who owns the golang.org/x/tools project ? It's whoever controls the dns name for golang.org and can host a web server to serve a meta redirect.

[thockin]

My 2 cents: There's not much here to disagree with (and yet...). I think versioning is good, but I also happen to work on a project that versions in exactly this way.

With that said, any changes to the go tool are outside the scope of this proposal.

Without some hypothetical usages of this convention, it might be hard to justify. For example, could go get HYPOTHETICALLY take a version string and fetch that specific revision? Could godep or gb vendor HYPOTHETICALLY take a version to import and/or rewrite imports (gopkg.in style) to versioned?

Those are more exciting than an abstract convention which, to be clear, is painfully obvious to me (but clearly not everyone).

TL;DR

+1 but how do we USE it?

[davecheney]

@thockin thanks for your comments. I understand your frustration that this does all appear to be hypothetical at the moment.

My rational for starting as small as possible is (and I think well founded) that a proposal that introduced semver and required everyone to tag their repos and had to make a change to go get and had to fit into the overfull 1.6 release cycle would be viewed with blank stares.

This is the smallest possible thing that I think can make the dependency management situation for Go programmers better. I think it is a reasonable thing that improves the life of Go developers and the ecosystem downstream from them even if the go tool does not grow to embrace it, as there are several dozen other competing Go dependency management tools in the market.

With that said, here are a few concrete areas that would benefit immediately

• Debian and other operating systems can use this information today.
• I plan to rev the manifest format that gb-vendor uses to include the tag information so I can offer my users a real upgrade command, not just one that grabs the latest head of the branch they are on.
• godoc.org can show users the docs for the version of the package they are using, not just show the latest that was in the upstream repo.

[thockin]

Oh, sorry if I mis-communicated. I think this is obviously correct. I'm
just a concrete sort of fellow, so having a mental model of what this would
enable helps me see the purpose. I assume I am not alone.

No "frustration" here, other than wanting to help get a resounding "YES" to
something like this.

On Tue, Aug 25, 2015 at 11:36 PM, Dave Cheney notifications@github.com
wrote:

@thockin https://github.com/thockin thanks for your comments. I
understand your frustration that this does all appear to be hypothetical at
the moment.

My rational for starting as small as possible is (and I think well
founded) that a proposal that introduced semver and required everyone
to tag their repos and had to make a change to go get and had to fit
into the overfull 1.6 release cycle would be viewed with blank stares.

This is the smallest possible thing that I think can make the dependency
management situation for Go programmers better. I think it is a reasonable
thing that improves the life of Go developers and the ecosystem downstream
from them even if the go tool does not grow to embrace it, as there are
several dozen other competing Go dependency management tools in the market.

With that said, here are a few concrete areas that would benefit
immediately

• Debian and other operating systems could use this information today.
• I plan to rev the manifest format that gb-vendor uses to include the
  tag information so I can offer my users a real upgrade command, not
  just one that grabs the latest head of the branch they are on.
• godoc.org can show users the docs for the version of the package
  they are using, not just show the latest that was in the upstream repo.

—
Reply to this email directly or view it on GitHub
#12302 (comment).

[mattfarina]

&tldr; +1

One question, why the v at the start of the version number? Should that be required or optional?

I looked and npm (for Node.js) and composer (for PHP) the v is optional. In others such as Cargo (Rust) or Bower (JavaScript) I don't see any mention of v. It appears to not be used.

There are Go projects with and without the v prefix today. Can it be optional?

As one of the developers of Glide, another package manager, I can say that handling this well is already a request and need.

Going beyond issues, such a distros, you can see what version of the project/package API you're using. If I've used a package who bumped their API version from 1.2.3 to 2.0.0 I need to know this because the API contract changed. Versioning can help with the API contract and expectations.

[technosophos]

To @mattfarina's point, SemVer 2.0 removed the requirement for the superfluous v. I don't see a compelling reason to require it in this context.

[kostya-sh]

A few concerns:

1. I beleive that incompatible versions of the same library should use different import paths. Otherwise they cannot co-exist as dependencies of a single project. Having them tagged with two different tags is not enough or even harmful.

   There a lot of examples of Java libraries where following this simple rule would have made life of library and application developers much easier. E.g. protobuf 2.4 vs 2.5, various versions of asm library, hadoop-1 vs hadoop-2, various versions of jetty, etc.

2. Very often for applications (not libraries) it makes little sense to use semver. Something like build number + git hash works well enough. Think about version numbers of such applications like Chrome, Firefox, systemd, less, etc.

[tianon]

@kostya-sh if the version number is in the import path, then every source file using the import must be updated every time the dependency is (which is a lot of churn and merge conflicts)

IMO if two versions of a package are different enough to be used concurrently, they ought to live under separate paths anyhow (or use a scheme like gopkg.in), especially since using both concurrently is probably an edge case for most.

[thockin]

This is why gopkg.in only versions to the major number. If I import
libfoo.v1 and libfoo.v2 at the same time, I have to rename something. If
the tooling doesn't handle it well, users do their own thing and everyone
ends up with bespoke vendoring scripts. Puke.

On Wed, Aug 26, 2015 at 7:43 AM, Tianon Gravi notifications@github.com
wrote:

@kostya-sh https://github.com/kostya-sh if the version number is in the
import path, then every source file using the import must be updated every
time the dependency is (which is a lot of churn and merge conflicts)

IMO if two versions of a package are different enough to be used
concurrently, they ought to live under separate paths anyhow (or use a
scheme like gopkg.in), especially since using both concurrently is
probably an edge case for most.

—
Reply to this email directly or view it on GitHub
#12302 (comment).

[kostya-sh]

@tianon, I didn't propose to include version number in the import path. What I said was that incompatible versions of the same package should be different packages (with different import paths). Though in practice this probably matters only for popular well established libraries used by many projects. E.g. Java protobuf 2.4 vs 2.5. Note that the major version is the same but they are still incompatible in a subtle way.

If the goal of this proposal is to create a document describing best practices for versioning then it should cover topics that I raised including backward compatibility. A document describing best practices for versioning, releasing, using vcs, maintain backward compatibility, etc would be very useful.

If the goal of this proposal is to make semver versions standard in the Go community then I think:

• tools support should come first. gofmt mentioned in the proposal is the excellent example why
• it is fairly easy to enforce the version format but it is much more difficult or even impossible to make people follow the rules behind it

[davecheney]

@technosophos @mattfarina thank you for your feedback. To reply specifically to your comments about the v prefix. This was the most contentious part of the discussion on go-pm that preceeded this proposal.

The preference for a v prefix comes from observations that three of the biggest Go projects, Docker, Kubernetes, and CoreOs all use that variation, it is also widely used by our language contemporaries like Rust, Nodejs and Bower (javascript). With that said, I'm sure an equal number of prominent examples could be found that do not have a v prefix.

If this proposal were amended to make the v prefix optional, would that be acceptable ?

[davecheney]

@kostya-sh thank you for your feedback. To address your points

I beleive that incompatible versions of the same library should use different import paths. Otherwise they cannot co-exist as dependencies of a single project. Having them tagged with two different tags is not enough or even harmful.

I'm sorry but I do not agree with you. There must be only one copy (not version) of any package in a Go binary; the linker requires it. To subvert this by renaming the package, even well intentioned, so as to permit go get to have some way of identifying different major versions of packages, is in my opinion a mistake.

To give some background:

Canonical were, I believe, one of the first to attempt to use this method with the mgo driver back in August 2012 when we had to make an incompatible change to the API. At the time we thought were were rather clever, but since that time I have come to believe this was a mistake, for the following reasons:

1. Putting the major version in the API is not sufficient for a repeatable build, at best it only guides Go get to the major API version. It does not give Go get sufficient information to fetch a specific release version or a specific revision. Using these versioned import paths, the Juju developers still had to develop an additional tool to make sure everyone had the same revision of every dependency checked out.
2. Encoding a version number in the import path has a high operational overhead, I recently saw a patch from a co-worker that touched 220 files to update a dependency that used this format via the gopkg.in proxy. What is worse, this changed from v5 to v6-unstable, so you just know that in a week or so another 220 file patch will be coming to remove the -unstable prefix.
3. Encoding a version number in the import path has a high cost for new adopters of the language. You have to find every reference in your project to one versioned import path, and replace it with another, even in the face of conditional compilation, even in tests, and so forth. As someone who is passionate about teaching newcomers about Go, the conceptual burden of educating newcomers about this requirement would be far to high.
4. Finally, even with the most rigorous development standards for their own code, Go developers are still at risk from the dependencies they rely on not getting it right. With versioned import paths it is very easy to see two arms of a dependency graph importing different versions of a database driver who's init() function registers the driver. Have a read of the documentation for sql.Register. This is not a hypothetical problem, and one that must be prevented at all costs.

There a lot of examples of Java libraries where following this simple rule would have made life of library and application developers much easier. E.g. protobuf 2.4 vs 2.5, various versions of asm library, hadoop-1 vs hadoop-2, various versions of jetty, etc.

I think java has made a serious mistake by developing more (OSGI) and more (Project Jigsaw) complicated methods of classpath resolution to continue to support multiple versions of a library in the same JVM runtime. This complexity is entirely self inflicted and not something I want to see in a Go dependency management solution.

Very often for applications (not libraries) it makes little sense to use semver. Something like build number + git hash works well enough. Think about version numbers of such applications like Chrome, Firefox, systemd, less, etc.

I agree that for end user applications, projects which produce a program or server binary, versioning makes less sense, as by definition no code consumes them.

However, there are many downstream consumers of all Go projects that do care about version numbers, operating system distributions are a major one. I don't think your examples provide a compelling exception to the rule to argue against the value of semver.

[davecheney]

@kostya-sh to address your points

If the goal of this proposal is to create a document describing best practices for versioning then it should cover topics that I raised including backward compatibility. A document describing best practices for versioning, releasing, using vcs, maintain backward compatibility, etc would be very useful.

These are good points, if this proposal moves forward to the design document stage it should address these points.

If the goal of this proposal is to make semver versions standard in the Go community then I think:
tools support should come first. gofmt mentioned in the proposal is the excellent example why
it is fairly easy to enforce the version format but it is much more difficult or even impossible to make people follow the rules behind it

I disagree, as I mentioned about to @thockin this is the smallest possible proposal I can think of to make forward progress. Yes, larger proposals would be more comprehensive, but would take significantly longer to achieve consensus and would then be further delayed by a requirement to be scheduled into the Go release cycle -- do you really want to wait up to a year to see this implemented in a released version of Go ?

This proposal is the smallest I can make. It pushes all the questions of what versioning means to an accepted standard, semver, which is widely used across languages who we consider to be contemporaries (or competitors) to Go, and lets us benefit from their experiences rather than reinventing what would likely be a very similar wheel.

If you agree with that position, then this proposal boils down to a format for vcs tags that I believe can achieve consensus easily and can be of use immediately for all the tool makers, godoc.org maintainers, and operating system distros, in the Go ecosystem.

Thanks

Dave

[perillo]

On Thu, Nov 26, 2015 at 7:51 PM, Axel Wagner notifications@github.com
wrote:

I am sorry to only have skimmed the discussion (it's very long), but I
think this point wasn't made before:
I would request that a proposal like this, that proposes semantic versions
also gives a good definition of what the individual levels of compatibility
as specified in the SemVer spec actually mean for go (as there seem to be
either no, or very inconsistent notions of this in the community). In
particular, as I've argued before, I don't think the language allows minor
minor, only patch releases and major releases./You can read my argument
here
http://blog.merovius.de/2015/07/29/backwards-compatibility-in-go.html
but it mainly comes down to the fact that due to dot-imports, type
embedding and strict typing, every API change potentially breaks the
build of some reverse dependency.

compatibility for Go is specified here:
https://golang.org/doc/go1compat
so the same definition of compatibility can be used for external packages.

I might be able to agree that this is unnecessarily nit-picky, but I do
firmly believe, that if the community agrees on a semantic versioning
scheme, they also need to agree on the semantics of that scheme (i.e. if
you arbitrarily allow some breakages in minor version, we should at
least agree which) and that agreement shouldn't be just hand-wavy (so
far, the strictest definition I could get out of people was "minor changes,
like adding a field").

Also, I'm a bit unhappy that this proposal conflates "releases" and
"versions", which I would consider two different things.

I agree that releases and versions are different.
Versions can be considered tagged releases.

I agree with lots of the cited benefits of releases but I am not at all a
fan of SemVer. I would prefer a release process like "master is always the
current stable release, development happens on a separate branch that is
periodically merged into master when it's considered stable", which would
make all the current tools pretty happy (but wouldn't provide nor be as
restrictive as requiring SemVer).

Note that, should versions support be added to standard Go tools, you will
never be forced to use them.
The current behavior of go get will be the same: if no version is specified
it will get the latest revision (not the latest version).
I expect that any external tool will offer you the option to get the latest
revision and not the latest version.

Also note that SemVer has nothing to do with this problem.
A release process like "master is always the current stable release," is
simply a release process where no version is actual assigned to a
revision.

[Merovius]

@davecheney

Your major objection to the semver standard is a lack of clarity over how version numbers are chosen.

Not really. My major objection is about the whole idea of defining "breakage" and "compatibility" in terms of an API in isolation, i.e. about the "semantic" part in "SemVer". I see the usefulness of versions to give human readable names to releases, I also acknowledge that it is useful to imply a total ordering on versions. But if you want to assign semantic meaning apart from that to a version, you would either a) have potentially false version-matches, e.g. I require 1.7.0 or above and 1.8.0 breaks me because the author considered a change-non-breaking but there is no such thing in go or b) have probable false version-mismatches, e.g. I require version 1.0 or above, the author did a minor change in the API that broke nothing in practice, but to avoid a) they chose to increment to version 2.0 which is assumed to break me as the major version increased.

I think it's hard to reconcile any SemVer-equivalent versioning scheme with these concerns, while a purely chronological versioning scheme (e.g. every release increments a counter) that isn't used to make semantic deductions could.

As such, a tool to help in determining the next version number can't really ameliorate my concerns. In fact, the mentioned blog posts and my position towards SemVer is born out of me trying to write exactly this tool about half a year ago. I realized half-way through that there is no useful (read: prevents breakages but does not restrict language use too much) notion of compatibility for go.

So I maintain my position that the best way would be to at least not use semantic versions, but treat release-tags as convenience identifiers that have no meaning apart from that (and possibly a purely chronological ordering). I don't think that versions are the correct way to tackle the compatibility problem, but that this should rather be done by better tooling and conventions to work around that and actually look at the code that uses an API, instead of the API itself. A set of tools that helps you to choose the correct version to install and package based on the actual API and the code of the actual users of that API, instead of some ultimately heuristic number assigned to it.

@perillo Thanks for your comments :) I have read the compatibility guarantee extensively, but I don't think it is a sufficient definition, because a) most community members I have talked to so far consider things that break this guarantee to be non-breaking and b) even the go stdlib constantly breaks it, if you are being precise. I see the go 1 compatibility guarantee more as a statement of intent then an actual definition of what compatibility means (and to be clear: That's still far better than anything else I've seen for other languages). I recommend reading the blog post I linked to earlier on why I think this needs more spelling out.

Also note that SemVer has nothing to do with this problem.

I respectfully disagree. It has everything to do with this problem, it is part of the proposal and my issues with the proposal are completely attributed to this.

A release process like "master is always the current stable release," is
simply a release process where no version is actual assigned to a
revision.

Precisely. I wanted to demonstrate that it is possible to have releases without having versions (which is what I would prefer).

[perillo]

@Merovius
I have read your blog post, but this phrase: I really love go, I really hate vendoring and up until now I didn't really get, why anyone would think go should need something like that means that we have different types of workflow when working with code.

I work as a freelance developer. A client ask me a software, I develop it, release it and after some time I forget about it. Suppose after a year the client call me telling he need a new feature or a bug fix: do you think I want to spend more time than necessary trying to update everything to the latest revision, checking that it really works?

SemVer does not solve the problem, but it, with the aid of good tools, will make my life better.

[peterbourgon]

@Merovius With respect, I believe you are letting the perfect become the enemy of the good. Semantic versioning is the de-facto standard for versioning in ~every programming ecosystem (to some order of approximation). Developers expect it, or something like it, when approaching this problem space. Its failures, which you identify, do not prevent it from being useful, as @perillo notes. And, it seems that adopting this proposal doesn't prevent package authors from opting out of versioning at their discretion, and asking their consumers to vendor at specific revisions. — What do you think?

[perillo]

@peterbourgon
Right; the tools should work as they work currently when no explicit version is required.
New tools should offer support for requiring revisions, instead of versions (but IMHO, they should only support fetching the latest revision (as go get); the user can always cd into the vendor directory and git checkout at a specific revision.

[perillo]

s/required/specified/

[Merovius]

@perillo But that isn't really an argument now, is it? SemVer does exactly zero to make that situation better. Just use the revision you used before, develop against that and if there were breakages, you need to figure that out either way. I don't understand what you are getting at with that.

@peterbourgon Note, that I started not by demanding that this should be abolished (though I think it should), but by stating that someone suggesting SemVer should also give a definition of what that means. I firmly stand by this, I will be unhappy, but can accept SemVer as an 80% solution, but only if we collectively agree what kinds of changes will require an increment of which version number. Because a simple definition leads to counterproductive version numbers.

[peterbourgon]

@Merovius

someone suggesting SemVer should also give a definition of what that means . . . [I] can accept SemVer as an 80% solution, but only if we collectively agree what kinds of changes will require an increment of which version number

Good news, then: this is already defined to a commonly-accepted level of precision in the spec:

Given a version number MAJOR.MINOR.PATCH, increment the:

MAJOR version when you make incompatible API changes,
MINOR version when you add functionality in a backwards-compatible manner, and
PATCH version when you make backwards-compatible bug fixes.

[Merovius]

@peterbourgon I don't understand why you'd think that's relevant to my points. I thought I made myself relatively clear before. I know and understand the SemVer spec, but if that's all the definition you are giv, then every API change in go is a major version increment. That is hardly a useful versioning scheme. To make SemVer useful for go, you need to define what "backwards-compatible" means, because that's less straightforward then you'd think and everyone seems to have a different opinion about that.

[davecheney]

@Merovius thank you for your comments. I can see you have strong views on versioning, releasing, and methodologies like semver itself. Like others who have replied, we'll have to agree to disagree, and as SemVer is key to this proposal, I'm sorry but I cannot accept your suggestions into this proposal.

[Merovius]

To repeat myself again: My suggestion was primarily to at least define what constitutes a "breaking change", or rather a "non-breaking change". I don't understand why this should be impossible or conflict with the target of using Semantic versioning, in fact I would think that it would promote the introduction of SemVer. I have strong opinions on the merits of SemVer, yes, but I agreed to disagree on those from the very beginning. But so far no one has even attempted to give this definition, acknowledge it's necessity or to give a reason to not give such a definition. Clarity on what constitutes a breakage seems like a hard requirement for introducing SemVer to me, because after all the spec references this term and only makes sense after this term is clear.

I am going to leave this discussion now, because I feel unwanted and ignored.

[davecheney]

@Merovius I'm sorry you feel that your views are not being respected. If I have given that impression, please accept my apologies.

With respect to your specific concerns about what defines "breaking or non breaking change", I hold it is whatever Semver says, which is clearly stated at the top of the summary section of http://semver.org/spec/v2.0.0.html.

You argue that what constitutes a breaking change is much larger than what is commonly accepted, as pointed out by your well written article. As I said above, I agree completely with what you wrote, and see this as a point which tools can help as the rules for "breaking vs non breaking" are not ambiguous -- we already have this with the Go api tool, although i think you are arguing for some even more stringent, and that is also fine.

In summary, I agree with your points on breaking changes being almost any addition to an existing Go type (sorry if I have not reproduced the depth of your argument), but that seems to me to be tangental to this proposal. This proposal is to adopt some kind of release process -- if there is no release process then arguing about the nuances of the application of Semver to Go APIs serves no purpose.

Thanks

Dave

[rsc]

@Merovius, a few points.

1. While it may be true that "every API change potentially breaks the build", that's not a terribly useful stance. For example, adding a method to an existing type is one kind of change; deleting that type is qualitatively different. It is helpful to distinguish these two, even if both are potential breaking changes.
2. Go itself is using SemVer numbering (although we'd never heard of it when we started) to good effect. So if we need a formal spec about what constitutes a breaking change and what does not, then Go's own rules (https://golang.org/doc/go1compat) might be a decent start.
3. Although the proposal does mention SemVer explicitly, it seems to me best not to get too hung up by the specific version numbering scheme. The reason this proposal is stalled is lack of a clear idea, assuming one can create a semantically useful versioning scheme, what you do next.

[mattfarina]

For anyone who wants to back SemVer into their toolchains I wrote up a quick tutorial (with a package) for the Gopher Academy 2015 Advent

[nathany]

This proposal would ratify what many people and tools are already doing. Docker, Kubernetes, and CoreOS were mentioned. When I wrote about tagging repositories over two years ago, there were already third-party tools that worked with the v<SemVer> format.

The fsnotify library currently uses gopkg.in to turn tagged releases into new import paths for each major version. One feature gopkg.in provides is the distinction between released code and what's sitting on master. The url http://gopkg.in/fsnotify.v1 retrieves the latest v1.x.y tag. Code sitting on master while preparing a release is not made available until tagged. It also has experimental support for prerelease versions. Adding a v2.0.0-unstable tag would provide an opt-in way to retrieve an upcoming release.

While gopkg.in has been useful, I think it is quickly becoming unnecessary. Many people vendor code now, and GO15VENDOREXPERIMENT is further formalizing that recommendation. Everything gopkg.in provides can be replaced with tools such as Godep and gb _if_ they can retrieve the latest tag rather than master, or pull in prerelease versions or previous versions.

Gopkg.in is built on the idea of taking a common VCS practice of tagging releases and providing multiple import paths, derived from the recommendation in the Go FAQ. @davecheney has already listed the issues his colleagues at Canonical had with the multiple import path approach.

There are situations where creating an entirely new library with a new name is appropriate. But for bumping a major version to remove deprecated APIs, there are better alternatives to "create a new package with a new import path", and I believe the FAQ deserves an update.

Providing more visibility into "a release process for Go repositories" will allow third-party tools to start expecting v<SemVer> tags, using them to benefit their users.

Updating GoDoc.org to show documentation for any version may or may not be worth the effort involved. A more reasonable start would be to simply display the version number for the most current release. That will help make this recommendation to tag releases more visible. The distinction between tagged and untagged repositories could also be used to help ranking, by boosting released libraries higher than hobby projects.

What comes after that? I don't know the future, but I believe this is a good step, and one worth taking. Thanks @davecheney.

[mattfarina]

@nathany If I understand what @rsc is say the issue is that adding SemVer as a recommendation without having part of the go toolchain using it is the issue. I'm not sure we'd find agreement on the right way to use SemVer in the toolchain right now. I think we really want SemVer so we can start to use it in tools and figure out directions it can be useful. So we can leverage versions like other communities.

If community things that aren't going to be built directly into the tools aren't appropriate for this forum do we need something similar to the PHP FIG for the community?

[davecheney]

If community things that aren't going to be built directly into the tools aren't appropriate for this forum do we need something similar to the PHP FIG for the community?

That sounds well meaning, but entirely too much bureaucracy. How about people just tag their releases ?

[rsc]

Yes, my objection is that this proposal does not propose any actual changes to any part of Go. It is not actionable in its current form. I'm still not even sure it's a good idea, in large part because one way I evaluate whether something is a good idea is to understand its effects, and this proposal has no direct effects.

Build a system that uses semantic versions effectively. Demonstrate what great tools can be built. Give people incentives, and they will tag their releases no matter what the Go team says. Certainly the Go team saying "you should do it this way" has not in the past been successful without these things (think not-vendoring).

[davecheney]

I feel that it is time to draw the discussion to a close. The proposal guidelines clearly state "a lack of agreement means [the proposal is] declined", as such it is appropriate that I withdraw this proposal.

I want to extend my sincere thanks to all who have contributed to this debate. I remain passionate about improving the tools Go programmers have to manage package dependencies in large Go projects and your feedback has been invaluable to me and my future efforts.

Thank you again

Dave

[rsc]

Fair enough Dave. Thanks for the time you spent on the proposal. I certainly agree that a solution to versioning would be great, and that SemVer seems like a very good basis for building such a solution. The devil is always in the details, of course.

[sapioit]

For the Versioning Numbering Cheme I would, any day, choose Explicit Versioning insted of Semantic Versioning. Why? It's so simple the first four words from the title of the link explain it for most of the developers. And besides that, no more ambiguity caused by the "Major" and "Minor" fields of the version number. Here's the promissed link: Release.Breaking.Feature.Fix or why the Semantic Versioning should be replaced with Explicit Versioning as soon as possible