Seamless upgrade from 0.x to 1.0

[kornelski]

Problem

Release of 1.0.0 is a semver-incompatible change, which is inconvenient when a crate has already established a stable API and a userbase on a 0.x version.

Crates usually start from 0.1.0 and evolve their API while releasing 0.x versions. Once the crate reaches maturity and becomes de-facto stable while still in a 0.x version, there's no easy way to call it 1.0.0 and move the library and its users to v1. Versions 0.x and 1.0 are semver-incompatible, requiring manual upgrades. This fragments the userbase, causes churn, and can cause incompatibilities between crates unless a semver-trick is used.

During development it's hard to predict which semver-major breaking change will be the last breaking change needed (at least for a while), and therefore hard to predict when to commit to releasing 1.0.0. This becomes obvious in retrospect after the latest release stays stable for a while.

Having to use the "semver trick" to move 0.x users to 1.0.0 is far from perfect. It's a trick. It's not obvious. Needs care to forward Cargo features. If the new version is re-exporting the old one, it's also a breaking change to remove the re-export.

The Rust ecosystem has many de-facto stable and widely-used crates in v0.x. Out of the top 250 Rust crates with most downloads and/or rev deps, the majority (143) is still on v0.x. Being able to bless the last 0.x version as 1.0.0 without migration headaches could help at least some of the crates to move out of "unstable" 0.x versions. I'm hoping this would include libc, which in v0.2 is one of the most used stable Rust crates.

Proposed Solution

There could be a mechanism that lets Cargo automatically unify selected semver-major versions of a crate, the same way it unifies minor and patch versions.

[package]
version = "0.12.34"
please-seamlessly-upgrade-me-to-version = "1.0.1"

or maybe the other way:

[package]
version = "1.0.0"
I'm-a-drop-in-replacement-for = "^0.12.34"

Either way the expected behavior would be that 0.12 and 1.0.0 would be treated as a compatible range, and all uses of 0.12 were replaced by 1.0.0, the same way that a v1.0.1234 would be unified and bumped to v1.1.0.

Notes

No response

[ia0]

I might be misunderstanding something, but this seems to assume that crates don't upgrade their dependencies between incompatible versions (including 0.x.y to 1.0.0). If this were true, I would argue this is the problem that needs fixing. Not upgrading between major versions is a sign of an unmaintained crate and an open door to transitive security vulnerabilities.

Crates should not be afraid to release the exact same code from a 0.x.y version to a 1.0.0 version (although it is good practice to provide a migration guide between incompatible versions in the crate documentation, which in the case of the exact same code could be something like "smooth migration, no breaking changes").

[kpreid]

Not upgrading between major versions is a sign of an unmaintained crate and an open door to transitive security vulnerabilities.

Because Rust has safe code, it is not the case that all code can be sources of security vulnerabilities. Many algorithms are unlikely to have security-relevant bugs even with arbitrary input and even if they do contain bugs.

Crates should not be afraid to release the exact same code from a 0.x.y version to a 1.0.0 version

Doing so will create completely unnecessary incompatibilities that may have a very large cost. For example, if log released a version 1.0 now (without doing the semver-trick to make them equal, since that would mean the code is not the same), it would fragment users of the global logging hook it provides, creating a messy upgrade problem for its dependents — applications have to hold back updates until all their transitive dependencies have released breaking updates, or use two versions of log simultaneously.

Even without the particular trouble of static state, breaking changes are still tedious any time you have a foundational library and dependents which build upon its types and traits. Not every library has this class of problem, but many can.

Of course, this is all a one-time cost in this case, since there is exactly one 1.0, but I think you under-weight the cost of incompatible upgrades in general. And, the point of this issue is that, empirically, 1.0 releases are uncommon, and making them easier would make them more common.

[ia0]

Because Rust has safe code, it is not the case that all code can be sources of security vulnerabilities. Many algorithms are unlikely to have security-relevant bugs even with arbitrary input and even if they do contain bugs.

I'm just going to ignore this comment because it is an invalid argument. It is of the type: You can't argue against A by saying "A sometimes implies B (which is bad)" because "A doesn't always implies B" (where A is not using the latest major version of direct dependencies and B is making it harder for your dependents to apply security fixes when available). It's like arguing that we don't need safety belts because not all cars end up having a deadly accidents (where A is not using safety belts and B is having a deadly accident).

For example, if log released a version 1.0 now

Again, similar issue as above. This is a corner case which should not alter the decision for the general case more than its actual proportion. If this proportion is big (of crates with global state that needs to be unique across the dependency tree), then we have an ecosystem issue. For those corner cases, it is acceptable to do the trick such that the ecosystem can be incrementally updated to the latest major version.

Even without the particular trouble of static state, breaking changes are still tedious any time you have a foundational library and dependents which build upon its types and traits. Not every library has this class of problem, but many can.

Again, same issue as above. Except that it's also mixing breaking changes (as in incompatible API change) and incompatible versions (as in different major versions). I'm not asking for breaking changes, I'm asking to not be afraid of incompatible versions when it actually doesn't break anything. In particular, users of #12425 won't even notice (as stated above, I believe crates should regularly use cargo update --breaking if they are maintained).

but I think you under-weight the cost of incompatible upgrades in general.

And I think you don't make the distinction between incompatible upgrades and incompatible changes.

And, the point of this issue is that, empirically, 1.0 releases are uncommon, and making them easier would make them more common.

I'll restate what you're trying to say but correctly: The point of this issue is that there are crates with both a stable API and a major version of zero (this is in violation of SemVer item 4). The issue claims that those crates are afraid of bumping the major version number because it requires a manual update for their users (and implicitly that maintained crates won't do this manual update).

The problem in your version is that you're missing the part where the crate has a stable API. It's expected that a lot of crates are 0.x.y because a lot of software is unstable. You won't magically make software stable. You also won't magically make software maintained. And trying to camouflage or not break unmaintained crates is a bad idea. Unmaintained crates should eventually be purged from the ecosystem so increasing the cost of their usage is a good thing.

[kpreid]

This is a corner case which should not alter the decision for the general case more than its actual proportion.

In hindsight, I should not have brought up log and its static state. Types and traits reused by other crates are a much more common scenario. In particular, consider the concept of public vs. private dependencies, which has not been implemented as a formal rule in Cargo but is still useful for thinking about dependencies. If a dependency is a private dependency (no public usage of the dependency's types, traits, or statics), then incompatible upgrades of dependencies are (usually) inconsequential. But if it is a public dependency, then an incompatible upgrade (usually) has to cascade to all public dependents.

In these terms, I think the place where we disagree is in how many dependencies are public dependencies. I think that there are enough chains of public dependencies that they are not a “corner case” and it is important to help them out.

Except that it's also mixing breaking changes (as in incompatible API change) and incompatible versions (as in different major versions).

Sorry, I used the wrong words. I meant to say incompatible versions.

I'm not asking for breaking changes, I'm asking to not be afraid of incompatible versions when it actually doesn't break anything.

I'm not claiming that having more incompatible versions will “break” anything. I'm claiming that that having a higher rate of incompatible versions gives maintainers more problems where they, collectively, have to pursue a sequence of incompatible version bumps, and there will be periods where they cannot perform those bumps. For example, suppose we have the following dependency graph (where all involved packages have different maintainers, and all packages make use of types from infrastructure):

infrastructure ← framework
framework ← helper-1
infrastructure ← helper-2
foo-app ← infrastructure
foo-app ← helper-1
foo-app ← helper-2

Then, when infrastructure releases a new major version (whether it is 1.0 or not), helper-1 cannot release a version making use of the newer infrastructure until framework does, and foo-app cannot release a version making use of the newer infrastructure until all of framework, helper-1, and helper-2 have published new incompatible versions. Assuming that all of these packages don't release minor/patch updates to old major versions (as is true for most Rust libraries), foo-app loses the ability to update any of these deps even for bug fixes until all of framework, helper-1, and helper-2 have updated.

Concretely, if we suppose that framework and helper-2 have released matching versions, but helper-1 has not yet, then cargo update --breaking will break foo-app's build due to a type mismatch between, for example, values of types from infrastructure@newer and function signatures from helper-1 that expect types from infrastructure@older).

The deeper one’s dependency graph, the longer the delay from these cascades is. When one avoids a incompatible version bump that is not technically necessary, one avoids creating these periods of delay for one’s dependents.

So, even given the premise that all transitive dependencies must be kept up to date (which I disagree with), it is desirable to have fewer incompatible versions, and therefore it is particularly likely that package authors will not bother to release an incompatible 1.0 just for “we are stable” documentation purposes.

The problem in your version is that you're missing the part where the crate has a stable API. It's expected that a lot of crates are 0.x.y because a lot of software is unstable. You won't magically make software stable.

Yes. The cases of interest are those where the API is in practice stable, but the maintainers don't want to release an incompatible version and fork the ecosystem of dependents, when it is not technically necessary to do so. The easier it is made for them to release 1.0 without creating such a fork, the more likely they will release a 1.0 when stable.

It might be that those cases are too few for it to be worth adding a special seamless upgrade feature to Cargo, though.

[kornelski]

Majority of the top most used most dependent on crates are at 0.x.

Why isn't libc at 1.0? Why isn't pkg-config 1.0? Why is Unicode-width for ten years at 0.1?

[ia0]

In these terms, I think the place where we disagree is in how many dependencies are public dependencies.

Yes and more precisely the impact of public dependencies. I agree public dependencies have a negative impact on incompatible versions. However, I don't think this should cascade. Authors of crates with public dependencies have the responsibility to reduce the negative impact by specifying dependencies with a range of major versions that are actually compatible as exposed (and used) by that crate. This solution is officially recommended even though it currently has caveats. This will ultimately be properly handled when public_private_dependencies will be a thing.

I'm claiming that that having a higher rate of incompatible versions gives maintainers more problems where they, collectively, have to pursue a sequence of incompatible version bumps

This example (I'm assuming the foo-app arrows are in the wrong direction and that when you say that all crates use types from infrastructure you mean that in the public dependencies sense defined above) is a good example and one that I'm familiar with. I agree the process of migrating will take time, but one nice thing about open-source is that you can help your dependencies migrate. This has 2 positive effects:

• You will be able to migrate faster.
• You can pay back your dependency for your usage of it.

That said, I totally agree that overdoing incompatible versions without purpose is a bad idea. There's some trade-off to be found.

Yes. The cases of interest are those where the API is in practice stable, but the maintainers don't want to release an incompatible version and fork the ecosystem of dependents, when it is not technically necessary to do so. The easier it is made for them to release 1.0 without creating such a fork, the more likely they will release a 1.0 when stable.

It might be that those cases are too few for it to be worth adding a special seamless upgrade feature to Cargo, though.

Yes, I think we agree on those points:

• For some crates (infrastructure crates that are stable with a zero major version), it is not technically necessary to publish an incompatible version from the code point of view, and only necessary from the documentation point of view (essentially guaranteeing stability).
• Those crates could use the semver-trick, but this is manual work and not always easy.
• Dependent crates exporting and/or using types of that crate could use multiple version requirements, but this is manual work (and currently has some bugs).
• Adding one more special semver handling to cargo will only make it more complicated to understand, surprising to use, and harder to debug.
• Cargo has other (and related) priorities that are more important, like public/private dependencies and cargo update --breaking.

With infinite energy, I guess we could have such feature. But with infinite energy, we could also have maintainers use multiple version requirements and semver-trick. It's a trade-off and my personal opinion is that such feature should not be considered for cargo because it has a net negative impact (although it comes from good intentions).

[kornelski]

I'm facing this problem with my RGB crate. It provides a shared type for interoperability, but bumping it to 1.0 is going to ruin the interoperability.

[ia0]

Why isn't libc at 1.0? Why isn't pkg-config 1.0? Why is Unicode-width for ten years at 0.1?

Because of missing education materials? Those crates should take the measure needed to go to 1.0. Those could be the semver-trick, providing a migration guide (including how to use multiple version requirements and which ones such that dependents can copy/paste), or a combination of both.

I'm facing this problem with my RGB crate. It provides a shared type for interoperability, but bumping it to 1.0 is going to ruin the interoperability.

Why don't you use the semver-trick? Why don't you help your dependents migrate and propagate the migration? Should the work be on cargo instead of the ecosystem?

[kornelski]

Because of missing education materials?

Who needs to be educated? The crates I listed were created by some of the old time contributors to Rust and Cargo.

Why should it be in Cargo? Because it can be implemented in one place for the benefit of the entire ecosystem.
It can change a migration process that is laborious, taking long time, and potentially disruptive and fragmenting userbase, to just a small metadata change that fixes it instantly.

[ChrisDenton]

Just an aside but libc is already planning a 1,0 release but it is going to be a breaking release so the proposed Cargo feature won't help.

[ia0]

Who needs to be educated?

• Crate authors afraid of publishing an incompatible version without incompatible change.
• Crates depending on such crates defined above (through a trivial migration guide published by such crates).

Some recent examples of crates matching the problematic type (i.e. largely used with re-exported types) that went stable:

• embedded-hal which provided a migration guide
• generic-array (I couldn't find a migration guide)

Note however that those were not only an incompatible version but also an incompatible change. So this issue wouldn't even have helped.

[ehuss]

The cargo team talked about this today and decided to close/postpone this for now. We acknowledge that this can be a friction for updating past 0.x, but were uncertain about how strong of a reason this particularly point is. There are quite a few complexities involved with this, such as impact on the resolver (which is already quite complex), modifications to the index, how this metadata gets added, etc. Something that could help is gathering survey information about reasons why maintainers stay with 0.x to better understand how common various reasons are.