From 9b38bbc2e788ce9ed1536fabe6b8d73e627cbc90 Mon Sep 17 00:00:00 2001 From: Ethan Brierley Date: Wed, 20 Sep 2023 09:01:36 +0100 Subject: [PATCH 01/32] RFC: Precise Pre-release `cargo update` This RFC proposes extending `cargo update` to allow updates to pre-release versions when requested with `--precise`. For example, a `cargo` user would be able to call `cargo update -p dep --precise 0.1.1-pre0` as long as the version of `dep` requested by their project and its dependencies are semver compatible with `0.1.1`. This effectively splits the notion of compatibility in `cargo`. A pre-release version may be considered compatible when the version is explicitly requested with `--precise`. Cargo will not automatically select that version via a basic `cargo update`. --- text/0000-precise-pre-release-cargo-update.md | 139 ++++++++++++++++++ 1 file changed, 139 insertions(+) create mode 100644 text/0000-precise-pre-release-cargo-update.md diff --git a/text/0000-precise-pre-release-cargo-update.md b/text/0000-precise-pre-release-cargo-update.md new file mode 100644 index 00000000000..78ab9afd940 --- /dev/null +++ b/text/0000-precise-pre-release-cargo-update.md @@ -0,0 +1,139 @@ +- Feature Name: precise-pre-release-cargo-update +- Start Date: 2023-09-20 +- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000) + +# Summary +[summary]: #summary + +This RFC proposes extending `cargo update` to allow updates to pre-release versions when requested with `--precise`. +For example, a `cargo` user would be able to call `cargo update -p dep --precise 0.1.1-pre0` as long as the version of `dep` requested by their project and its dependencies are semver compatible with `0.1.1`. +This effectively splits the notion of compatibility in `cargo`. +A pre-release version may be considered compatible when the version is explicitly requested with `--precise`. +Cargo will not automatically select that version via a basic `cargo update`. + +# Motivation +[motivation]: #motivation + +Pre-release crates are currently challenging to use in large projects with complex dependency trees. +For example, if a maintainer releases `dep = "0.1.1-pre0"`. +They may ask one of their users to try the new API additions in a large project so that the user can give feedback on the release before the maintainer stabilises the new parts of the API. +Unfortunately, since `dep = "0.1.0"` is a transitive dependency of several dependencies of the large project, `cargo` refuses the upgrade, stating that `0.1.1-pre0` is incompatible with `0.1.0`. +The user is left with no upgrade path to the pre-release unless they are able to convince all of their transitive uses of `dep` to release pre-releases of their own. + +# Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + +When Cargo considers `Cargo.toml` requirements for dependencies it always favours selecting stable versions over pre-release versions. +When the specification is itself a pre-release version, Cargo will always select a pre-release. +Cargo is unable to resolve a project with a `Cargo.toml` specification for a pre-release version if any of its dependencies request a stable release. + +If a user does want to select a pre-release version they are able to do so by explicitly requesting Cargo to update to that version. +This is done by passing the `--precise` flag to Cargo. +Cargo will refuse to select pre-release versions that are "incompatible" with the requirement in the projects `Cargo.toml`. +A pre-release version is considered compatible for a precise upgrade if its major, minor, and patch versions are compatible with the requirement, ignoring its pre-release version. +`x.y.z-pre0` is considered compatible with `a.b.c` when requested `--precise`ly if `x.y.z` is semver compatible with `a.b.c` and `a.b.c` `!=` `x.y.z`. + +Consider a `Cargo.toml` with this `[dependencies]` section + +``` +[dependencies] +example = "1.0.0" +``` + +It is possible to update to `1.2.0-pre0` because `1.2.0` is semver compatible with `1.0.0` + +``` +> cargo update -p example --precise 1.2.0-pre0 + Updating crates.io index + Updating example v1.0.0 -> v1.2.0-pre0 +``` + +It is not possible to update to `2.0.0-pre0` because `2.0.0` is not semver compatible with `1.0.0` + +``` +> cargo update -p example --precise 2.0.0-pre0 + Updating crates.io index +error: failed to select a version for the requirement `example = "^1"` +candidate versions found which didn't match: 2.0.0-pre0 +location searched: crates.io index +required by package `tmp-oyyzsf v0.1.0 (/home/ethan/.cache/cargo-temp/tmp-OYyZsF)` +``` + +Once `1.2.0` is released `cargo update` will warn the user that a stable version is available + +``` +> cargo update +warning: version `1.2.0` of `example` has been released but `1.2.0-pre0` was previously selected +note: to use the stable version run `cargo update -p example --precise 1.2.0` +``` + +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +Consider this table where `a.b.c` is compatible with `x.y.z` and `x.y.z > a.b.c` + +| Cargo.toml spec | Cargo.lock version | Target version | Selected by cargo update | Selected by cargo update --precise | +| --------------- | ------------------ | -------------- | ------------------------- | ----------------------------------- | +| `a.b.c` | `a.b.c` | `x.y.z` | ✅ | ✅ | +| `a.b.c` | `a.b.c` | `x.y.z-pre0` | ❌ | ✅ | +| `a.b.c` | `x.y.z-pre0` | `x.y.z-pre1` | ❌ | ✅ | +| `a.b.c-pre0` | `a.b.c-pre0` | `a.b.c-pre1` | ✅¹ | ✅ | +| `a.b.c` | `a.b.c` | `a.b.c-pre0` | ❌ | ❌ | +| `a.b.c-pre0` | `a.b.c-pre0` | `x.y.z` | ❌² | ✅ | + +✅: Will upgrade + +❌: Will not upgrade + +¹For backwards compatibility with Cargo's current behaviour (see [RFC: Precise Pre-release Deps](https://github.com/rust-lang/rfcs/pull/3263)) + +²Emits a warning + +# Drawbacks +[drawbacks]: #drawbacks + +- Pre-release versions are not easily auditable when they are only specified in the lock file. + A change that makes use of a pre-release version may not be noticed during code review as reviewers don't always check for changes in the lock file. +- Library crates that require a pre-release version are not well supported since their lock files are ignored by their users (see [future-possibilities]) + +# Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +The main alternative to this would be to accept that pre-release versions are not very usable and discourage their use. +Cargo overrides can be used instead using `[patch]`. +These provide a similar experience to pre-releases, however, they require that the library's code is somehow vendored outside of the registry, usually with git. +This can cause issues particularly in CI where jobs may have permission to fetch from a private registry but not from private git repositories. +Resolving issues around not being able to fetch pre-releases from the registry usually wastes a significant amount of time. + +Another alternative would be to resolve pre-release versions in `Cargo.toml`s even when another dependency specifies a stable version. +This is explored in [future-possibilities]. +This would require significant changes to the resolver since the latest compatible version would depend on the versions required by other parts of the dependency tree. +This RFC may be a stepping stone in that direction since it lays the groundwork for pre-release compatibility rules, however, I consider detailing such a change outside of the scope of this RFC. + +# Prior art +[prior-art]: #prior-art + +[RFC: Precise Pre-release Deps](https://github.com/rust-lang/rfcs/pull/3263) aims to solve a similar but different issue where `cargo update` opts to upgrade +pre-release versions to new pre-releases when one is released. + +# Unresolved questions +[unresolved-questions]: #unresolved-questions + +# Future possibilities +[future-possibilities]: #future-possibilities + +It would be nice if dependencies could specify their requirements for pre-release versions. + +Take for example this dependency tree. + +``` +example +├── a ^0.1.0 +│ └── b =0.1.1-pre0 +└── b ^0.1.0 +``` + +Since crates ignore the lock files of their dependencies there is no way for `a` to communicate with `example` that it requires features from `b = 0.1.1-pre0` without breaking `example`'s direct dependency on `b`. +To enable this we could use the same concept of compatible pre-releases in `Cargo.toml`, not just `Cargo.lock`. +This would require that pre-releases are specified with `=` and would allow pre-release versions to be requested anywhere within the dependency tree without causing the resolver to throw an error. + From e52e806d9a831086c9f7fd9db93fb5898e5ee8c6 Mon Sep 17 00:00:00 2001 From: Ethan Brierley Date: Wed, 20 Sep 2023 09:11:12 +0100 Subject: [PATCH 02/32] Update links to PR --- ...cargo-update.md => 3493-precise-pre-release-cargo-update.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename text/{0000-precise-pre-release-cargo-update.md => 3493-precise-pre-release-cargo-update.md} (99%) diff --git a/text/0000-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md similarity index 99% rename from text/0000-precise-pre-release-cargo-update.md rename to text/3493-precise-pre-release-cargo-update.md index 78ab9afd940..7ee811a97bc 100644 --- a/text/0000-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -1,6 +1,6 @@ - Feature Name: precise-pre-release-cargo-update - Start Date: 2023-09-20 -- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000) +- RFC PR: [rust-lang/rfcs#3493](https://github.com/rust-lang/rfcs/pull/3493) # Summary [summary]: #summary From fbc31fad7cfd7cedd88b9d77f48c1570e8b4b343 Mon Sep 17 00:00:00 2001 From: Ethan Brierley Date: Wed, 20 Sep 2023 11:29:52 +0100 Subject: [PATCH 03/32] `-pre0` -> `pre.0` --- text/3493-precise-pre-release-cargo-update.md | 36 +++++++++---------- 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index 7ee811a97bc..74c5be7526d 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -6,7 +6,7 @@ [summary]: #summary This RFC proposes extending `cargo update` to allow updates to pre-release versions when requested with `--precise`. -For example, a `cargo` user would be able to call `cargo update -p dep --precise 0.1.1-pre0` as long as the version of `dep` requested by their project and its dependencies are semver compatible with `0.1.1`. +For example, a `cargo` user would be able to call `cargo update -p dep --precise 0.1.1-pre.0` as long as the version of `dep` requested by their project and its dependencies are semver compatible with `0.1.1`. This effectively splits the notion of compatibility in `cargo`. A pre-release version may be considered compatible when the version is explicitly requested with `--precise`. Cargo will not automatically select that version via a basic `cargo update`. @@ -15,9 +15,9 @@ Cargo will not automatically select that version via a basic `cargo update`. [motivation]: #motivation Pre-release crates are currently challenging to use in large projects with complex dependency trees. -For example, if a maintainer releases `dep = "0.1.1-pre0"`. +For example, if a maintainer releases `dep = "0.1.1-pre.0"`. They may ask one of their users to try the new API additions in a large project so that the user can give feedback on the release before the maintainer stabilises the new parts of the API. -Unfortunately, since `dep = "0.1.0"` is a transitive dependency of several dependencies of the large project, `cargo` refuses the upgrade, stating that `0.1.1-pre0` is incompatible with `0.1.0`. +Unfortunately, since `dep = "0.1.0"` is a transitive dependency of several dependencies of the large project, `cargo` refuses the upgrade, stating that `0.1.1-pre.0` is incompatible with `0.1.0`. The user is left with no upgrade path to the pre-release unless they are able to convince all of their transitive uses of `dep` to release pre-releases of their own. # Guide-level explanation @@ -31,7 +31,7 @@ If a user does want to select a pre-release version they are able to do so by ex This is done by passing the `--precise` flag to Cargo. Cargo will refuse to select pre-release versions that are "incompatible" with the requirement in the projects `Cargo.toml`. A pre-release version is considered compatible for a precise upgrade if its major, minor, and patch versions are compatible with the requirement, ignoring its pre-release version. -`x.y.z-pre0` is considered compatible with `a.b.c` when requested `--precise`ly if `x.y.z` is semver compatible with `a.b.c` and `a.b.c` `!=` `x.y.z`. +`x.y.z-pre.0` is considered compatible with `a.b.c` when requested `--precise`ly if `x.y.z` is semver compatible with `a.b.c` and `a.b.c` `!=` `x.y.z`. Consider a `Cargo.toml` with this `[dependencies]` section @@ -40,21 +40,21 @@ Consider a `Cargo.toml` with this `[dependencies]` section example = "1.0.0" ``` -It is possible to update to `1.2.0-pre0` because `1.2.0` is semver compatible with `1.0.0` +It is possible to update to `1.2.0-pre.0` because `1.2.0` is semver compatible with `1.0.0` ``` -> cargo update -p example --precise 1.2.0-pre0 +> cargo update -p example --precise 1.2.0-pre.0 Updating crates.io index - Updating example v1.0.0 -> v1.2.0-pre0 + Updating example v1.0.0 -> v1.2.0-pre.0 ``` -It is not possible to update to `2.0.0-pre0` because `2.0.0` is not semver compatible with `1.0.0` +It is not possible to update to `2.0.0-pre.0` because `2.0.0` is not semver compatible with `1.0.0` ``` -> cargo update -p example --precise 2.0.0-pre0 +> cargo update -p example --precise 2.0.0-pre.0 Updating crates.io index error: failed to select a version for the requirement `example = "^1"` -candidate versions found which didn't match: 2.0.0-pre0 +candidate versions found which didn't match: 2.0.0-pre.0 location searched: crates.io index required by package `tmp-oyyzsf v0.1.0 (/home/ethan/.cache/cargo-temp/tmp-OYyZsF)` ``` @@ -63,7 +63,7 @@ Once `1.2.0` is released `cargo update` will warn the user that a stable version ``` > cargo update -warning: version `1.2.0` of `example` has been released but `1.2.0-pre0` was previously selected +warning: version `1.2.0` of `example` has been released but `1.2.0-pre.0` was previously selected note: to use the stable version run `cargo update -p example --precise 1.2.0` ``` @@ -75,11 +75,11 @@ Consider this table where `a.b.c` is compatible with `x.y.z` and `x.y.z > a.b.c` | Cargo.toml spec | Cargo.lock version | Target version | Selected by cargo update | Selected by cargo update --precise | | --------------- | ------------------ | -------------- | ------------------------- | ----------------------------------- | | `a.b.c` | `a.b.c` | `x.y.z` | ✅ | ✅ | -| `a.b.c` | `a.b.c` | `x.y.z-pre0` | ❌ | ✅ | -| `a.b.c` | `x.y.z-pre0` | `x.y.z-pre1` | ❌ | ✅ | -| `a.b.c-pre0` | `a.b.c-pre0` | `a.b.c-pre1` | ✅¹ | ✅ | -| `a.b.c` | `a.b.c` | `a.b.c-pre0` | ❌ | ❌ | -| `a.b.c-pre0` | `a.b.c-pre0` | `x.y.z` | ❌² | ✅ | +| `a.b.c` | `a.b.c` | `x.y.z-pre.0` | ❌ | ✅ | +| `a.b.c` | `x.y.z-pre.0` | `x.y.z-pre.1` | ❌ | ✅ | +| `a.b.c-pre.0` | `a.b.c-pre.0` | `a.b.c-pre.1` | ✅¹ | ✅ | +| `a.b.c` | `a.b.c` | `a.b.c-pre.0` | ❌ | ❌ | +| `a.b.c-pre.0` | `a.b.c-pre.0` | `x.y.z` | ❌² | ✅ | ✅: Will upgrade @@ -129,11 +129,11 @@ Take for example this dependency tree. ``` example ├── a ^0.1.0 -│ └── b =0.1.1-pre0 +│ └── b =0.1.1-pre.0 └── b ^0.1.0 ``` -Since crates ignore the lock files of their dependencies there is no way for `a` to communicate with `example` that it requires features from `b = 0.1.1-pre0` without breaking `example`'s direct dependency on `b`. +Since crates ignore the lock files of their dependencies there is no way for `a` to communicate with `example` that it requires features from `b = 0.1.1-pre.0` without breaking `example`'s direct dependency on `b`. To enable this we could use the same concept of compatible pre-releases in `Cargo.toml`, not just `Cargo.lock`. This would require that pre-releases are specified with `=` and would allow pre-release versions to be requested anywhere within the dependency tree without causing the resolver to throw an error. From a3696be465ef07e5a71326ef91f362dc84da0c7c Mon Sep 17 00:00:00 2001 From: Ethan Brierley Date: Wed, 20 Sep 2023 11:32:50 +0100 Subject: [PATCH 04/32] Reflect learning that cargo will currently upgrade from pre to stable --- text/3493-precise-pre-release-cargo-update.md | 12 +----------- 1 file changed, 1 insertion(+), 11 deletions(-) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index 74c5be7526d..469c30e572d 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -59,14 +59,6 @@ location searched: crates.io index required by package `tmp-oyyzsf v0.1.0 (/home/ethan/.cache/cargo-temp/tmp-OYyZsF)` ``` -Once `1.2.0` is released `cargo update` will warn the user that a stable version is available - -``` -> cargo update -warning: version `1.2.0` of `example` has been released but `1.2.0-pre.0` was previously selected -note: to use the stable version run `cargo update -p example --precise 1.2.0` -``` - # Reference-level explanation [reference-level-explanation]: #reference-level-explanation @@ -78,8 +70,8 @@ Consider this table where `a.b.c` is compatible with `x.y.z` and `x.y.z > a.b.c` | `a.b.c` | `a.b.c` | `x.y.z-pre.0` | ❌ | ✅ | | `a.b.c` | `x.y.z-pre.0` | `x.y.z-pre.1` | ❌ | ✅ | | `a.b.c-pre.0` | `a.b.c-pre.0` | `a.b.c-pre.1` | ✅¹ | ✅ | +| `a.b.c-pre.0` | `a.b.c-pre.0` | `x.y.z` | ✅¹ | ✅ | | `a.b.c` | `a.b.c` | `a.b.c-pre.0` | ❌ | ❌ | -| `a.b.c-pre.0` | `a.b.c-pre.0` | `x.y.z` | ❌² | ✅ | ✅: Will upgrade @@ -87,8 +79,6 @@ Consider this table where `a.b.c` is compatible with `x.y.z` and `x.y.z > a.b.c` ¹For backwards compatibility with Cargo's current behaviour (see [RFC: Precise Pre-release Deps](https://github.com/rust-lang/rfcs/pull/3263)) -²Emits a warning - # Drawbacks [drawbacks]: #drawbacks From 8abe7c0caf84b42bc2c409e9a5738e8057add4e6 Mon Sep 17 00:00:00 2001 From: Ethan Brierley Date: Wed, 20 Sep 2023 15:39:16 +0100 Subject: [PATCH 05/32] Clarify undesirable behaviour --- text/3493-precise-pre-release-cargo-update.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index 469c30e572d..dfb4173e54b 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -77,7 +77,9 @@ Consider this table where `a.b.c` is compatible with `x.y.z` and `x.y.z > a.b.c` ❌: Will not upgrade -¹For backwards compatibility with Cargo's current behaviour (see [RFC: Precise Pre-release Deps](https://github.com/rust-lang/rfcs/pull/3263)) +¹This behaviour is considered by some to be undesirable and may change as proposed in [RFC: Precise Pre-release Deps](https://github.com/rust-lang/rfcs/pull/3263). +This RFC preserves this behaviour to remain backwards compatible. +Since this RFC is concerned with the behaviour of `cargo update --precise` changes to bare `cargo update` made in future RFCs should have no impact on this proposal. # Drawbacks [drawbacks]: #drawbacks From d34db12d51a3959d14ca01cdafd873852b03172e Mon Sep 17 00:00:00 2001 From: Ethan Brierley Date: Wed, 20 Sep 2023 16:14:21 +0100 Subject: [PATCH 06/32] Add more about versions that aren't caret --- text/3493-precise-pre-release-cargo-update.md | 27 ++++++++++++++----- 1 file changed, 21 insertions(+), 6 deletions(-) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index dfb4173e54b..4f22ab580db 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -66,12 +66,12 @@ Consider this table where `a.b.c` is compatible with `x.y.z` and `x.y.z > a.b.c` | Cargo.toml spec | Cargo.lock version | Target version | Selected by cargo update | Selected by cargo update --precise | | --------------- | ------------------ | -------------- | ------------------------- | ----------------------------------- | -| `a.b.c` | `a.b.c` | `x.y.z` | ✅ | ✅ | -| `a.b.c` | `a.b.c` | `x.y.z-pre.0` | ❌ | ✅ | -| `a.b.c` | `x.y.z-pre.0` | `x.y.z-pre.1` | ❌ | ✅ | -| `a.b.c-pre.0` | `a.b.c-pre.0` | `a.b.c-pre.1` | ✅¹ | ✅ | -| `a.b.c-pre.0` | `a.b.c-pre.0` | `x.y.z` | ✅¹ | ✅ | -| `a.b.c` | `a.b.c` | `a.b.c-pre.0` | ❌ | ❌ | +| `^a.b.c` | `a.b.c` | `x.y.z` | ✅ | ✅ | +| `^a.b.c` | `a.b.c` | `x.y.z-pre.0` | ❌ | ✅ | +| `^a.b.c` | `x.y.z-pre.0` | `x.y.z-pre.1` | ❌ | ✅ | +| `^a.b.c-pre.0` | `a.b.c-pre.0` | `a.b.c-pre.1` | ✅¹ | ✅ | +| `^a.b.c-pre.0` | `a.b.c-pre.0` | `x.y.z` | ✅¹ | ✅ | +| `^a.b.c` | `a.b.c` | `a.b.c-pre.0` | ❌ | ❌ | ✅: Will upgrade @@ -81,6 +81,21 @@ Consider this table where `a.b.c` is compatible with `x.y.z` and `x.y.z > a.b.c` This RFC preserves this behaviour to remain backwards compatible. Since this RFC is concerned with the behaviour of `cargo update --precise` changes to bare `cargo update` made in future RFCs should have no impact on this proposal. +To determine if a version can be selected with `--precise` for a specification that isn't listed above cosider where pre-releases exist within version ranges. + +For example consider the version `~1.2.3`. +The range for `~1.2.3` is [stated in the cargo book](https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#tilde-requirements). + +``` +~1.2.3 := >=1.2.3, <1.3.0 +``` + +Intuitively `1.2.4-pre.0` satisfies this inequality, therefore it can be selected with `cargo update --precise`. +Since it is a pre-release and the specification is not, `1.2.4-pre.0` would not be selected by a bare `cargo update`. +`1.3.0-pre.0` also satisfies the inequality but `1.2.3-pre.0` and `1.3.1-pre.0` do not. + +Put in simple terms the relationship between a pre-release and its stable release is always `a.b.c-pre.0 < a.b.c`. + # Drawbacks [drawbacks]: #drawbacks From c1fe92034c567f42e3e1293bb110ec618774a9d5 Mon Sep 17 00:00:00 2001 From: Ethan Brierley Date: Wed, 20 Sep 2023 16:43:32 +0100 Subject: [PATCH 07/32] Alt: extend `[patch]` --- text/3493-precise-pre-release-cargo-update.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index 4f22ab580db..abc3bc6a9b9 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -106,12 +106,26 @@ Put in simple terms the relationship between a pre-release and its stable releas # Rationale and alternatives [rationale-and-alternatives]: #rationale-and-alternatives +## Discourage pre-releases + The main alternative to this would be to accept that pre-release versions are not very usable and discourage their use. Cargo overrides can be used instead using `[patch]`. These provide a similar experience to pre-releases, however, they require that the library's code is somehow vendored outside of the registry, usually with git. This can cause issues particularly in CI where jobs may have permission to fetch from a private registry but not from private git repositories. Resolving issues around not being able to fetch pre-releases from the registry usually wastes a significant amount of time. +## Extend `[patch]` + +It could be possible to build upon `[patch]` to [allow it to use crates published in the registry](https://github.com/rust-lang/cargo/issues/9227). +This could be combined with [version overrides](https://github.com/rust-lang/cargo/issues/5640) to pretend that the pre-release crate is a stable version. + +My concern with this approach is that it doesn't introduce the concept of compatible pre-releases. +This would allow any version to masquerade as another. +Without the concept of compatible pre-releases there would be no path forward towards being able to express pre-release requirements in library crates. +This is explored in [future-possibilities]. + +## Pre-releases in `Cargo.toml` + Another alternative would be to resolve pre-release versions in `Cargo.toml`s even when another dependency specifies a stable version. This is explored in [future-possibilities]. This would require significant changes to the resolver since the latest compatible version would depend on the versions required by other parts of the dependency tree. From 8ce1563a98a14ddb56594812c0eb7b78d82799b3 Mon Sep 17 00:00:00 2001 From: Ethan Brierley Date: Wed, 20 Sep 2023 16:55:43 +0100 Subject: [PATCH 08/32] Alt: cargo update incompat --- text/3493-precise-pre-release-cargo-update.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index abc3bc6a9b9..8df2b8408f3 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -124,6 +124,14 @@ This would allow any version to masquerade as another. Without the concept of compatible pre-releases there would be no path forward towards being able to express pre-release requirements in library crates. This is explored in [future-possibilities]. +## Change the version in `Cargo.toml` rather than `Cargo.lock` when using `--precise` + +This [accepted proposal](https://github.com/rust-lang/cargo/issues/12425) allows cargo to update a projects `Cargo.toml` when the version is incompatible. + +The issue here is that cargo will not unify a pre-release version with a stable version. +If the crate being updated is used pervasively this will more than likely cause a resolver error. +This makes this alternative unfit for our [motivation]. + ## Pre-releases in `Cargo.toml` Another alternative would be to resolve pre-release versions in `Cargo.toml`s even when another dependency specifies a stable version. From fcfaa2aefe1af2aed00a51382f00357da2847fb8 Mon Sep 17 00:00:00 2001 From: Ethan Brierley Date: Wed, 20 Sep 2023 17:01:18 +0100 Subject: [PATCH 09/32] RFC <-> cargo update incompat --- text/3493-precise-pre-release-cargo-update.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index 8df2b8408f3..11593d3ed16 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -132,6 +132,9 @@ The issue here is that cargo will not unify a pre-release version with a stable If the crate being updated is used pervasively this will more than likely cause a resolver error. This makes this alternative unfit for our [motivation]. +The [accepted proposal](https://github.com/rust-lang/cargo/issues/12425) is affected by this RFC, +insofar as it will not update the `Cargo.toml` in cases when the pre-release can be considered compatible for upgrade in `Cargo.lock`. + ## Pre-releases in `Cargo.toml` Another alternative would be to resolve pre-release versions in `Cargo.toml`s even when another dependency specifies a stable version. From b2450dc3684792b58ec3a10a78cdfaa9d9305ba6 Mon Sep 17 00:00:00 2001 From: Ethan Brierley Date: Wed, 20 Sep 2023 17:02:51 +0100 Subject: [PATCH 10/32] Wording --- text/3493-precise-pre-release-cargo-update.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index 11593d3ed16..7a37d4108da 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -106,9 +106,8 @@ Put in simple terms the relationship between a pre-release and its stable releas # Rationale and alternatives [rationale-and-alternatives]: #rationale-and-alternatives -## Discourage pre-releases +## Use overrides -The main alternative to this would be to accept that pre-release versions are not very usable and discourage their use. Cargo overrides can be used instead using `[patch]`. These provide a similar experience to pre-releases, however, they require that the library's code is somehow vendored outside of the registry, usually with git. This can cause issues particularly in CI where jobs may have permission to fetch from a private registry but not from private git repositories. From b10592ddb53b9eaa41d66cf86cef052124a17b78 Mon Sep 17 00:00:00 2001 From: Ethan Brierley Date: Wed, 20 Sep 2023 17:04:40 +0100 Subject: [PATCH 11/32] Unify --- text/3493-precise-pre-release-cargo-update.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index 7a37d4108da..330f5a36366 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -153,7 +153,7 @@ pre-release versions to new pre-releases when one is released. # Future possibilities [future-possibilities]: #future-possibilities -It would be nice if dependencies could specify their requirements for pre-release versions. +It would be nice if cargo could unify pre-release version requirements with stable versions. Take for example this dependency tree. From d69bf5b1fbe240510fe9ccee39559536cb916712 Mon Sep 17 00:00:00 2001 From: Ed Page Date: Tue, 12 Dec 2023 12:20:20 -0600 Subject: [PATCH 12/32] Add yanked as prior art --- text/3493-precise-pre-release-cargo-update.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index 330f5a36366..c66763f2a90 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -11,6 +11,10 @@ This effectively splits the notion of compatibility in `cargo`. A pre-release version may be considered compatible when the version is explicitly requested with `--precise`. Cargo will not automatically select that version via a basic `cargo update`. +One way to think of this is that we are changing from the the version +requirements syntax requiring opt-in to match pre-release of higher versions to +the resolver ignoring pre-releases like yanked packages, with an override flag. + # Motivation [motivation]: #motivation @@ -147,6 +151,13 @@ This RFC may be a stepping stone in that direction since it lays the groundwork [RFC: Precise Pre-release Deps](https://github.com/rust-lang/rfcs/pull/3263) aims to solve a similar but different issue where `cargo update` opts to upgrade pre-release versions to new pre-releases when one is released. +Implementation-wise, this is very similar to how yanked packages work. +- Not selected under normal conditions +- Once its in the lockfile, that gets respected and stays in the lockfile + +The only difference being that `--precise` does not allow overriding the "ignore yank" behavior +(though [it is desired by some](https://github.com/rust-lang/cargo/issues/4225)). + # Unresolved questions [unresolved-questions]: #unresolved-questions From ffc34744a3c4333b2eaa7c06a3e2224396ad66ce Mon Sep 17 00:00:00 2001 From: Ed Page Date: Tue, 12 Dec 2023 12:20:31 -0600 Subject: [PATCH 13/32] Add cargo-upgrade as prior art --- text/3493-precise-pre-release-cargo-update.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index c66763f2a90..cdc71103fc3 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -158,6 +158,11 @@ Implementation-wise, this is very similar to how yanked packages work. The only difference being that `--precise` does not allow overriding the "ignore yank" behavior (though [it is desired by some](https://github.com/rust-lang/cargo/issues/4225)). +For `--precise` forcing a version through, we have precedence in +[an approved-but-not-implemented proposal](https://github.com/rust-lang/cargo/issues/12425) +for `cargo update --precise` for incompatible versions to force its way +through by modifying `Cargo.toml`. + # Unresolved questions [unresolved-questions]: #unresolved-questions From eddc6bc2256b87accb16a062f2422da574adb50a Mon Sep 17 00:00:00 2001 From: Ed Page Date: Tue, 12 Dec 2023 12:21:53 -0600 Subject: [PATCH 14/32] Add title to future possibility --- text/3493-precise-pre-release-cargo-update.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index cdc71103fc3..865ff136586 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -169,6 +169,8 @@ through by modifying `Cargo.toml`. # Future possibilities [future-possibilities]: #future-possibilities +## Pre-release dep "allows" pre-release everywhere + It would be nice if cargo could unify pre-release version requirements with stable versions. Take for example this dependency tree. From 0f8bf4180284e4d2fa4d32443c4f08b983e6b045 Mon Sep 17 00:00:00 2001 From: Ed Page Date: Tue, 12 Dec 2023 12:25:51 -0600 Subject: [PATCH 15/32] Potential for --allow-prerelease --- text/3493-precise-pre-release-cargo-update.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index 865ff136586..2d84799dc7d 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -186,3 +186,8 @@ Since crates ignore the lock files of their dependencies there is no way for `a` To enable this we could use the same concept of compatible pre-releases in `Cargo.toml`, not just `Cargo.lock`. This would require that pre-releases are specified with `=` and would allow pre-release versions to be requested anywhere within the dependency tree without causing the resolver to throw an error. +## `--allow-prelease` + +Instead of manually selecting a version with `--precise`, we could support `cargo update --package foo --allow-prelease`. + +If we made this flag work without `--package`, we could the extend it also to `cargo generate-lockfile`. From c48e1bb7d8535d4fae0f2b20899626d969353f8f Mon Sep 17 00:00:00 2001 From: Ed Page Date: Tue, 12 Dec 2023 12:35:58 -0600 Subject: [PATCH 16/32] Provide more context in the motivation --- text/3493-precise-pre-release-cargo-update.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index 2d84799dc7d..d797817cd78 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -18,7 +18,14 @@ the resolver ignoring pre-releases like yanked packages, with an override flag. # Motivation [motivation]: #motivation -Pre-release crates are currently challenging to use in large projects with complex dependency trees. +Today, version requirements ignore pre-release versions, +so `1.0.0` cannot be used with `1.1.0-alpha.1`. +Specifying a pre-release in a version requirement has two affects +- Specifies the minimum compatible pre-release. +- Opts-in to matching version requirements (within a version) + +However, coupling these concerns makes it difficult to try out pre-releases +because every dependency in the tree has to opt-in. For example, if a maintainer releases `dep = "0.1.1-pre.0"`. They may ask one of their users to try the new API additions in a large project so that the user can give feedback on the release before the maintainer stabilises the new parts of the API. Unfortunately, since `dep = "0.1.0"` is a transitive dependency of several dependencies of the large project, `cargo` refuses the upgrade, stating that `0.1.1-pre.0` is incompatible with `0.1.0`. From 78f0ccb9acf8a3fb3aeca20a558b8893eeae23cd Mon Sep 17 00:00:00 2001 From: Ed Page Date: Tue, 12 Dec 2023 12:37:36 -0600 Subject: [PATCH 17/32] Clean up structure of motivating example --- text/3493-precise-pre-release-cargo-update.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index d797817cd78..01626f91682 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -26,8 +26,9 @@ Specifying a pre-release in a version requirement has two affects However, coupling these concerns makes it difficult to try out pre-releases because every dependency in the tree has to opt-in. -For example, if a maintainer releases `dep = "0.1.1-pre.0"`. -They may ask one of their users to try the new API additions in a large project so that the user can give feedback on the release before the maintainer stabilises the new parts of the API. +For example, a maintainer asks one of their users to try new API additions in +`dep = "0.1.1-pre.0"` in a large project so that the user can give feedback on +the release before the maintainer stabilises the new parts of the API. Unfortunately, since `dep = "0.1.0"` is a transitive dependency of several dependencies of the large project, `cargo` refuses the upgrade, stating that `0.1.1-pre.0` is incompatible with `0.1.0`. The user is left with no upgrade path to the pre-release unless they are able to convince all of their transitive uses of `dep` to release pre-releases of their own. From cf16352cccc6e5988d5f6b25adb7951a3753a109 Mon Sep 17 00:00:00 2001 From: Ed Page Date: Tue, 12 Dec 2023 12:39:40 -0600 Subject: [PATCH 18/32] Add the risk / cost drawback --- text/3493-precise-pre-release-cargo-update.md | 1 + 1 file changed, 1 insertion(+) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index 01626f91682..8956f1015aa 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -114,6 +114,7 @@ Put in simple terms the relationship between a pre-release and its stable releas - Pre-release versions are not easily auditable when they are only specified in the lock file. A change that makes use of a pre-release version may not be noticed during code review as reviewers don't always check for changes in the lock file. - Library crates that require a pre-release version are not well supported since their lock files are ignored by their users (see [future-possibilities]) +- This is an invasive change to cargo with a significant risk for bugs. This also extends out to packages in the ecosystem that deal with depednency versions. # Rationale and alternatives [rationale-and-alternatives]: #rationale-and-alternatives From e29bef6a7b1b69e6fc78cc515749e8b6a600e419 Mon Sep 17 00:00:00 2001 From: Ed Page Date: Tue, 12 Dec 2023 13:11:40 -0600 Subject: [PATCH 19/32] Add Alt for implied updates --- text/3493-precise-pre-release-cargo-update.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index 8956f1015aa..cdd391a6327 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -119,6 +119,17 @@ Put in simple terms the relationship between a pre-release and its stable releas # Rationale and alternatives [rationale-and-alternatives]: #rationale-and-alternatives +## One-time opt-in + +With this proposal, pre-release is like yanked: +having `foo@1.2.3-alpha.0` in your `Cargo.lock` does not implicitly mean you can use `foo@1.2.3-alpha.1`. +To update within pre-releases, you'll have to use `--precise` again. + +Instead, the lockfile could identify that `foo@1.2.3-alpha.0` is a pre-release allow updating to any `1.2.3` pre-release. +`cargo update` focuses on compatible updates and pre-releases aren't necesarrily compatible with each other +(see also [RFC: Precise Pre-release Deps](https://github.com/rust-lang/rfcs/pull/3263)). +Alternatively, in [future-possibilities] is `cargo update -p foo --allow-prerelease` which would be an explicit way to update. + ## Use overrides Cargo overrides can be used instead using `[patch]`. From 36ffc26a2a0e925d9afb58362ad6f06dd49ef589 Mon Sep 17 00:00:00 2001 From: Ed Page Date: Tue, 12 Dec 2023 13:14:47 -0600 Subject: [PATCH 20/32] Fix an over-generalization --- text/3493-precise-pre-release-cargo-update.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index cdd391a6327..14daf63a04b 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -18,7 +18,7 @@ the resolver ignoring pre-releases like yanked packages, with an override flag. # Motivation [motivation]: #motivation -Today, version requirements ignore pre-release versions, +Today, version requirements ignore pre-release versions by default, so `1.0.0` cannot be used with `1.1.0-alpha.1`. Specifying a pre-release in a version requirement has two affects - Specifies the minimum compatible pre-release. From 24062430563255c8c07a3e7fb8279aa6e782482b Mon Sep 17 00:00:00 2001 From: Ed Page Date: Tue, 12 Dec 2023 13:18:37 -0600 Subject: [PATCH 21/32] Clean up table formatting --- text/3493-precise-pre-release-cargo-update.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index 14daf63a04b..cabc5df961b 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -78,12 +78,12 @@ Consider this table where `a.b.c` is compatible with `x.y.z` and `x.y.z > a.b.c` | Cargo.toml spec | Cargo.lock version | Target version | Selected by cargo update | Selected by cargo update --precise | | --------------- | ------------------ | -------------- | ------------------------- | ----------------------------------- | -| `^a.b.c` | `a.b.c` | `x.y.z` | ✅ | ✅ | -| `^a.b.c` | `a.b.c` | `x.y.z-pre.0` | ❌ | ✅ | -| `^a.b.c` | `x.y.z-pre.0` | `x.y.z-pre.1` | ❌ | ✅ | -| `^a.b.c-pre.0` | `a.b.c-pre.0` | `a.b.c-pre.1` | ✅¹ | ✅ | -| `^a.b.c-pre.0` | `a.b.c-pre.0` | `x.y.z` | ✅¹ | ✅ | -| `^a.b.c` | `a.b.c` | `a.b.c-pre.0` | ❌ | ❌ | +| `^a.b.c` | `a.b.c` | `x.y.z` | ✅ | ✅ | +| `^a.b.c` | `a.b.c` | `x.y.z-pre.0` | ❌ | ✅ | +| `^a.b.c` | `x.y.z-pre.0` | `x.y.z-pre.1` | ❌ | ✅ | +| `^a.b.c-pre.0` | `a.b.c-pre.0` | `a.b.c-pre.1` | ✅¹ | ✅ | +| `^a.b.c-pre.0` | `a.b.c-pre.0` | `x.y.z` | ✅¹ | ✅ | +| `^a.b.c` | `a.b.c` | `a.b.c-pre.0` | ❌ | ❌ | ✅: Will upgrade From a67ae6dea7f5e0f38552e238caeefd55093c9cd4 Mon Sep 17 00:00:00 2001 From: Ed Page Date: Tue, 12 Dec 2023 13:19:35 -0600 Subject: [PATCH 22/32] Move table to Guide --- text/3493-precise-pre-release-cargo-update.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index cabc5df961b..b022922d4ae 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -71,9 +71,6 @@ location searched: crates.io index required by package `tmp-oyyzsf v0.1.0 (/home/ethan/.cache/cargo-temp/tmp-OYyZsF)` ``` -# Reference-level explanation -[reference-level-explanation]: #reference-level-explanation - Consider this table where `a.b.c` is compatible with `x.y.z` and `x.y.z > a.b.c` | Cargo.toml spec | Cargo.lock version | Target version | Selected by cargo update | Selected by cargo update --precise | @@ -93,6 +90,9 @@ Consider this table where `a.b.c` is compatible with `x.y.z` and `x.y.z > a.b.c` This RFC preserves this behaviour to remain backwards compatible. Since this RFC is concerned with the behaviour of `cargo update --precise` changes to bare `cargo update` made in future RFCs should have no impact on this proposal. +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + To determine if a version can be selected with `--precise` for a specification that isn't listed above cosider where pre-releases exist within version ranges. For example consider the version `~1.2.3`. From a66dcafa3b0d66fefeef2776e5572c6c3cc4efa3 Mon Sep 17 00:00:00 2001 From: Ed Page Date: Tue, 12 Dec 2023 13:22:06 -0600 Subject: [PATCH 23/32] Tie table into example --- text/3493-precise-pre-release-cargo-update.md | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index b022922d4ae..d7456713d08 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -71,16 +71,14 @@ location searched: crates.io index required by package `tmp-oyyzsf v0.1.0 (/home/ethan/.cache/cargo-temp/tmp-OYyZsF)` ``` -Consider this table where `a.b.c` is compatible with `x.y.z` and `x.y.z > a.b.c` - | Cargo.toml spec | Cargo.lock version | Target version | Selected by cargo update | Selected by cargo update --precise | | --------------- | ------------------ | -------------- | ------------------------- | ----------------------------------- | -| `^a.b.c` | `a.b.c` | `x.y.z` | ✅ | ✅ | -| `^a.b.c` | `a.b.c` | `x.y.z-pre.0` | ❌ | ✅ | -| `^a.b.c` | `x.y.z-pre.0` | `x.y.z-pre.1` | ❌ | ✅ | -| `^a.b.c-pre.0` | `a.b.c-pre.0` | `a.b.c-pre.1` | ✅¹ | ✅ | -| `^a.b.c-pre.0` | `a.b.c-pre.0` | `x.y.z` | ✅¹ | ✅ | -| `^a.b.c` | `a.b.c` | `a.b.c-pre.0` | ❌ | ❌ | +| `^1.0.0` | `1.0.0` | `1.2.0` | ✅ | ✅ | +| `^1.0.0` | `1.0.0` | `1.2.0-pre.0` | ❌ | ✅ | +| `^1.0.0` | `1.2.0-pre.0` | `1.2.0-pre.1` | ❌ | ✅ | +| `^1.0.0-pre.0` | `1.0.0-pre.0` | `1.0.0-pre.1` | ✅¹ | ✅ | +| `^1.0.0-pre.0` | `1.0.0-pre.0` | `1.2.0` | ✅¹ | ✅ | +| `^1.0.0` | `1.0.0` | `1.0.0-pre.0` | ❌ | ❌ | ✅: Will upgrade From d3ebf36e6444c9affe83c97b7a035bf65987c8d8 Mon Sep 17 00:00:00 2001 From: Ed Page Date: Tue, 12 Dec 2023 13:27:39 -0600 Subject: [PATCH 24/32] Adjust table for easier comparisons --- text/3493-precise-pre-release-cargo-update.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index d7456713d08..432f3fb7f68 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -73,12 +73,12 @@ required by package `tmp-oyyzsf v0.1.0 (/home/ethan/.cache/cargo-temp/tmp-OYyZsF | Cargo.toml spec | Cargo.lock version | Target version | Selected by cargo update | Selected by cargo update --precise | | --------------- | ------------------ | -------------- | ------------------------- | ----------------------------------- | +| `^1.0.0` | `1.0.0` | `1.0.0-pre.0` | ❌ | ❌ | | `^1.0.0` | `1.0.0` | `1.2.0` | ✅ | ✅ | | `^1.0.0` | `1.0.0` | `1.2.0-pre.0` | ❌ | ✅ | | `^1.0.0` | `1.2.0-pre.0` | `1.2.0-pre.1` | ❌ | ✅ | -| `^1.0.0-pre.0` | `1.0.0-pre.0` | `1.0.0-pre.1` | ✅¹ | ✅ | -| `^1.0.0-pre.0` | `1.0.0-pre.0` | `1.2.0` | ✅¹ | ✅ | -| `^1.0.0` | `1.0.0` | `1.0.0-pre.0` | ❌ | ❌ | +| `^1.2.0-pre.0` | `1.2.0-pre.0` | `1.2.0-pre.1` | ✅¹ | ✅ | +| `^1.2.0-pre.0` | `1.2.0-pre.0` | `1.3.0` | ✅¹ | ✅ | ✅: Will upgrade From b43204abda58f36da95c053b7936692473c27082 Mon Sep 17 00:00:00 2001 From: Ed Page Date: Tue, 12 Dec 2023 13:59:34 -0600 Subject: [PATCH 25/32] Expand on Reference details --- text/3493-precise-pre-release-cargo-update.md | 45 +++++++++++++++---- 1 file changed, 37 insertions(+), 8 deletions(-) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index 432f3fb7f68..7dc4f7ff942 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -91,20 +91,49 @@ Since this RFC is concerned with the behaviour of `cargo update --precise` chang # Reference-level explanation [reference-level-explanation]: #reference-level-explanation -To determine if a version can be selected with `--precise` for a specification that isn't listed above cosider where pre-releases exist within version ranges. +## Version requirements -For example consider the version `~1.2.3`. -The range for `~1.2.3` is [stated in the cargo book](https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#tilde-requirements). +Version requirement operator semantics will change to encompass pre-release versions, compared to before where the presence of pre-release would change matching modes. +This will also better align with the mathematical properties associated with some of the operators (see the closed [RFC 3266](https://github.com/rust-lang/rfcs/pull/3266)). +So before, ``` -~1.2.3 := >=1.2.3, <1.3.0 +1.2.3 -> ^1.2.3 -> >=1.2.3, <2.0.0 (with implicit holes excluding pre-release versions) ``` +would become +``` +1.2.3 -> ^1.2.3 -> >=1.2.3, <2.0.0-0 +``` +Note that the old syntax implicitly excluded `2.0.0-` which we have have to explicitly exclude by referencing the smallest possible pre-release version of `-0`. + +This change applies to all operators. + +## Dependency Resolution + +The intent is to mirror the behavior of yanked today. + +When parsing a `Cargo.lock`, any pre-release version would be tracked in an allow-list. +When resolving, we would exclude from consideration any pre-release version unless: +- It is in the allow-list +- It matches the version requirement under the old pre-release version requirement semantics. + +## `cargo update` + +The version passed in via `--precise` would be added to the allow-list. + +**Note:** overriding of yanked via this mechanism is not meant to be assumed to be a part of this proposal. +Support for selecting yanked with `--precise` should be decided separately from this RFC, instead see [rust-lang/cargo#4225](https://github.com/rust-lang/cargo/issues/4225) -Intuitively `1.2.4-pre.0` satisfies this inequality, therefore it can be selected with `cargo update --precise`. -Since it is a pre-release and the specification is not, `1.2.4-pre.0` would not be selected by a bare `cargo update`. -`1.3.0-pre.0` also satisfies the inequality but `1.2.3-pre.0` and `1.3.1-pre.0` do not. +## [`semver`](https://crates.io/crates/semver) -Put in simple terms the relationship between a pre-release and its stable release is always `a.b.c-pre.0 < a.b.c`. +`cargo` will need both the old and new behavior exposed. +To reduce risk of tools in the ecosystem unintentionally matching pre-releases (despite them still needing an opt-in), +it might be reasonable for the +`semver` +package to offer this new matching behavior under a different name +(e.g. `VersionReq::matches_prerelease` in contrast to the existing `VersionReq::matches`) +(also avoiding a breaking change). +However, we leave the exact API details to the maintainer of the `semver` package. # Drawbacks [drawbacks]: #drawbacks From 78ad06011cc2dde3990567678568d59488db27cd Mon Sep 17 00:00:00 2001 From: Ed Page Date: Tue, 12 Dec 2023 14:00:29 -0600 Subject: [PATCH 26/32] Fix typo --- text/3493-precise-pre-release-cargo-update.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index 7dc4f7ff942..b2ce2edea4e 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -11,7 +11,7 @@ This effectively splits the notion of compatibility in `cargo`. A pre-release version may be considered compatible when the version is explicitly requested with `--precise`. Cargo will not automatically select that version via a basic `cargo update`. -One way to think of this is that we are changing from the the version +One way to think of this is that we are changing from the version requirements syntax requiring opt-in to match pre-release of higher versions to the resolver ignoring pre-releases like yanked packages, with an override flag. From 19fc1b6e5e34b2319325d634e3ac18d249e3e98e Mon Sep 17 00:00:00 2001 From: Ed Page Date: Tue, 12 Dec 2023 14:07:11 -0600 Subject: [PATCH 27/32] Make table narrower / clearer --- text/3493-precise-pre-release-cargo-update.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index b2ce2edea4e..b47371434a4 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -71,14 +71,14 @@ location searched: crates.io index required by package `tmp-oyyzsf v0.1.0 (/home/ethan/.cache/cargo-temp/tmp-OYyZsF)` ``` -| Cargo.toml spec | Cargo.lock version | Target version | Selected by cargo update | Selected by cargo update --precise | -| --------------- | ------------------ | -------------- | ------------------------- | ----------------------------------- | -| `^1.0.0` | `1.0.0` | `1.0.0-pre.0` | ❌ | ❌ | -| `^1.0.0` | `1.0.0` | `1.2.0` | ✅ | ✅ | -| `^1.0.0` | `1.0.0` | `1.2.0-pre.0` | ❌ | ✅ | -| `^1.0.0` | `1.2.0-pre.0` | `1.2.0-pre.1` | ❌ | ✅ | -| `^1.2.0-pre.0` | `1.2.0-pre.0` | `1.2.0-pre.1` | ✅¹ | ✅ | -| `^1.2.0-pre.0` | `1.2.0-pre.0` | `1.3.0` | ✅¹ | ✅ | +| Cargo.toml | Cargo.lock | Desired | Selectable w/o `--precise` | Selectable w/ `--precise` | +| ---------------| --------------| --------------| -------------------------- | ---------------------------| +| `^1.0.0` | `1.0.0` | `1.0.0-pre.0` | ❌ | ❌ | +| `^1.0.0` | `1.0.0` | `1.2.0` | ✅ | ✅ | +| `^1.0.0` | `1.0.0` | `1.2.0-pre.0` | ❌ | ✅ | +| `^1.0.0` | `1.2.0-pre.0` | `1.2.0-pre.1` | ❌ | ✅ | +| `^1.2.0-pre.0` | `1.2.0-pre.0` | `1.2.0-pre.1` | ✅¹ | ✅ | +| `^1.2.0-pre.0` | `1.2.0-pre.0` | `1.3.0` | ✅¹ | ✅ | ✅: Will upgrade From d8db5b1b0557138e16a2de01fbf538dd865c8f7a Mon Sep 17 00:00:00 2001 From: Ed Page Date: Tue, 12 Dec 2023 14:21:29 -0600 Subject: [PATCH 28/32] Address resolver error messages --- text/3493-precise-pre-release-cargo-update.md | 1 + 1 file changed, 1 insertion(+) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index b47371434a4..8f96b1eb88b 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -142,6 +142,7 @@ However, we leave the exact API details to the maintainer of the `semver` packag A change that makes use of a pre-release version may not be noticed during code review as reviewers don't always check for changes in the lock file. - Library crates that require a pre-release version are not well supported since their lock files are ignored by their users (see [future-possibilities]) - This is an invasive change to cargo with a significant risk for bugs. This also extends out to packages in the ecosystem that deal with depednency versions. +- There is a risk that error messages from the resolver may be negatively affected and we might be limited in fixes due to the resolver's current design. # Rationale and alternatives [rationale-and-alternatives]: #rationale-and-alternatives From ec033c91da1e36cf6c422f10043df95160c1e85b Mon Sep 17 00:00:00 2001 From: Ethan Brierley Date: Wed, 3 Jan 2024 10:05:49 +0000 Subject: [PATCH 29/32] Version ranges with pre-release upper bounds --- text/3493-precise-pre-release-cargo-update.md | 24 +++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index 8f96b1eb88b..07edb39abf0 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -214,6 +214,30 @@ through by modifying `Cargo.toml`. # Unresolved questions [unresolved-questions]: #unresolved-questions +# Version ranges with pre-release upper bounds + +[As stated earlier](#reference-level-explanation), this RFC proposes that semver version semantics change to encompass pre-release versions. + +``` +^1.2.3 -> >=1.2.3, <2.0.0-0 +``` + +The addition of an implicit `-0` excludes `2.0.0-` releases. +This transformation will also be made when the user explicitly specifies a multiple-version requirement range. + +``` +>=1.2.3, <2.0.0 -> >=1.2.3, <2.0.0-0 +``` + +This leaves a corner case for `>=0.14-0, <0.14.0`. +Intuitively the user may expect this range to match all `0.14` pre-releases, however, due to the implicit `-0` on the second version requirement, this range will not match any versions. +There are two ways we could go about solving this. + +- Only add the implicit `-0` to the upper bound if there is no pre-release on the lower bound. +- Accept the existance of this unexpected behaviour. + +Either way, it may be desirable to introduce a dedicated warning for this case. + # Future possibilities [future-possibilities]: #future-possibilities From ada56af098cd45dad805dec164e3e4f4c91cb68a Mon Sep 17 00:00:00 2001 From: Ethan Brierley Date: Wed, 3 Jan 2024 10:11:55 +0000 Subject: [PATCH 30/32] Fix minor typo --- text/3493-precise-pre-release-cargo-update.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index 07edb39abf0..9da1d29c5fd 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -234,7 +234,7 @@ Intuitively the user may expect this range to match all `0.14` pre-releases, how There are two ways we could go about solving this. - Only add the implicit `-0` to the upper bound if there is no pre-release on the lower bound. -- Accept the existance of this unexpected behaviour. +- Accept the existence of this unexpected behaviour. Either way, it may be desirable to introduce a dedicated warning for this case. From efc037469a280c81c30c6a8f13bb64830d4a0042 Mon Sep 17 00:00:00 2001 From: Ethan Brierley Date: Wed, 3 Jan 2024 20:16:25 +0000 Subject: [PATCH 31/32] Fix typos Co-authored-by: Eric Huss --- text/3493-precise-pre-release-cargo-update.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index 9da1d29c5fd..2e23b0424d7 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -104,7 +104,7 @@ would become ``` 1.2.3 -> ^1.2.3 -> >=1.2.3, <2.0.0-0 ``` -Note that the old syntax implicitly excluded `2.0.0-` which we have have to explicitly exclude by referencing the smallest possible pre-release version of `-0`. +Note that the old syntax implicitly excluded `2.0.0-` which we have have to explicitly exclude by referencing the smallest possible pre-release version of `-0`. This change applies to all operators. @@ -141,7 +141,7 @@ However, we leave the exact API details to the maintainer of the `semver` packag - Pre-release versions are not easily auditable when they are only specified in the lock file. A change that makes use of a pre-release version may not be noticed during code review as reviewers don't always check for changes in the lock file. - Library crates that require a pre-release version are not well supported since their lock files are ignored by their users (see [future-possibilities]) -- This is an invasive change to cargo with a significant risk for bugs. This also extends out to packages in the ecosystem that deal with depednency versions. +- This is an invasive change to cargo with a significant risk for bugs. This also extends out to packages in the ecosystem that deal with dependency versions. - There is a risk that error messages from the resolver may be negatively affected and we might be limited in fixes due to the resolver's current design. # Rationale and alternatives @@ -154,7 +154,7 @@ having `foo@1.2.3-alpha.0` in your `Cargo.lock` does not implicitly mean you can To update within pre-releases, you'll have to use `--precise` again. Instead, the lockfile could identify that `foo@1.2.3-alpha.0` is a pre-release allow updating to any `1.2.3` pre-release. -`cargo update` focuses on compatible updates and pre-releases aren't necesarrily compatible with each other +`cargo update` focuses on compatible updates and pre-releases aren't necessarily compatible with each other (see also [RFC: Precise Pre-release Deps](https://github.com/rust-lang/rfcs/pull/3263)). Alternatively, in [future-possibilities] is `cargo update -p foo --allow-prerelease` which would be an explicit way to update. @@ -222,7 +222,7 @@ through by modifying `Cargo.toml`. ^1.2.3 -> >=1.2.3, <2.0.0-0 ``` -The addition of an implicit `-0` excludes `2.0.0-` releases. +The addition of an implicit `-0` excludes `2.0.0-` releases. This transformation will also be made when the user explicitly specifies a multiple-version requirement range. ``` @@ -258,8 +258,8 @@ Since crates ignore the lock files of their dependencies there is no way for `a` To enable this we could use the same concept of compatible pre-releases in `Cargo.toml`, not just `Cargo.lock`. This would require that pre-releases are specified with `=` and would allow pre-release versions to be requested anywhere within the dependency tree without causing the resolver to throw an error. -## `--allow-prelease` +## `--allow-prerelease` -Instead of manually selecting a version with `--precise`, we could support `cargo update --package foo --allow-prelease`. +Instead of manually selecting a version with `--precise`, we could support `cargo update --package foo --allow-prerelease`. If we made this flag work without `--package`, we could the extend it also to `cargo generate-lockfile`. From d15d22d7e4f30c09aa65230020d86ebb08fe1226 Mon Sep 17 00:00:00 2001 From: Eric Huss Date: Sat, 13 Jan 2024 13:33:03 -0800 Subject: [PATCH 32/32] Add tracking issue --- text/3493-precise-pre-release-cargo-update.md | 1 + 1 file changed, 1 insertion(+) diff --git a/text/3493-precise-pre-release-cargo-update.md b/text/3493-precise-pre-release-cargo-update.md index 2e23b0424d7..cc22c375be2 100644 --- a/text/3493-precise-pre-release-cargo-update.md +++ b/text/3493-precise-pre-release-cargo-update.md @@ -1,6 +1,7 @@ - Feature Name: precise-pre-release-cargo-update - Start Date: 2023-09-20 - RFC PR: [rust-lang/rfcs#3493](https://github.com/rust-lang/rfcs/pull/3493) +- Tracking Issue: [rust-lang/cargo#13290](https://github.com/rust-lang/cargo/issues/13290) # Summary [summary]: #summary