RFC: Allow crates to specify the version of Rust in which they are written

[Ericson2314]

This is an alternative to #1707 . I also plan to write an RFC covering the "Cargo.toml specification language", but the implementation for that is quite different because older Cargo can't even attempt to to parse the package, rather than merely not use it in the build plan. For this reason, perhap the two concepts should be handled separately.

Rendered

[oli-obk]

Adding explicit versions to Cargo.toml is rather tedious. Couldn't cargo publish also notify the repository (crates.io) about the version it was compiled with (maybe by automatically inserting some info into Cargo.toml).

Then, the developer could cargo publish the same version multiple times with different compiler versions, and these compiler versions would be added to the published crate.

[Ericson2314]

Hmm, I don't think information should be added to Cargo.toml on upload. But cargo new can add the field based whatever rustc is on PATH, and cargo publish can complain if it's not present while proposing how to fill in the field the same way.

[jethrogb]

This field is added in which section of Cargo.toml?

Could this be a dependency instead à la #1133 ?

Can you add a 'rendered' link to the first post?

[Ericson2314]

@jethrogb

This field is added in which section of Cargo.toml?

Top level. @brson and I discovered that it's basically impossible to add anything not on the top level to Cargo.toml without breaking old Cargo versions. This will half to be solved before #1133 can be merged.

Could this be a dependency instead à la #1133 ?

Dependency of what?

Can you add a 'rendered' link to the first post?

Done

[eternaleye]

From IRC:

not sure what, if anything, is needed to prepare for also tracking language features
Ericson2314: Well, it's kind of like OpenGL
OpenGL X is an aggregation of some number of OpenGL extensions
Substitute OpenGL version for Rust version, and extensions for language #[feature] flags that have been stabilized.
Personally, I'd favor rust = "version" overloaded with rust = { feature1 = stable, feature2 = stable, feature3 = * }
that makes sense
the -version is sort of redundant
Or possibly them as separate keys, both optional - version and features
As an example of where it matters, consider an alternate rust compiler, which implements associated items very early, but otherwise is only compatible up to Rust 1.2
that was my other thought
I'm fine either way
yeah definitely
should features either be '' or stable?
Also, feature1 = stable means it implements feature in the form it was stabilized; not that "this implements the version of rust it was first stable in"
Well, that's the thing
If we ever want to support actually using unstable features, we should track when those features change pre-stabilization
And then you have, say, "simd = 0" vs "simd = 1" vs "simd = 2", and "simd = stable" is the same as "simd = 2"
eternaleye: agree on meaning of stable
yeah that sounds nice, but probably more controversial
as its convenient to break unstable thing whenever
Yeah, hence my use of "" - it's compatible with our current meaning of "*" around semver
yeah, good move :)
Moreover, banning * from crates.io fits current policies
While still allowing feature-based Rust requirements

[joshtriplett]

Rather than making this a separate item, why not define this in a similar style to #1707, with a versioned dependency on lang:rust? That would allow, for instance, declaring a different dependency for dev-dependencies (perhaps tests need newer Rust to check the interaction with some feature), or declaring a different dependency based on a feature flag (enable a foo feature to use the foo feature of latest Rust).

[joshtriplett]

I'd also ask whether it makes more sense to version the language or to use versions of rustc itself. In principle, I like the idea of versioning the language, but doing so would require someone to maintain a semver for the language itself separately from rustc.

If you decide to do that, I'd suggest separating the version of the standard library from the version of the language. You can determine the version of the standard library you depend on fairly easily: feed that version to rustc and tell it to not allow use of anything with a stability marker for a newer version.

The language seems a bit harder to track a semver for.

[Ericson2314]

Rather than making this a separate item, why not define this in a similar style to #1707, with a versioned dependency on lang:rust?

I'm not necessarily against that. I think it would be even more useful for features than tests. As I said in your rfc's thread, feel free to mine this RFC for motivation or whatever. One concern is that your proposal may break old versions of Cargo---Cargo only ignores certain unexpected keys, and mainly on the top level. Then again, this isn't really RFC's fault and we should have a better story for changing the Cargo.toml schema.

I'd also ask whether it makes more sense to version the language or to use versions of rustc itself. In principle, I like the idea of versioning the language, but doing so would require someone to maintain a semver for the language itself separately from rustc.

I think for the foreseeable future rust and Rust will have the same version, so this isn't really a burden right now? In practice each rustc release defines the rustc version cause we don't have a formal spec.

If you decide to do that, I'd suggest separating the version of the standard library from the version of the language.

I like that. @eternaleye and I both hope parts of the standard library can be written in stable rust and migrated to crates.io. I think I'd amend #1133 to make standard library version requirements optional: if they are not filled in they are inferred from the language version.

You can determine the version of the standard library you depend on fairly easily: feed that version to rustc and tell it to not allow use of anything with a stability marker for a newer version.

True. I was hoping CI could be automated to use these versions---that would work for stdlib and language alike.

[cuviper]

I think it's fine to leave the language and libstd versions tied to rustc. It's similar to how Python versions are tied to CPython as the reference implementation.

[Ericson2314]

@cuviper even if the language version is defined by rustc, it's still nice to have that field in the verbose output for the sake of other implementations being able to inter-operate with cargo.

[eternaleye]

@joshtriplett @cuviper: I personally see it as that (currently), the rustc implementation is the spec and includes within itself the only stdlib, and thus the "version" of the spec is the same as that of rustc is the same as that of the stdlib.

In the future, the stdlib may be freed from rustc (by any number of means, not relevant here) and that may diverge in terms of numbering; further on, the language spec may be maintained in its own right and be versioned apart from any implementation.

In the very long run, my hope is that we'd have a process around

1. Specifying experimental language features
2. Getting them implemented in independent implementations
3. Iterating until the design is "ready"
4. Include the language feature in a language version
5. Implementations declare that they support that version

Language versions then become simple sets of features. This resembles (in different ways) both the OpenGL and C/C++ processes, all of which are quite successful.

[joshtriplett]

@Ericson2314

One concern is that your proposal may break old versions of Cargo---Cargo only ignores certain unexpected keys, and mainly on the top level.

I tested that specifically, and documented the result in RFC #1707. Cargo just treats it as an unknown crate, and handles it gracefully. If you depend on tool:cargo or tool:rustc, and you actually require that dependency, Cargo will say that it couldn't find that crate; if you don't actually require that dependency, such as for a non-default named feature, Cargo will ignore it and continue. That seems like the correct behavior.

[Ericson2314]

@joshtriplett OK that's a pretty reasonable failure case.

[ruuda]

If the field is absent, we assume the package is written in Rust 1.0.

Does it make sense to assume 1.0 at this point? With the release trains, how much code still uses 1.0 nowadays? On the other hand, assuming any other version would be arbitrary.

eternaleye Substitute OpenGL version for Rust version, and extensions for language #[feature] flags that have been stabilized.
eternaleye Personally, I'd favor rust = "version" overloaded with rust = { feature1 = stable, feature2 = stable, feature3 = * }
Ericson2314 that makes sense

This smells of the web platform versioning havoc, where people were relying on user agent versions to infer capabilities. Everything was a mess, and eventually they figured out that it is much better to query for individual features. I’m not sure whether it would be an issue in the case of Rust, because at least we have semantic versioning. But if we would specify features instead of versions, then they should be enforced (the compiler should not allow any features that were not specified explicitly), otherwise it would be easy to accidentally use features. It is reminiscent of Haskell’s extensions field in the Cabal file.

[eternaleye]

@ruuda:

This smells of the web platform versioning havoc, where people were relying on user agent versions to infer capabilities.

I strongly disagree: Which unstable features were stabilized in a Rust version is the single largest differentiator of Rust language versions.

Moreover, by banning "*" from crates.io, the only features published crates could legally depend on would be those that have been included in a stable release - which, as I mentioned above, is currently equivalent to being included in a version of the language spec.

This RFC isn't about specifying the version of any specific implementation (which is what user agent sniffing tries to do); it's about specifying the version of the language standard - more like saying "I want associated consts, which were standardized in 1.whatever, but other than that I don't need any features beyond those standardized in 1.2"

In the OpenGL world, this works very, very well: OpenGL Whatever is defined exactly by the extensions it contains - a version is merely a shorthand for listing extensions. The result is quite close to what you're advocating - listing what extensions you require - just with a shorthand that makes it actually feasible, rather than an incredible pain in the ass given we have quite a few stabilized features already.

Your other suggestion - preventing the use of features not explicitly listed - is (as far as I can tell) 100% infeasible and would require drastic surgery on rustc. Sadly.

[Ericson2314]

@ruuda

Everything was a mess, and eventually they figured out that it is much better to query for individual features.

I suppose this could be considered decoratively querying for them all at once? It's certainly better than vendor prefixes, to take another web example.

But if we would specify features instead of versions, then they should be enforced (the compiler should not allow any features that were not specified explicitly), otherwise it would be easy to accidentally use features.

Your other suggestion - preventing the use of features not explicitly listed - is (as far as I can tell) 100% infeasible and would require drastic surgery on rustc. Sadly.

With already stable features, this is true: they could "blend" with other stable features after they've been stabilized and thus there is no way to toggle them individually. For unstable features this is quite possible and I agree a good idea.

It is reminiscent of Haskell’s extensions field in the Cabal file.

It is weird that this does fairly little for Cabal. I think the Haskell idiom of trying out new ideas as features the people use is good. Right now, yes there is the nightly ecosystem (and almost everything I do is part of it) but the "semi-stability" of versioned unstable features (may not work forever but no sudden breakage) might invite more users.

[ruuda]

With already stable features, this is true: they could "blend" with other stable features after they've been stabilized and thus there is no way to toggle them individually.

The result is quite close to what you're advocating - listing what extensions you require - just with a shorthand that makes it actually feasible, rather than an incredible pain in the ass given we have quite a few stabilized features already.

I agree that specifying all features would be infeasible (both for the compiler to be able to toggle them individually, and for the user to specify them all), I just wanted to point out the similarity. You are right, in a sense Rust versions would be a shorthand for feature sets, and this will probably work out fine.

But the issue of enforcing that the version is correct remains: what if I say rust-version = "1.5" in my Cargo.toml, but actually the code relies on e.g. vec::drain which was stabilized in 1.6? Then if my local compiler is semver-compatible with 1.6, everything will compile just fine locally, but it will fail on a 1.5 compiler, even though Cargo will think everything is compatible. Edit: to clarify, my concern here is that it is easy to accidentally get the version requirement wrong without knowing.

But perhaps enforcing this is not Cargo’s responsibility? It could be left to crate authors, who could rely on CI for instance. With Travis CI it’s only a few lines to test on older compilers.

[joshtriplett]

The library already include stability annotations that say "stable as of version N". It seems potentially feasible to have the compiler let you pass a version number, and enforce that you can't access anything with a stability annotation for a newer version.

[Ericson2314]

@ruuda yeah outside of CI, this is honor system and unenforceable. Unfortunately, but no worse then version reqs on crates in that regard. :/

[Ericson2314]

@joshtriplett That's a lot https://internals.rust-lang.org/t/pre-rfc-a-vision-for-platform-architecture-configuration-specific-apis/3502 preventing one from accidentally using unportable stuff. I think it's a good idea, but best left as future work https://internals.rust-lang.org/t/pre-rfc-a-vision-for-platform-architecture-configuration-specific-apis/3502 (a generalization is when two dependencies are unified, make sure their consumers only get what they asked for).

[Ericson2314]

Working on implementation https://github.com/QuiltOS/cargo/tree/rust-version

[ruuda]

What is the status of this? I ran into an issue today that I believe could have been prevented entirely by this RFC in its current form.

[DanielKeep]

Relevant prior RFC and discussion in #457.

[nrc]

I like the idea here, and I like the simplicity. I believe we will want to expand into features and more precise nightly versioning, etc. But I think it is best to leave that for the future. Likewise, Cargo (in conjunction with Rustup) could automatically get the right version to build a crate and use that, but that seems like something to leave until later.

I think a language version, and not versions for compiler, libs, etc. is the right approach.

I don't have a preference between the proposed ad hoc value and using a dependency.

My only worry is that 1.0 is the wrong default and that there should be an 'unspecified' default. The reason is that 1.0 is effectively the 'strongest' possible value (the hardest for a program to specify) and just because the program compiles today, there is no guarantee that it compiles with version 1.0.

Random idea: the lockfile (or some other metadata store) could contain two versions - the specified version and the tested version, where the former is the value specified in the toml file and the later is the value Cargo actually used to build the crate. CI could expand the tested value to a list of tested versions. For anything user-facing, if the specified value is not specified, then the tested value can be used.

[joshtriplett]

Reposting some concern discussed in #1707:

• This approach doesn't integrate with all the existing Cargo facilities to make dependencies optional. For instance, consider a crate that needs Rust 1.14 with a given feature enabled, but Rust 1.10 with that feature disabled.
• This approach does not parse with existing Cargo.
• This approach doesn't support versions of Cargo in addition to versions of Rust.

[eternaleye]

@joshtriplett: As one of the more outspoken proponents of the "depend on features of the compiler" end of things, I do feel a need to raise a cautionary note that the semantics of this may well not match up as expected with cargo features on crate dependencies.

As a result, I'd be hesitant to say that #1707's syntax does in fact "allow it" - it allows specifying crate-style feature requirements, but it's unknown whether the semantics actually line up.

In essence, the "language version" consists of five parts:

1. The language syntax, which is determined by the stabilized #![feature(...)]'s like associated_constants or type_ascription
2. The controls on compiler behavior, which are determined by the stabilization of things like Macros 1.1, cdylib, and custom allocators (which also have #![feature(...)] annotations, or should)
3. The stabilization of std APIs, which have their own #![feature(...)] flags
4. The default Cargo features the facade and the crates behind it are built with
5. Soundness fixes

Of all of these, only 5 is not expressible as a "feature" of some description - but 1-3 are a different kind of "feature" than 4, and only 4 necessarily maps to what Cargo usually means. In addition, 4 is perhaps better handled by hanging features off of dependencies on the stdlib, not the compiler. (I also feel like bug fixes in the stdlib are better handled with stdlib deps, and not compiler deps).

Beyond that, if there's ever going to be support for using dependencies on #![feature(...)]'s to make nightly-only things more principled/declarative, then these features need to, themselves, be versioned in some manner.

I don't really have a solution to these concerns; this is more a plea for caution in presuming that the syntax we have (even extended) matches up with the tooling we would need for this use case.

[joshtriplett]

@eternaleye I agree that Cargo features don't fit perfectly, and soundness fixes or other semantically significant fixes are exactly what I had in mind when I said that versions would still be required.

Personally, I do think that items 1 and 2 would work as Cargo features of rustc, and 3 and 4 belong either on rustc or std depending on if we split out standard library dependencies as separate. Distinctions like those are precisely why I think it makes sense to separate "syntax for tool dependencies" from "features", and separate both from "dependencies on std" (e.g. #1133). Supporting versioned dependencies on rustc would provide an improvement over the current situation, and due to things like soundness and semantic fixes, we know we'll need versioned dependencies even if we also support feature dependencies.

[eternaleye]

@joshtriplett: Well, I'm very bullish on splitting out std at least logically (in terms of dependencies), even if it's not split out physically (in terms of deployment/maintenance).

However, treating both 3 and 4 as Cargo features introduces another problem: You are now trying to fit two namespaces into one. In addition, I'm firmly of the opinion that 3 is a matter of std's contract, which should be part of the language spec version, while 4 is a matter of the implementation (and, incidentally, covers things like "sanitizers" and "default to jemallic").

As a result, I am very much not convinced 3 and 4 should both be mapped to Cargo features of std, even (especially) if it's split out.

To be honest, I see this as "The Revenge of the Son of Interface vs. Implementation" - there's the contract of any implementation of std, denoted by #![feature(...)]s and spec version, and there's the tunables of the current sole implementation (with its own version, currently equal due to failing to define a spec), denoted by the Cargo features the std crate already has.

[joshtriplett]

@eternaleye All of those seem like reasonable concerns to me. And I agree that all of them need resolving as part of any RFC to support feature dependencies on rustc or std.

I just don't think we need to resolve that before we can introduce any mechanism to allow specifying dependencies on tools like rustc or cargo.

[Ericson2314]

I think the first step is introducing Cargo.toml schema versions. This would be a single top-level key (like this), ignored by older Cargo. Then we can experiment more freely. This is also needed for std-aware cargo and, if we do the same for lockfiles, deterministic crate source hashes.

Note that cargo.toml schema breaking changes are no big deal because Cargo can easily learn how to parse all versions and make plans with a mixes.

[joshtriplett]

@Ericson2314 "Ignored by older Cargo" doesn't seem like a feature here; older Cargo needs to know that it doesn't understand the newer file.

[Ericson2314]

@joshtriplett True. We can find a different syntax for that, then. Or just live with it and declare the first Cargo understanding the field as truly forward compatible, cause we currently have no story right now one what older versions of Cargo do with other schema that post-date them.

[joshtriplett]

Summarizing some discussions today: @Ericson2314, @eternaleye, and I met on IRC, talked through the problem, and ended up drafting a successor RFC via Etherpad for both #1707 and #1709. We'll submit that RFC in the near future; that will serve as a baseline to solve the most critical core problem, and then we can build further work on top of that.

[nrc]

@joshtriplett is this RFC superseded by #1953? Should it be closed?

[Ericson2314]

@nrc technically this and that do different things: this is about the rust langauge requirements, that is about the cargo langauge requirements.

The teams seem to want a single version for everything, as expressed in the thread of that other RFC. That would entail taking elements from this and rewriting heavily.

[joshtriplett]

@nrc Our intention with #1953 is that the mechanism it provided for cargo versioning would supersede both, but that there would then be a new version of this RFC built atop #1953. So, the hope would be to resolve #1953 first, then adapt #1709 based on however that turns out.

[nrc]

Unassigning myself because this moved to a team I'm not a part of.

Ping @rust-lang/cargo for a new shepherd (and maybe an update).

[withoutboats]

@rfcbot fcp close

We're pretty certain we want to use a single version for both Rust and cargo, seems to make sense to track this discussion in one place, which is currently RFC #1953.

[rfcbot]

Team member @withoutboats has proposed to close this. The next step is review by the rest of the tagged teams:

• @alexcrichton
• @aturon
• @carols10cents
• @matklad
• @withoutboats
• @wycats

No concerns currently listed.

Once these reviewers reach consensus, this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[Ericson2314]

Oh, just saw this. Fine with me.

[kornelski]

#2495