From b921f08dbea89d3888ba05b93ad233a868557fed Mon Sep 17 00:00:00 2001 From: Joshua Nelson Date: Mon, 30 Nov 2020 18:12:04 -0500 Subject: [PATCH 01/34] Add RFC for dependencies on binaries --- text/0000-cargo-binary-dependencies.md | 140 +++++++++++++++++++++++++ 1 file changed, 140 insertions(+) create mode 100644 text/0000-cargo-binary-dependencies.md diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md new file mode 100644 index 00000000000..b2b6cb00dfd --- /dev/null +++ b/text/0000-cargo-binary-dependencies.md @@ -0,0 +1,140 @@ +- Feature Name: (`bindeps`) +- Start Date: 2020-11-30 +- RFC PR: [rust-lang/rfcs#3028](https://github.com/rust-lang/rfcs/pull/3028) +- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000) + +# Summary +[summary]: #summary + +Allow Cargo packages to depend on `bin`, `cdylib`, and `staticlib` crates, and use the artifacts built by those crates. + +# Motivation +[motivation]: #motivation + +There are many different possible use cases. + +- [Testing the behavior of a binary](https://github.com/rust-lang/cargo/issues/4316#issuecomment-361641639). Currently, this requires invoking `cargo build` recursively, or running `cargo build` before running `cargo test`. +- [Running a binary that depends on another](https://github.com/rust-lang/rustc-perf/tree/master/collector#how-to-benchmark-a-change-on-your-own-machine). Currently, this requires running `cargo build`, making it difficult to keep track of when the binary was rebuilt. The use case for `rustc-perf` is to have a main binary that acts as an 'executor', which executes `rustc` many times, and a smaller 'shim' which wraps `rustc` with additional environment variables and arguments. +- [Building tools needed at build time](https://github.com/rust-lang/rust/pull/79540#unresolved-questions). Currently, this requires either splitting the tool into a library crate (if written in Rust), or telling the user to install the tool on the host and detecting the availability of it. This feature would allow building the necessary tool from source and then invoking it from a `build.rs` script later in the build. +- Building and embedding binaries for another target, such as firmware or WebAssembly. This feature would allow a versioned dependency on an appropriate crate providing the firmware or WebAssembly binary, and then embedding the binary (or a compressed or otherwise transformed version of it) into the final crate. For instance, a virtual machine could build its system firmware, or a WebAssembly runtime could build helper libraries. +- Building and embedding a shared library for use at runtime. For instance, a binary could depend on a shared library used with [`LD_PRELOAD`](https://man7.org/linux/man-pages/man8/ld.so.8.html#ENVIRONMENT), or used in the style of the Linux kernel's [VDSO](https://man7.org/linux/man-pages/man7/vdso.7.html). + +# Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + +Cargo allows you to depend on binary or C ABI artifacts of another package; this is known as a "binary dependency" or "artifact dependency". For example, you can depend on the `cmake` binary in your `build.rs` like so: + +```toml +[build-dependencies] +cmake = { version = "1.0", type = "bin" } +``` + +Cargo will build the `cmake` binary, then make it available to your `build.rs` through an environment variable: + +```rust +// build.rs +use std::{env, process::Command}; + +fn main() { + let cmake_path = env::var_os("CARGO_BIN_FILE_CMAKE_cmake").expect("cmake binary"); + let mut cmake = Command::new(cmake_path).arg("--version"); + assert!(cmake.status().expect("cmake --version failed").success()); +} +``` + +You can optionally specify specific binaries to depend on using `bins`: + +```toml +[build-dependencies] +cmake = { version = "1.0", type = "bin", bins = ["cmake"] } +``` + +If no binaries are specified, all the binaries in the package will be built and made available. + +You can obtain the directory containing all binaries built by the `cmake` crate with `CARGO_BIN_DIR_CMAKE`, such as to add it to `$PATH` before invoking another build system or a script. + +Cargo also allows depending on `cdylib` or `staticlib` artifacts. For example, you can embed a dynamic library in your binary: + +```rust +// main.rs +const MY_PRELOAD_LIB: &[u8] = include_bytes!(env!("CARGO_CDYLIB_FILE_MYPRELOAD")); +``` + +Note that cargo cannot help you ensure these artifacts are available at runtime for an installed version of a binary; cargo can only supply these artifacts at build time. Runtime requirements for installed crates are out of scope for this change. + +If you need to depend on multiple variants of a crate, such as both the binary and library of a crate, you can supply an array of strings for `type`: `type = ["bin", "lib"]`. + +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +There are four `type`s available: +1. `"lib"`, the default +2. `"bin"`, a crate building one or more binaries +3. `"cdylib"`, a C-compatible dynamic library +4. `"staticlib"`, a C-compatible static library + +Artifact dependencies can appear in any of the three sections of dependencies (or in target-specific versions of these sections): +- `[build-dependencies]` +- `[dependencies]` +- `[dev-dependencies]` + +By default, `build-dependencies` are built for the host, while `dependencies` and `dev-dependencies` are built for the target. You can specify the `target` attribute to build for a specific target, such as `target = "wasm32-wasi"`; a literal `target = "target"` will build for the target even if specifing a build dependency. (If the target is not available, this will result in an error at build time, just as if building the specified crate with a `--target` option for an unavailable target.) + +Cargo provides the following environment variables to the crate being built: + +- `CARGO__DIR_`, where `` is the `type` of the artifact (uppercased) and `` is the package of the crate being depended on. (As with other Cargo environment variables, crate names are converted to uppercase, with dashes replaced by underscores.) This is the directory containing all the artifacts from the crate. +- `CARGO__FILE__`, where `` is the `type` of the artifact, `` is the package of the crate being depended on, and `` is the name of the artifact. (Note that `` is *not* modified in any way from the `name` specified in the crate supplying the artifact, or the crate name if not specified; for instance, it may be in lowercase, or contain dashes.) This is the full path to the artifact. + - For the crate types `cdylib` and `staticlib` that can (currently) only build one artifact, cargo additionally supplies this variable with the `_` suffix omitted. + +For each kind of dependency, these variables are supplied to the same part of the build process that has access to that kind of dependency: +- For `build-dependencies`, these variables are supplied to the `build.rs` script, and can be accessed using `std::env::var_os`. (As with any OS file path, these may or may not be valid UTF-8.) +- For `dependencies`, these variables are supplied during the compilation of the crate, and can be accessed using `env!`. +- For `dev-dependencies`, these variables are supplied during the compilation of examples, tests, and benchmarks, and can be accessed using `env!`. + +(See the "Future possibilities" section for a note about the use of `env!`.) + +Similar to features, if other crates in your dependencies also depend on the same binary crate, and request different binaries, Cargo will build the union of all binaries requested. + +Cargo will unify features and versions across dependencies of all types, just as it does for multiple dependencies on the same crate throughout a dependency tree. + +`type` may be a string, or a list of strings; in the latter case, this specifies a dependency on the crate with each of those types, and is equivalent to specifying multiple dependencies with different `type` values. For instance, you may specify a build dependency on both the binary and library of the same crate. You may also specify separate dependencies of different `type`s; for instance, you may have a build dependency on the binary of a crate and a dependency on the library of the same crate. + +Cargo does not take the specified `type` values into account when resolving a crate's version; it will resolve the version as normal, and then produce an error if that version does not support all the specified `type` values. Similarly, Cargo will produce an error if that version does not build all the binaries required by the `bins` value. Removing a crate type or binary is a semver-incompatible change. (Any further semver requirements on the interface provided by a binary or library depend on the nature of the binary or library in question.) + +Until this feature is stabilized, it will require specifying the nightly-only option `-Z bindeps` to `cargo`. If `cargo` encounters a binary dependency or artifact dependency and does not have this option specified, it will emit an error and immediately stop building. + +# Drawbacks +[drawbacks]: #drawbacks + +Some of the motivating use cases have alternative solutions, such as extracting a library from a tool written in Rust, and making the tool a thin wrapper around the library. Making this change may potentially reduce the motivation to extract such libraries. However, many of the other use cases do not currently have any solutions, and extracted libraries have additional value even after this feature becomes available, so we don't see this as a reason to avoid introducing this feature. + +Adding this feature will make Cargo usable for many more use cases, which may motivate people to use Cargo in more places and stretch it even further; this may, in turn, generate more support and more feature requests. + +# Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +This RFC proposes supplying both the root directory and the path to each specific artifact. The path to specific artifacts is useful for accessing that specific artifact, and avoids needing target-specific knowledge about the names of executables (`.exe`) or libraries (`lib*.so`, `*.dll`, ...). The root directory is useful for `$PATH`, `$LD_LIBRARY_PATH`, and similar. Going from one to the other requires making assumptions. We believe there's value in supplying both. + +We could specify a `target = "host"` value to build for the host even for `[dependencies]` or `[dev-dependencies]` which would normally default to building for the target. If any use case arises for such a dependency, we can easily add that. + +We could make information about artifact dependencies in `[dependencies]` available to the `build.rs` script, which would allow running arbitrary Rust code to work with such dependencies at build time (rather than being limited to `env!`, proc macros, and constant evaluation). However, we can achieve the same effect with an entry in `[build-dependencies]` that has `target = "target"`, and that model seems simpler to explain and to work with. + +# Prior art +[prior-art]: #prior-art + +- Cargo already provides something similar to this for C library dependencies of -sys crates. A `-sys` crate can supply arbitrary artifact paths, for libraries, headers, and similar. Crates depending on the `-sys` crate can obtain those paths via environment variables supplied via Cargo, such as to compile other libraries using the same C library. This proposal provides a similar feature for other types of crates and libraries. +- `make`, `cmake`, and many other build systems allow setting arbitrary goals as the dependencies of others. This allows building a binary and then running that binary in a rule that depends on that binary. + +# Unresolved questions +[unresolved-questions]: #unresolved-questions + +How easily can Cargo handle a dependency with a different target specified? How will that interact with dependency resolution? Cargo already has to handle dependencies for both host and target (for cross-compilation), so those cases should already work. + +# Future possibilities +[future-possibilities]: #future-possibilities + +Currently, there's no mechanism to obtain an environment variable's value at compile time if that value is not valid UTF-8. In the future, we may want an `env_os!` macro, analogous to `std::env::var_os`, which returns a `&'static OsStr` rather than a `&'static str`. This is already an issue for existing environment variables supplied to the build that contain file paths. + +In some cases, a crate may want to depend on a binary without unifying features or dependency versions with that binary. A future extension to this mechanism could allow cargo to build a binary crate in isolation, without attempting to do any unification. + +Just as a -sys crate can supply additional artifacts other than the built binary, this mechanism could potentially expand in the future to allow building artifacts other than the built binary, such as C-compatible include files, various types of interface definition or protocol definition files, or arbitrary data files. From fac809f5f5b12afbf87056a83575eb1fb919852f Mon Sep 17 00:00:00 2001 From: Joshua Nelson Date: Tue, 1 Dec 2020 14:29:56 -0500 Subject: [PATCH 02/34] Clarify what `lib` means --- text/0000-cargo-binary-dependencies.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index b2b6cb00dfd..129f302cd31 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -73,6 +73,10 @@ There are four `type`s available: 3. `"cdylib"`, a C-compatible dynamic library 4. `"staticlib"`, a C-compatible static library +`"lib"` corresponds to all crates that can be depended on currently, +including `lib`, `rlib`, and `proc-macro` libraries. +See [linkage](https://doc.rust-lang.org/reference/linkage.html) for more information. + Artifact dependencies can appear in any of the three sections of dependencies (or in target-specific versions of these sections): - `[build-dependencies]` - `[dependencies]` @@ -137,4 +141,4 @@ Currently, there's no mechanism to obtain an environment variable's value at com In some cases, a crate may want to depend on a binary without unifying features or dependency versions with that binary. A future extension to this mechanism could allow cargo to build a binary crate in isolation, without attempting to do any unification. -Just as a -sys crate can supply additional artifacts other than the built binary, this mechanism could potentially expand in the future to allow building artifacts other than the built binary, such as C-compatible include files, various types of interface definition or protocol definition files, or arbitrary data files. +Just as a `-sys` crate can supply additional artifacts other than the built binary, this mechanism could potentially expand in the future to allow building artifacts other than the built binary, such as C-compatible include files, various types of interface definition or protocol definition files, or arbitrary data files. From d45fb22b599a4442ebf9b415b8c4fd45355725e1 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Tue, 8 Dec 2020 12:57:31 -0800 Subject: [PATCH 03/34] Add note about the alternative of a hardcoded binary path --- text/0000-cargo-binary-dependencies.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index 129f302cd31..aafb2344217 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -123,6 +123,8 @@ We could specify a `target = "host"` value to build for the host even for `[depe We could make information about artifact dependencies in `[dependencies]` available to the `build.rs` script, which would allow running arbitrary Rust code to work with such dependencies at build time (rather than being limited to `env!`, proc macros, and constant evaluation). However, we can achieve the same effect with an entry in `[build-dependencies]` that has `target = "target"`, and that model seems simpler to explain and to work with. +We could install all binaries into a common binary directory with a well-known path under `$OUT_DIR`, and expect crates to use that directory, rather than passing in paths via environment variables. `npm` takes an approach like this. However, this would not allow dependencies on multiple distinct binaries with the same name, either provided by different crates or provided by the same crate built for different targets. Hardcoded paths would also reduce the flexibility of Cargo to change these paths in the future, such as to accommodate new features or extensions. + # Prior art [prior-art]: #prior-art From 0670af007fb780a01eeac55653c0e8ea0c0f4bbf Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Sun, 13 Dec 2020 15:19:34 -0800 Subject: [PATCH 04/34] Add SPIR-V shaders as an example --- text/0000-cargo-binary-dependencies.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index aafb2344217..ad27939c569 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -16,7 +16,7 @@ There are many different possible use cases. - [Testing the behavior of a binary](https://github.com/rust-lang/cargo/issues/4316#issuecomment-361641639). Currently, this requires invoking `cargo build` recursively, or running `cargo build` before running `cargo test`. - [Running a binary that depends on another](https://github.com/rust-lang/rustc-perf/tree/master/collector#how-to-benchmark-a-change-on-your-own-machine). Currently, this requires running `cargo build`, making it difficult to keep track of when the binary was rebuilt. The use case for `rustc-perf` is to have a main binary that acts as an 'executor', which executes `rustc` many times, and a smaller 'shim' which wraps `rustc` with additional environment variables and arguments. - [Building tools needed at build time](https://github.com/rust-lang/rust/pull/79540#unresolved-questions). Currently, this requires either splitting the tool into a library crate (if written in Rust), or telling the user to install the tool on the host and detecting the availability of it. This feature would allow building the necessary tool from source and then invoking it from a `build.rs` script later in the build. -- Building and embedding binaries for another target, such as firmware or WebAssembly. This feature would allow a versioned dependency on an appropriate crate providing the firmware or WebAssembly binary, and then embedding the binary (or a compressed or otherwise transformed version of it) into the final crate. For instance, a virtual machine could build its system firmware, or a WebAssembly runtime could build helper libraries. +- Building and embedding binaries for another target, such as firmware, WebAssembly, or SPIR-V shaders. This feature would allow a versioned dependency on an appropriate crate providing the binary, and then embedding the binary (or a compressed or otherwise transformed version of it) into the final crate. For instance, a virtual machine could build its system firmware, or a WebAssembly runtime could build helper libraries. - Building and embedding a shared library for use at runtime. For instance, a binary could depend on a shared library used with [`LD_PRELOAD`](https://man7.org/linux/man-pages/man8/ld.so.8.html#ENVIRONMENT), or used in the style of the Linux kernel's [VDSO](https://man7.org/linux/man-pages/man7/vdso.7.html). # Guide-level explanation From 9bc481dcec8fdd457d3f7670f5e43687efccb725 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Sun, 13 Dec 2020 15:33:04 -0800 Subject: [PATCH 05/34] Clarify that type and crate name transformations happen in both cases --- text/0000-cargo-binary-dependencies.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index ad27939c569..11b8deccc4c 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -87,7 +87,7 @@ By default, `build-dependencies` are built for the host, while `dependencies` a Cargo provides the following environment variables to the crate being built: - `CARGO__DIR_`, where `` is the `type` of the artifact (uppercased) and `` is the package of the crate being depended on. (As with other Cargo environment variables, crate names are converted to uppercase, with dashes replaced by underscores.) This is the directory containing all the artifacts from the crate. -- `CARGO__FILE__`, where `` is the `type` of the artifact, `` is the package of the crate being depended on, and `` is the name of the artifact. (Note that `` is *not* modified in any way from the `name` specified in the crate supplying the artifact, or the crate name if not specified; for instance, it may be in lowercase, or contain dashes.) This is the full path to the artifact. +- `CARGO__FILE__`, where `` is the `type` of the artifact (transformed as above), `` is the package of the crate being depended on (transformed as above), and `` is the name of the artifact. (Note that `` is *not* modified in any way from the `name` specified in the crate supplying the artifact, or the crate name if not specified; for instance, it may be in lowercase, or contain dashes.) This is the full path to the artifact. - For the crate types `cdylib` and `staticlib` that can (currently) only build one artifact, cargo additionally supplies this variable with the `_` suffix omitted. For each kind of dependency, these variables are supplied to the same part of the build process that has access to that kind of dependency: From 10566225de85ed5e5a7ae991df8c778871289c60 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Sun, 13 Dec 2020 15:33:42 -0800 Subject: [PATCH 06/34] Move long multi-sentence parenthetical to a sub-bullet --- text/0000-cargo-binary-dependencies.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index 11b8deccc4c..69f76827e94 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -87,7 +87,8 @@ By default, `build-dependencies` are built for the host, while `dependencies` a Cargo provides the following environment variables to the crate being built: - `CARGO__DIR_`, where `` is the `type` of the artifact (uppercased) and `` is the package of the crate being depended on. (As with other Cargo environment variables, crate names are converted to uppercase, with dashes replaced by underscores.) This is the directory containing all the artifacts from the crate. -- `CARGO__FILE__`, where `` is the `type` of the artifact (transformed as above), `` is the package of the crate being depended on (transformed as above), and `` is the name of the artifact. (Note that `` is *not* modified in any way from the `name` specified in the crate supplying the artifact, or the crate name if not specified; for instance, it may be in lowercase, or contain dashes.) This is the full path to the artifact. +- `CARGO__FILE__`, where `` is the `type` of the artifact (transformed as above), `` is the package of the crate being depended on (transformed as above), and `` is the name of the artifact. This is the full path to the artifact. + - Note that `` is *not* modified in any way from the `name` specified in the crate supplying the artifact, or the crate name if not specified; for instance, it may be in lowercase, or contain dashes. - For the crate types `cdylib` and `staticlib` that can (currently) only build one artifact, cargo additionally supplies this variable with the `_` suffix omitted. For each kind of dependency, these variables are supplied to the same part of the build process that has access to that kind of dependency: From c6fde3ca4fe2a6301e687512d37e5858c7b07480 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Sun, 13 Dec 2020 15:35:43 -0800 Subject: [PATCH 07/34] Change the case for omitting the artifact suffix Rather than omitting it only for cdylib and staticlib, omit it whenever the name matches the crate name. --- text/0000-cargo-binary-dependencies.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index 69f76827e94..b2c75915000 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -89,7 +89,7 @@ Cargo provides the following environment variables to the crate being built: - `CARGO__DIR_`, where `` is the `type` of the artifact (uppercased) and `` is the package of the crate being depended on. (As with other Cargo environment variables, crate names are converted to uppercase, with dashes replaced by underscores.) This is the directory containing all the artifacts from the crate. - `CARGO__FILE__`, where `` is the `type` of the artifact (transformed as above), `` is the package of the crate being depended on (transformed as above), and `` is the name of the artifact. This is the full path to the artifact. - Note that `` is *not* modified in any way from the `name` specified in the crate supplying the artifact, or the crate name if not specified; for instance, it may be in lowercase, or contain dashes. - - For the crate types `cdylib` and `staticlib` that can (currently) only build one artifact, cargo additionally supplies this variable with the `_` suffix omitted. + - For convenience, if the artifact name is the same as the crate name, cargo additionally supplies a copy of this variable with the `_` suffix omitted. For each kind of dependency, these variables are supplied to the same part of the build process that has access to that kind of dependency: - For `build-dependencies`, these variables are supplied to the `build.rs` script, and can be accessed using `std::env::var_os`. (As with any OS file path, these may or may not be valid UTF-8.) From b06d9cb27b136e0bfd1bf6d4db0e0b86950dad85 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Sun, 13 Dec 2020 15:50:27 -0800 Subject: [PATCH 08/34] Discuss interaction with hypothetical automatic handling of staticlib/cdylib deps --- text/0000-cargo-binary-dependencies.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index b2c75915000..f7f2c858468 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -126,6 +126,11 @@ We could make information about artifact dependencies in `[dependencies]` availa We could install all binaries into a common binary directory with a well-known path under `$OUT_DIR`, and expect crates to use that directory, rather than passing in paths via environment variables. `npm` takes an approach like this. However, this would not allow dependencies on multiple distinct binaries with the same name, either provided by different crates or provided by the same crate built for different targets. Hardcoded paths would also reduce the flexibility of Cargo to change these paths in the future, such as to accommodate new features or extensions. +We could explicitly tag dependencies as artifact dependencies, rather than just treating dependencies with a `type` of `"cdylib"` or `"staticlib"` as artifact dependencies. This would allow for potential future dependencies of those types with automatic handling, such as automatically linking staticlib libraries. This would have several downsides, however: +- It reserves shorter syntax for a hypothesized dependency mechanism that doesn't exist yet (and may never be possible, in the case of `"cdylib"`, since Cargo can't guarantee runtime dependencies). +- Depending on the mechanism used to specify artifact dependencies, this may make it more difficult to have both a standard dependency and an artifact dependency on the same crate. This would be the case if we used `artifact = true`, for instance. We could potentially use syntax like `type = ["artifact:cdylib"]`, but such in-band flags introduce additional complexity. +This does not preclude adding support in Cargo for more "native" handling of cdylib/staticlib dependencies, if Cargo can provide a reasonable default; such a dependency could use a different syntax (e.g. `somedep = { version = "...", link = ["library-name"] }`). + # Prior art [prior-art]: #prior-art From 42ae02fb9a51fda9a8322993e5a9c8c20fd9dcb8 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Sun, 13 Dec 2020 15:51:15 -0800 Subject: [PATCH 09/34] Remove trailing space --- text/0000-cargo-binary-dependencies.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index f7f2c858468..66bc8f6917f 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -13,7 +13,7 @@ Allow Cargo packages to depend on `bin`, `cdylib`, and `staticlib` crates, and u There are many different possible use cases. -- [Testing the behavior of a binary](https://github.com/rust-lang/cargo/issues/4316#issuecomment-361641639). Currently, this requires invoking `cargo build` recursively, or running `cargo build` before running `cargo test`. +- [Testing the behavior of a binary](https://github.com/rust-lang/cargo/issues/4316#issuecomment-361641639). Currently, this requires invoking `cargo build` recursively, or running `cargo build` before running `cargo test`. - [Running a binary that depends on another](https://github.com/rust-lang/rustc-perf/tree/master/collector#how-to-benchmark-a-change-on-your-own-machine). Currently, this requires running `cargo build`, making it difficult to keep track of when the binary was rebuilt. The use case for `rustc-perf` is to have a main binary that acts as an 'executor', which executes `rustc` many times, and a smaller 'shim' which wraps `rustc` with additional environment variables and arguments. - [Building tools needed at build time](https://github.com/rust-lang/rust/pull/79540#unresolved-questions). Currently, this requires either splitting the tool into a library crate (if written in Rust), or telling the user to install the tool on the host and detecting the availability of it. This feature would allow building the necessary tool from source and then invoking it from a `build.rs` script later in the build. - Building and embedding binaries for another target, such as firmware, WebAssembly, or SPIR-V shaders. This feature would allow a versioned dependency on an appropriate crate providing the binary, and then embedding the binary (or a compressed or otherwise transformed version of it) into the final crate. For instance, a virtual machine could build its system firmware, or a WebAssembly runtime could build helper libraries. From 8faaab8e4c95521c765ebfa149891f7effdd3c9e Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Mon, 14 Dec 2020 14:09:24 -0800 Subject: [PATCH 10/34] Discuss alternative of omitting `"lib"` --- text/0000-cargo-binary-dependencies.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index 66bc8f6917f..bbcb1254eee 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -131,6 +131,8 @@ We could explicitly tag dependencies as artifact dependencies, rather than just - Depending on the mechanism used to specify artifact dependencies, this may make it more difficult to have both a standard dependency and an artifact dependency on the same crate. This would be the case if we used `artifact = true`, for instance. We could potentially use syntax like `type = ["artifact:cdylib"]`, but such in-band flags introduce additional complexity. This does not preclude adding support in Cargo for more "native" handling of cdylib/staticlib dependencies, if Cargo can provide a reasonable default; such a dependency could use a different syntax (e.g. `somedep = { version = "...", link = ["library-name"] }`). +Relatedly, we could omit the `"lib"` value for `type`. This would avoid potential conflation of dependency types (since `type = ["lib", "bin"]` expresses both a normal dependency on a library and an artifact dependency on a binary). This might serve as future-proofing. However, this could imply a future possibility of "artifact dependency" handling for native Rust libraries, which seems undesirable. Furthermore, this would substantially complicate the case of simultaneous dependencies on a crate's Rust library and other artifacts, a case which arises in several common use cases. + # Prior art [prior-art]: #prior-art From 6d95b60658bc0bc7781f838506462b5118a75a25 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Mon, 14 Dec 2020 14:22:23 -0800 Subject: [PATCH 11/34] Add note about handling of cross-compiled artifact dependencies rust-toolchain helps, but we may want first-class support in Cargo one day. --- text/0000-cargo-binary-dependencies.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index bbcb1254eee..935116966e2 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -152,3 +152,5 @@ Currently, there's no mechanism to obtain an environment variable's value at com In some cases, a crate may want to depend on a binary without unifying features or dependency versions with that binary. A future extension to this mechanism could allow cargo to build a binary crate in isolation, without attempting to do any unification. Just as a `-sys` crate can supply additional artifacts other than the built binary, this mechanism could potentially expand in the future to allow building artifacts other than the built binary, such as C-compatible include files, various types of interface definition or protocol definition files, or arbitrary data files. + +If a dependency has a specific `target` (other than the host or target), and the target is not available, cargo can only emit an error at build time that tells the user to install the target. Some projects may wish to use `rustup`'s support for `rust-toolchain` TOML files to specify targets they or their dependencies require. However, in the future, Cargo could have more native support for targets, either by downloading precompiled targets as rustup does, or by building support for those targets using `build-std` or equivalent. Integrating such support into Cargo would improve support for cross-compiled artifact dependencies. From 2e1aabf844c36c29fdcf43ce94e15a1e50eb8d01 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Mon, 14 Dec 2020 14:24:44 -0800 Subject: [PATCH 12/34] We may want `env_os!` or `env_path!` --- text/0000-cargo-binary-dependencies.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index 935116966e2..7200270e82e 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -147,7 +147,7 @@ How easily can Cargo handle a dependency with a different target specified? How # Future possibilities [future-possibilities]: #future-possibilities -Currently, there's no mechanism to obtain an environment variable's value at compile time if that value is not valid UTF-8. In the future, we may want an `env_os!` macro, analogous to `std::env::var_os`, which returns a `&'static OsStr` rather than a `&'static str`. This is already an issue for existing environment variables supplied to the build that contain file paths. +Currently, there's no mechanism to obtain an environment variable's value at compile time if that value is not valid UTF-8. In the future, we may want macros like `env_os!` or `env_path!`, which return a `&'static OsStr` or `&'static Path` respectively, rather than a `&'static str`. This is already an issue for existing environment variables supplied to the build that contain file paths. In some cases, a crate may want to depend on a binary without unifying features or dependency versions with that binary. A future extension to this mechanism could allow cargo to build a binary crate in isolation, without attempting to do any unification. From 32f1a50a64c648a91d188138c86483e03863555a Mon Sep 17 00:00:00 2001 From: Joshua Nelson Date: Wed, 16 Dec 2020 17:10:38 -0500 Subject: [PATCH 13/34] Remove unclear use case --- text/0000-cargo-binary-dependencies.md | 1 - 1 file changed, 1 deletion(-) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index 7200270e82e..024b6831c2c 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -13,7 +13,6 @@ Allow Cargo packages to depend on `bin`, `cdylib`, and `staticlib` crates, and u There are many different possible use cases. -- [Testing the behavior of a binary](https://github.com/rust-lang/cargo/issues/4316#issuecomment-361641639). Currently, this requires invoking `cargo build` recursively, or running `cargo build` before running `cargo test`. - [Running a binary that depends on another](https://github.com/rust-lang/rustc-perf/tree/master/collector#how-to-benchmark-a-change-on-your-own-machine). Currently, this requires running `cargo build`, making it difficult to keep track of when the binary was rebuilt. The use case for `rustc-perf` is to have a main binary that acts as an 'executor', which executes `rustc` many times, and a smaller 'shim' which wraps `rustc` with additional environment variables and arguments. - [Building tools needed at build time](https://github.com/rust-lang/rust/pull/79540#unresolved-questions). Currently, this requires either splitting the tool into a library crate (if written in Rust), or telling the user to install the tool on the host and detecting the availability of it. This feature would allow building the necessary tool from source and then invoking it from a `build.rs` script later in the build. - Building and embedding binaries for another target, such as firmware, WebAssembly, or SPIR-V shaders. This feature would allow a versioned dependency on an appropriate crate providing the binary, and then embedding the binary (or a compressed or otherwise transformed version of it) into the final crate. For instance, a virtual machine could build its system firmware, or a WebAssembly runtime could build helper libraries. From 983638e347315ec685ed122d6a053fe2094f4206 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 16 Dec 2020 14:52:28 -0800 Subject: [PATCH 14/34] Elaborate on the use case for depending on a binary --- text/0000-cargo-binary-dependencies.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index 024b6831c2c..9bc3ab6951d 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -13,7 +13,7 @@ Allow Cargo packages to depend on `bin`, `cdylib`, and `staticlib` crates, and u There are many different possible use cases. -- [Running a binary that depends on another](https://github.com/rust-lang/rustc-perf/tree/master/collector#how-to-benchmark-a-change-on-your-own-machine). Currently, this requires running `cargo build`, making it difficult to keep track of when the binary was rebuilt. The use case for `rustc-perf` is to have a main binary that acts as an 'executor', which executes `rustc` many times, and a smaller 'shim' which wraps `rustc` with additional environment variables and arguments. +- [Running a binary that depends on another](https://github.com/rust-lang/rustc-perf/tree/master/collector#how-to-benchmark-a-change-on-your-own-machine). Currently, this requires running `cargo build`, making it difficult to keep track of when the binary was rebuilt. The use case for `rustc-perf` is to have a main binary that acts as an 'executor', which executes `rustc` many times, and a smaller 'shim' which wraps `rustc` with additional environment variables and arguments. This RFC would allow splitting the shim into a separate crate, building that crate as an artifact dependency, and invoking it as part of the top-level crate. - [Building tools needed at build time](https://github.com/rust-lang/rust/pull/79540#unresolved-questions). Currently, this requires either splitting the tool into a library crate (if written in Rust), or telling the user to install the tool on the host and detecting the availability of it. This feature would allow building the necessary tool from source and then invoking it from a `build.rs` script later in the build. - Building and embedding binaries for another target, such as firmware, WebAssembly, or SPIR-V shaders. This feature would allow a versioned dependency on an appropriate crate providing the binary, and then embedding the binary (or a compressed or otherwise transformed version of it) into the final crate. For instance, a virtual machine could build its system firmware, or a WebAssembly runtime could build helper libraries. - Building and embedding a shared library for use at runtime. For instance, a binary could depend on a shared library used with [`LD_PRELOAD`](https://man7.org/linux/man-pages/man8/ld.so.8.html#ENVIRONMENT), or used in the style of the Linux kernel's [VDSO](https://man7.org/linux/man-pages/man7/vdso.7.html). From c547a5806780f6d2265489d5e826832a7bd6c78f Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 16 Dec 2020 14:55:09 -0800 Subject: [PATCH 15/34] Add a use case for tools needed for testing --- text/0000-cargo-binary-dependencies.md | 1 + 1 file changed, 1 insertion(+) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index 9bc3ab6951d..2344d620962 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -15,6 +15,7 @@ There are many different possible use cases. - [Running a binary that depends on another](https://github.com/rust-lang/rustc-perf/tree/master/collector#how-to-benchmark-a-change-on-your-own-machine). Currently, this requires running `cargo build`, making it difficult to keep track of when the binary was rebuilt. The use case for `rustc-perf` is to have a main binary that acts as an 'executor', which executes `rustc` many times, and a smaller 'shim' which wraps `rustc` with additional environment variables and arguments. This RFC would allow splitting the shim into a separate crate, building that crate as an artifact dependency, and invoking it as part of the top-level crate. - [Building tools needed at build time](https://github.com/rust-lang/rust/pull/79540#unresolved-questions). Currently, this requires either splitting the tool into a library crate (if written in Rust), or telling the user to install the tool on the host and detecting the availability of it. This feature would allow building the necessary tool from source and then invoking it from a `build.rs` script later in the build. +- Building tools needed for testing. A crate might build a binary or module designed to work in conjunction with some other tool. The test harness for the top-level crate could have an artifact dependency on the tool, and invoke that tool as part of the testsuite. - Building and embedding binaries for another target, such as firmware, WebAssembly, or SPIR-V shaders. This feature would allow a versioned dependency on an appropriate crate providing the binary, and then embedding the binary (or a compressed or otherwise transformed version of it) into the final crate. For instance, a virtual machine could build its system firmware, or a WebAssembly runtime could build helper libraries. - Building and embedding a shared library for use at runtime. For instance, a binary could depend on a shared library used with [`LD_PRELOAD`](https://man7.org/linux/man-pages/man8/ld.so.8.html#ENVIRONMENT), or used in the style of the Linux kernel's [VDSO](https://man7.org/linux/man-pages/man7/vdso.7.html). From 22c8a57db4f0d41d3ee2169e5742c0808eed3014 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 16 Dec 2020 14:59:19 -0800 Subject: [PATCH 16/34] Mention Swift packages and "products" Thanks to Eric Huss for the suggestion. --- text/0000-cargo-binary-dependencies.md | 1 + 1 file changed, 1 insertion(+) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index 2344d620962..faa600e11c4 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -137,6 +137,7 @@ Relatedly, we could omit the `"lib"` value for `type`. This would avoid potentia [prior-art]: #prior-art - Cargo already provides something similar to this for C library dependencies of -sys crates. A `-sys` crate can supply arbitrary artifact paths, for libraries, headers, and similar. Crates depending on the `-sys` crate can obtain those paths via environment variables supplied via Cargo, such as to compile other libraries using the same C library. This proposal provides a similar feature for other types of crates and libraries. +- The Swift package manager has a concept of ["products"](https://docs.swift.org/package-manager/PackageDescription/PackageDescription.html#product), which can be either libraries or executables. Expressing a dependency on a package allows you to make use of either the library or executable products of that package. - `make`, `cmake`, and many other build systems allow setting arbitrary goals as the dependencies of others. This allows building a binary and then running that binary in a rule that depends on that binary. # Unresolved questions From 9aebb5a5f2fce501420ff9a621d69e4970240e14 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 16 Dec 2020 15:04:56 -0800 Subject: [PATCH 17/34] Remove a link from the use cases; the text explains the use case better --- text/0000-cargo-binary-dependencies.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index faa600e11c4..515d17ed202 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -14,7 +14,7 @@ Allow Cargo packages to depend on `bin`, `cdylib`, and `staticlib` crates, and u There are many different possible use cases. - [Running a binary that depends on another](https://github.com/rust-lang/rustc-perf/tree/master/collector#how-to-benchmark-a-change-on-your-own-machine). Currently, this requires running `cargo build`, making it difficult to keep track of when the binary was rebuilt. The use case for `rustc-perf` is to have a main binary that acts as an 'executor', which executes `rustc` many times, and a smaller 'shim' which wraps `rustc` with additional environment variables and arguments. This RFC would allow splitting the shim into a separate crate, building that crate as an artifact dependency, and invoking it as part of the top-level crate. -- [Building tools needed at build time](https://github.com/rust-lang/rust/pull/79540#unresolved-questions). Currently, this requires either splitting the tool into a library crate (if written in Rust), or telling the user to install the tool on the host and detecting the availability of it. This feature would allow building the necessary tool from source and then invoking it from a `build.rs` script later in the build. +- Building tools needed at build time. Currently, this requires either splitting the tool into a library crate (if written in Rust), or telling the user to install the tool on the host and detecting the availability of it. This feature would allow building the necessary tool from source and then invoking it from a `build.rs` script later in the build. - Building tools needed for testing. A crate might build a binary or module designed to work in conjunction with some other tool. The test harness for the top-level crate could have an artifact dependency on the tool, and invoke that tool as part of the testsuite. - Building and embedding binaries for another target, such as firmware, WebAssembly, or SPIR-V shaders. This feature would allow a versioned dependency on an appropriate crate providing the binary, and then embedding the binary (or a compressed or otherwise transformed version of it) into the final crate. For instance, a virtual machine could build its system firmware, or a WebAssembly runtime could build helper libraries. - Building and embedding a shared library for use at runtime. For instance, a binary could depend on a shared library used with [`LD_PRELOAD`](https://man7.org/linux/man-pages/man8/ld.so.8.html#ENVIRONMENT), or used in the style of the Linux kernel's [VDSO](https://man7.org/linux/man-pages/man7/vdso.7.html). From c46b0686c7571488e26d986547eae07f1f47fe70 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 16 Dec 2020 15:16:53 -0800 Subject: [PATCH 18/34] Discuss alternative of recursive cargo or alternative build systems Document some of the drawbacks of this approach. --- text/0000-cargo-binary-dependencies.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index 515d17ed202..c705dcf2313 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -118,6 +118,13 @@ Adding this feature will make Cargo usable for many more use cases, which may mo # Rationale and alternatives [rationale-and-alternatives]: #rationale-and-alternatives +This RFC teaches Cargo to understand artifact dependencies. As an alternative, people writing crates with artifact dependencies could invoke `cargo` from `build.rs`, or could wrap the entire build in a separate build system that invokes Cargo multiple times. This would have many drawbacks, including: +- Cargo could not do dependency resolution in a unified way across dependencies, and thus could not help ensure consistency of dependency versions. This would break several use cases, without substantial additional complexity (e.g. vendored crates, or replacement of more of Cargo). +- Crates that have artifact dependencies would be less usable as dependencies themselves. Crates using a different build system would not work as Cargo dependencies at all. Crates using recursive invocations of cargo would introduce fragility, quirks, and limitations. +- Encouraging people to use build systems other than Cargo will remove the opportunity for Cargo and its defaults to set norms across the ecosystem. +- Crates manually implementing this via other build systems or recursive cargo invocations would make crates less uniform, and reduce consistency for users of Rust crates. +- Multiple/recursive invocations of Cargo will introduce challenges for Linux distributions, enterprises, and others who need to carefully manage/package/vendor dependencies. Crate metadata would not reflect its full dependencies. Manual invocations of cargo may handle dependency versioning inconsistently or not at all. Invocations of cargo may or may not pass through necessary options that were supplied to the top-level cargo invocation. Users may not have as many abilities to limit network access. + This RFC proposes supplying both the root directory and the path to each specific artifact. The path to specific artifacts is useful for accessing that specific artifact, and avoids needing target-specific knowledge about the names of executables (`.exe`) or libraries (`lib*.so`, `*.dll`, ...). The root directory is useful for `$PATH`, `$LD_LIBRARY_PATH`, and similar. Going from one to the other requires making assumptions. We believe there's value in supplying both. We could specify a `target = "host"` value to build for the host even for `[dependencies]` or `[dev-dependencies]` which would normally default to building for the target. If any use case arises for such a dependency, we can easily add that. From 442dcb0200333b736e399ebcc9fd01ed08cc19e6 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 16 Dec 2020 15:19:32 -0800 Subject: [PATCH 19/34] Clarify a mention of "binary" to cover other artifact types --- text/0000-cargo-binary-dependencies.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index c705dcf2313..9d072778b74 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -104,7 +104,7 @@ Cargo will unify features and versions across dependencies of all types, just as `type` may be a string, or a list of strings; in the latter case, this specifies a dependency on the crate with each of those types, and is equivalent to specifying multiple dependencies with different `type` values. For instance, you may specify a build dependency on both the binary and library of the same crate. You may also specify separate dependencies of different `type`s; for instance, you may have a build dependency on the binary of a crate and a dependency on the library of the same crate. -Cargo does not take the specified `type` values into account when resolving a crate's version; it will resolve the version as normal, and then produce an error if that version does not support all the specified `type` values. Similarly, Cargo will produce an error if that version does not build all the binaries required by the `bins` value. Removing a crate type or binary is a semver-incompatible change. (Any further semver requirements on the interface provided by a binary or library depend on the nature of the binary or library in question.) +Cargo does not take the specified `type` values into account when resolving a crate's version; it will resolve the version as normal, and then produce an error if that version does not support all the specified `type` values. Similarly, Cargo will produce an error if that version does not build all the binaries required by the `bins` value. Removing a crate type or an artifact is a semver-incompatible change. (Any further semver requirements on the interface provided by a binary or library depend on the nature of the binary or library in question.) Until this feature is stabilized, it will require specifying the nightly-only option `-Z bindeps` to `cargo`. If `cargo` encounters a binary dependency or artifact dependency and does not have this option specified, it will emit an error and immediately stop building. From 4e6f6ea8f962f4dfb6a0afabe22b8014b5ebe229 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 16 Dec 2020 15:38:36 -0800 Subject: [PATCH 20/34] Provide an example for the artifact name matching the crate name --- text/0000-cargo-binary-dependencies.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index 9d072778b74..b9e0ef5d428 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -89,7 +89,7 @@ Cargo provides the following environment variables to the crate being built: - `CARGO__DIR_`, where `` is the `type` of the artifact (uppercased) and `` is the package of the crate being depended on. (As with other Cargo environment variables, crate names are converted to uppercase, with dashes replaced by underscores.) This is the directory containing all the artifacts from the crate. - `CARGO__FILE__`, where `` is the `type` of the artifact (transformed as above), `` is the package of the crate being depended on (transformed as above), and `` is the name of the artifact. This is the full path to the artifact. - Note that `` is *not* modified in any way from the `name` specified in the crate supplying the artifact, or the crate name if not specified; for instance, it may be in lowercase, or contain dashes. - - For convenience, if the artifact name is the same as the crate name, cargo additionally supplies a copy of this variable with the `_` suffix omitted. + - For convenience, if the artifact name is the same as the crate name, cargo additionally supplies a copy of this variable with the `_` suffix omitted. For instance, if the `cmake` crate supplies a binary named `cmake`, Cargo supplies both `CARGO_BIN_FILE_CMAKE` and `CARGO_BIN_FILE_CMAKE_cmake`. For each kind of dependency, these variables are supplied to the same part of the build process that has access to that kind of dependency: - For `build-dependencies`, these variables are supplied to the `build.rs` script, and can be accessed using `std::env::var_os`. (As with any OS file path, these may or may not be valid UTF-8.) From 7f51fc44517e57668ffd91730d03d08abc4151a9 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 16 Dec 2020 15:51:48 -0800 Subject: [PATCH 21/34] Change `type` to `artifact` everywhere, and split out "lib" Avoid conflating artifact dependencies and non-artifact dependencies in a single `type` list. A dependency that would previously use `type = ["lib", "bin"]` will now use `artifact = "bin", lib = true`. --- text/0000-cargo-binary-dependencies.md | 39 +++++++++++++------------- 1 file changed, 19 insertions(+), 20 deletions(-) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index b9e0ef5d428..01acfd8b085 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -26,7 +26,7 @@ Cargo allows you to depend on binary or C ABI artifacts of another package; this ```toml [build-dependencies] -cmake = { version = "1.0", type = "bin" } +cmake = { version = "1.0", artifact = "bin" } ``` Cargo will build the `cmake` binary, then make it available to your `build.rs` through an environment variable: @@ -46,7 +46,7 @@ You can optionally specify specific binaries to depend on using `bins`: ```toml [build-dependencies] -cmake = { version = "1.0", type = "bin", bins = ["cmake"] } +cmake = { version = "1.0", artifact = "bin", bins = ["cmake"] } ``` If no binaries are specified, all the binaries in the package will be built and made available. @@ -62,16 +62,17 @@ const MY_PRELOAD_LIB: &[u8] = include_bytes!(env!("CARGO_CDYLIB_FILE_MYPRELOAD") Note that cargo cannot help you ensure these artifacts are available at runtime for an installed version of a binary; cargo can only supply these artifacts at build time. Runtime requirements for installed crates are out of scope for this change. -If you need to depend on multiple variants of a crate, such as both the binary and library of a crate, you can supply an array of strings for `type`: `type = ["bin", "lib"]`. +If you need to depend on multiple types of artifacts from a crate, such as both a binary and a cdylib from of a crate, you can supply an array of strings for `artifact`: `artifact = ["bin", "cdylib"]`. + +By default, a dependency with `artifact` specified will serve only as an artifact dependency, and will not serve as a normal Rust dependency, even if the dependency normally supplies a Rust library. If you need to depend on artifacts from a crate, and also express a normal Rust dependency on the same crate, you can add `lib = true` to the dependency; for instance: `cratename = { version = "1.2.3", lib = true, artifact = "bin" }`. (This applies to Rust `lib`, `rlib`, or `proc-macro` crates, all of which use the same `lib = true` option.) # Reference-level explanation [reference-level-explanation]: #reference-level-explanation -There are four `type`s available: -1. `"lib"`, the default -2. `"bin"`, a crate building one or more binaries -3. `"cdylib"`, a C-compatible dynamic library -4. `"staticlib"`, a C-compatible static library +There are three valid values for `artifact` available: +1. `"bin"`, a compiled binary, corresponding to a `[[bin]]` section in the dependency's manifest. +2. `"cdylib"`, a C-compatible dynamic library, corresponding to a `[lib]` section with `crate-type = "cdylib"` in the dependency's manifest. +3. `"staticlib"`, a C-compatible static library, corresponding to a `[lib]` section with `crate-type = "staticlib"` in the dependency's manifest. `"lib"` corresponds to all crates that can be depended on currently, including `lib`, `rlib`, and `proc-macro` libraries. @@ -86,10 +87,11 @@ By default, `build-dependencies` are built for the host, while `dependencies` a Cargo provides the following environment variables to the crate being built: -- `CARGO__DIR_`, where `` is the `type` of the artifact (uppercased) and `` is the package of the crate being depended on. (As with other Cargo environment variables, crate names are converted to uppercase, with dashes replaced by underscores.) This is the directory containing all the artifacts from the crate. -- `CARGO__FILE__`, where `` is the `type` of the artifact (transformed as above), `` is the package of the crate being depended on (transformed as above), and `` is the name of the artifact. This is the full path to the artifact. - - Note that `` is *not* modified in any way from the `name` specified in the crate supplying the artifact, or the crate name if not specified; for instance, it may be in lowercase, or contain dashes. - - For convenience, if the artifact name is the same as the crate name, cargo additionally supplies a copy of this variable with the `_` suffix omitted. For instance, if the `cmake` crate supplies a binary named `cmake`, Cargo supplies both `CARGO_BIN_FILE_CMAKE` and `CARGO_BIN_FILE_CMAKE_cmake`. +- `CARGO__DIR_`, where `` is the `artifact` specified for the dependency (uppercased) and `` is the name of the dependency. (As with other Cargo environment variables, dependency names are converted to uppercase, with dashes replaced by underscores.) This is the directory containing all the artifacts from the dependency. + - If your manifest [renames the dependency](https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#renaming-dependencies-in-cargotoml), `` corresponds to the name you specify, not the original package name. +- `CARGO__FILE__`, where `` is the `artifact` specified for the dependency (uppercased as above), `` is the package of the crate being depended on (transformed as above), and `` is the name of the artifact from the dependency. This is the full path to the artifact. + - Note that `` is *not* modified in any way from the `name` specified in the crate supplying the artifact, or the crate name if not specified; for instance, it may be in lowercase, or contain dashes. + - For convenience, if the artifact name matches the original package name, cargo additionally supplies a copy of this variable with the `_` suffix omitted. For instance, if the `cmake` crate supplies a binary named `cmake`, Cargo supplies both `CARGO_BIN_FILE_CMAKE` and `CARGO_BIN_FILE_CMAKE_cmake`. For each kind of dependency, these variables are supplied to the same part of the build process that has access to that kind of dependency: - For `build-dependencies`, these variables are supplied to the `build.rs` script, and can be accessed using `std::env::var_os`. (As with any OS file path, these may or may not be valid UTF-8.) @@ -100,11 +102,11 @@ For each kind of dependency, these variables are supplied to the same part of th Similar to features, if other crates in your dependencies also depend on the same binary crate, and request different binaries, Cargo will build the union of all binaries requested. -Cargo will unify features and versions across dependencies of all types, just as it does for multiple dependencies on the same crate throughout a dependency tree. +Cargo will unify features and versions across all kinds of dependencies, including artifact dependencies, just as it does for multiple dependencies on the same crate throughout a dependency tree. -`type` may be a string, or a list of strings; in the latter case, this specifies a dependency on the crate with each of those types, and is equivalent to specifying multiple dependencies with different `type` values. For instance, you may specify a build dependency on both the binary and library of the same crate. You may also specify separate dependencies of different `type`s; for instance, you may have a build dependency on the binary of a crate and a dependency on the library of the same crate. +`artifact` may be a string, or a list of strings; in the latter case, this specifies a dependency on the crate with each of those artifact types, and is equivalent to specifying multiple dependencies with different `artifact` values. For instance, you may specify a build dependency on both a binary and a cdylib from the same crate. You may also specify separate dependencies with different `artifact` values, as well as dependencies on the same crate without `artifact` specified; for instance, you may have a build dependency on the binary of a crate and a normal dependency on the Rust library of the same crate. -Cargo does not take the specified `type` values into account when resolving a crate's version; it will resolve the version as normal, and then produce an error if that version does not support all the specified `type` values. Similarly, Cargo will produce an error if that version does not build all the binaries required by the `bins` value. Removing a crate type or an artifact is a semver-incompatible change. (Any further semver requirements on the interface provided by a binary or library depend on the nature of the binary or library in question.) +Cargo does not take the specified `artifact` values into account when resolving a crate's version; it will resolve the version as normal, and then produce an error if that version does not support all the specified `artifact` values. Similarly, Cargo will produce an error if that version does not build all the binaries required by the `bins` value. Removing a crate type or an artifact is a semver-incompatible change. (Any further semver requirements on the interface provided by a binary or library depend on the nature of the binary or library in question.) Until this feature is stabilized, it will require specifying the nightly-only option `-Z bindeps` to `cargo`. If `cargo` encounters a binary dependency or artifact dependency and does not have this option specified, it will emit an error and immediately stop building. @@ -133,12 +135,9 @@ We could make information about artifact dependencies in `[dependencies]` availa We could install all binaries into a common binary directory with a well-known path under `$OUT_DIR`, and expect crates to use that directory, rather than passing in paths via environment variables. `npm` takes an approach like this. However, this would not allow dependencies on multiple distinct binaries with the same name, either provided by different crates or provided by the same crate built for different targets. Hardcoded paths would also reduce the flexibility of Cargo to change these paths in the future, such as to accommodate new features or extensions. -We could explicitly tag dependencies as artifact dependencies, rather than just treating dependencies with a `type` of `"cdylib"` or `"staticlib"` as artifact dependencies. This would allow for potential future dependencies of those types with automatic handling, such as automatically linking staticlib libraries. This would have several downsides, however: -- It reserves shorter syntax for a hypothesized dependency mechanism that doesn't exist yet (and may never be possible, in the case of `"cdylib"`, since Cargo can't guarantee runtime dependencies). -- Depending on the mechanism used to specify artifact dependencies, this may make it more difficult to have both a standard dependency and an artifact dependency on the same crate. This would be the case if we used `artifact = true`, for instance. We could potentially use syntax like `type = ["artifact:cdylib"]`, but such in-band flags introduce additional complexity. -This does not preclude adding support in Cargo for more "native" handling of cdylib/staticlib dependencies, if Cargo can provide a reasonable default; such a dependency could use a different syntax (e.g. `somedep = { version = "...", link = ["library-name"] }`). +This RFC does not preclude future support in Cargo for more "native" handling of cdylib/staticlib dependencies, if Cargo can provide a reasonable default; such a dependency could use a different syntax (e.g. `somedep = { version = "...", link = ["cdylib-name"] }`). -Relatedly, we could omit the `"lib"` value for `type`. This would avoid potential conflation of dependency types (since `type = ["lib", "bin"]` expresses both a normal dependency on a library and an artifact dependency on a binary). This might serve as future-proofing. However, this could imply a future possibility of "artifact dependency" handling for native Rust libraries, which seems undesirable. Furthermore, this would substantially complicate the case of simultaneous dependencies on a crate's Rust library and other artifacts, a case which arises in several common use cases. +In place of `lib = true`, we could rename `artifact` and have a `"lib"` or similar value for that field. This would provide simpler syntax (with a single list of dependency types), but could potentially conflate different dependency types (since a `"lib"` dependency type would express a normal dependency on a Rust library, while `"bin"` would express an artifact dependency). # Prior art [prior-art]: #prior-art From 58577cbc0daa1c7b9aa2d3316186bf6316db43b1 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 16 Dec 2020 16:03:11 -0800 Subject: [PATCH 22/34] Use "artifact dependency" consistently, for clarity --- text/0000-cargo-binary-dependencies.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index 01acfd8b085..0c750587733 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -108,7 +108,7 @@ Cargo will unify features and versions across all kinds of dependencies, includi Cargo does not take the specified `artifact` values into account when resolving a crate's version; it will resolve the version as normal, and then produce an error if that version does not support all the specified `artifact` values. Similarly, Cargo will produce an error if that version does not build all the binaries required by the `bins` value. Removing a crate type or an artifact is a semver-incompatible change. (Any further semver requirements on the interface provided by a binary or library depend on the nature of the binary or library in question.) -Until this feature is stabilized, it will require specifying the nightly-only option `-Z bindeps` to `cargo`. If `cargo` encounters a binary dependency or artifact dependency and does not have this option specified, it will emit an error and immediately stop building. +Until this feature is stabilized, it will require specifying the nightly-only option `-Z bindeps` to `cargo`. If `cargo` encounters an artifact dependency and does not have this option specified, it will emit an error and immediately stop building. # Drawbacks [drawbacks]: #drawbacks From ff2b254deff28ebefd87bbee440bcf6f6aa4f595 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 16 Dec 2020 16:03:21 -0800 Subject: [PATCH 23/34] Drop `bins` in favor of `artifact = "bin:name"` --- text/0000-cargo-binary-dependencies.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index 0c750587733..b2ed8c67e04 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -42,11 +42,13 @@ fn main() { } ``` -You can optionally specify specific binaries to depend on using `bins`: +If you need to depend on multiple types of artifacts from a crate, such as both a binary and a cdylib from of a crate, you can supply an array of strings for `artifact`: `artifact = ["bin", "cdylib"]`. + +You can optionally depend on specific binary artifacts from a crate using `bin:name`: ```toml [build-dependencies] -cmake = { version = "1.0", artifact = "bin", bins = ["cmake"] } +somedep = { version = "1.0", artifact = ["bin:somebinary", "bin:anotherbinary"] } ``` If no binaries are specified, all the binaries in the package will be built and made available. @@ -62,8 +64,6 @@ const MY_PRELOAD_LIB: &[u8] = include_bytes!(env!("CARGO_CDYLIB_FILE_MYPRELOAD") Note that cargo cannot help you ensure these artifacts are available at runtime for an installed version of a binary; cargo can only supply these artifacts at build time. Runtime requirements for installed crates are out of scope for this change. -If you need to depend on multiple types of artifacts from a crate, such as both a binary and a cdylib from of a crate, you can supply an array of strings for `artifact`: `artifact = ["bin", "cdylib"]`. - By default, a dependency with `artifact` specified will serve only as an artifact dependency, and will not serve as a normal Rust dependency, even if the dependency normally supplies a Rust library. If you need to depend on artifacts from a crate, and also express a normal Rust dependency on the same crate, you can add `lib = true` to the dependency; for instance: `cratename = { version = "1.2.3", lib = true, artifact = "bin" }`. (This applies to Rust `lib`, `rlib`, or `proc-macro` crates, all of which use the same `lib = true` option.) # Reference-level explanation @@ -106,7 +106,7 @@ Cargo will unify features and versions across all kinds of dependencies, includi `artifact` may be a string, or a list of strings; in the latter case, this specifies a dependency on the crate with each of those artifact types, and is equivalent to specifying multiple dependencies with different `artifact` values. For instance, you may specify a build dependency on both a binary and a cdylib from the same crate. You may also specify separate dependencies with different `artifact` values, as well as dependencies on the same crate without `artifact` specified; for instance, you may have a build dependency on the binary of a crate and a normal dependency on the Rust library of the same crate. -Cargo does not take the specified `artifact` values into account when resolving a crate's version; it will resolve the version as normal, and then produce an error if that version does not support all the specified `artifact` values. Similarly, Cargo will produce an error if that version does not build all the binaries required by the `bins` value. Removing a crate type or an artifact is a semver-incompatible change. (Any further semver requirements on the interface provided by a binary or library depend on the nature of the binary or library in question.) +Cargo does not take the specified `artifact` values into account when resolving a crate's version; it will resolve the version as normal, and then produce an error if that version does not support all the specified `artifact` values. Similarly, Cargo will produce an error if that version does not build all the binary artifacts required by `"bin:name"` values. Removing a crate type or an artifact is a semver-incompatible change. (Any further semver requirements on the interface provided by a binary or library depend on the nature of the binary or library in question.) Until this feature is stabilized, it will require specifying the nightly-only option `-Z bindeps` to `cargo`. If `cargo` encounters an artifact dependency and does not have this option specified, it will emit an error and immediately stop building. From 82baedf01de8caa50bb64f9be38c347dd998db1a Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 16 Dec 2020 20:55:48 -0800 Subject: [PATCH 24/34] Elaborate on the LD_PRELOAD and VDSO use cases --- text/0000-cargo-binary-dependencies.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index b2ed8c67e04..85890b85b73 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -17,7 +17,7 @@ There are many different possible use cases. - Building tools needed at build time. Currently, this requires either splitting the tool into a library crate (if written in Rust), or telling the user to install the tool on the host and detecting the availability of it. This feature would allow building the necessary tool from source and then invoking it from a `build.rs` script later in the build. - Building tools needed for testing. A crate might build a binary or module designed to work in conjunction with some other tool. The test harness for the top-level crate could have an artifact dependency on the tool, and invoke that tool as part of the testsuite. - Building and embedding binaries for another target, such as firmware, WebAssembly, or SPIR-V shaders. This feature would allow a versioned dependency on an appropriate crate providing the binary, and then embedding the binary (or a compressed or otherwise transformed version of it) into the final crate. For instance, a virtual machine could build its system firmware, or a WebAssembly runtime could build helper libraries. -- Building and embedding a shared library for use at runtime. For instance, a binary could depend on a shared library used with [`LD_PRELOAD`](https://man7.org/linux/man-pages/man8/ld.so.8.html#ENVIRONMENT), or used in the style of the Linux kernel's [VDSO](https://man7.org/linux/man-pages/man7/vdso.7.html). +- Building and embedding a shared library for use at runtime. For instance, a tool for profiling or debugging other programs could depend on a shared library that it loads into those programs using [`LD_PRELOAD`](https://man7.org/linux/man-pages/man8/ld.so.8.html#ENVIRONMENT). Or, an operating system kernel could build a userspace API library that it loads into userspace applications running on it, in the style of the Linux kernel's [VDSO](https://man7.org/linux/man-pages/man7/vdso.7.html). # Guide-level explanation [guide-level-explanation]: #guide-level-explanation From 0f3933bff73ef4f66793f3f1c4ff950823e7f57c Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 16 Dec 2020 21:12:59 -0800 Subject: [PATCH 25/34] Minor grammar tweak --- text/0000-cargo-binary-dependencies.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index 85890b85b73..8ec60ec2113 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -22,7 +22,7 @@ There are many different possible use cases. # Guide-level explanation [guide-level-explanation]: #guide-level-explanation -Cargo allows you to depend on binary or C ABI artifacts of another package; this is known as a "binary dependency" or "artifact dependency". For example, you can depend on the `cmake` binary in your `build.rs` like so: +Cargo allows you to depend on binary or C ABI artifacts of another package; this is known as a "binary dependency" or "artifact dependency". For example, you can depend on the `cmake` binary in your `build.rs` like this: ```toml [build-dependencies] From 6c844368b58618cf5636e09b9c270f8fd2def792 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 16 Dec 2020 21:15:09 -0800 Subject: [PATCH 26/34] Clarify scope statement --- text/0000-cargo-binary-dependencies.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index 8ec60ec2113..73c4ea5e92c 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -62,7 +62,7 @@ Cargo also allows depending on `cdylib` or `staticlib` artifacts. For example, y const MY_PRELOAD_LIB: &[u8] = include_bytes!(env!("CARGO_CDYLIB_FILE_MYPRELOAD")); ``` -Note that cargo cannot help you ensure these artifacts are available at runtime for an installed version of a binary; cargo can only supply these artifacts at build time. Runtime requirements for installed crates are out of scope for this change. +Note that cargo only supplies these dependencies when building your crate. If your program or library requires artifacts at runtime, you will still need to handle that yourself by some other means. Runtime requirements for installed crates are out of scope for this change. By default, a dependency with `artifact` specified will serve only as an artifact dependency, and will not serve as a normal Rust dependency, even if the dependency normally supplies a Rust library. If you need to depend on artifacts from a crate, and also express a normal Rust dependency on the same crate, you can add `lib = true` to the dependency; for instance: `cratename = { version = "1.2.3", lib = true, artifact = "bin" }`. (This applies to Rust `lib`, `rlib`, or `proc-macro` crates, all of which use the same `lib = true` option.) From eaf80fb5d51f55e7e6fca9ed3a29f54c66ec67be Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 16 Dec 2020 21:20:41 -0800 Subject: [PATCH 27/34] Add a cross-reference from drawbacks to alternatives --- text/0000-cargo-binary-dependencies.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index 73c4ea5e92c..5eaf1383357 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -113,7 +113,7 @@ Until this feature is stabilized, it will require specifying the nightly-only op # Drawbacks [drawbacks]: #drawbacks -Some of the motivating use cases have alternative solutions, such as extracting a library from a tool written in Rust, and making the tool a thin wrapper around the library. Making this change may potentially reduce the motivation to extract such libraries. However, many of the other use cases do not currently have any solutions, and extracted libraries have additional value even after this feature becomes available, so we don't see this as a reason to avoid introducing this feature. +Some of the motivating use cases have alternative solutions, such as extracting a library from a tool written in Rust, and making the tool a thin wrapper around the library. Making this change may potentially reduce the motivation to extract such libraries. However, many of the other use cases do not currently have any solutions available (other than using an alternative build system, per the alternatives section), and extracted libraries have additional value even after this feature becomes available, so we don't see this as a reason to avoid introducing this feature. Adding this feature will make Cargo usable for many more use cases, which may motivate people to use Cargo in more places and stretch it even further; this may, in turn, generate more support and more feature requests. From be85559f5364257a146cff22eb9050c1c7959409 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 16 Dec 2020 21:55:02 -0800 Subject: [PATCH 28/34] Expand an example --- text/0000-cargo-binary-dependencies.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index 5eaf1383357..d6ef8608fa5 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -57,6 +57,11 @@ You can obtain the directory containing all binaries built by the `cmake` crate Cargo also allows depending on `cdylib` or `staticlib` artifacts. For example, you can embed a dynamic library in your binary: +```toml +[dependencies] +mypreload = { version = "1.2.3", artifact = "cdylib" } +``` + ```rust // main.rs const MY_PRELOAD_LIB: &[u8] = include_bytes!(env!("CARGO_CDYLIB_FILE_MYPRELOAD")); From 93c3094ba8cd13a963740e05dc0792240c57ab6c Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 16 Dec 2020 22:03:10 -0800 Subject: [PATCH 29/34] Adjust a missed `` to `` to match the rest --- text/0000-cargo-binary-dependencies.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index d6ef8608fa5..3d927ccf9a4 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -94,7 +94,7 @@ Cargo provides the following environment variables to the crate being built: - `CARGO__DIR_`, where `` is the `artifact` specified for the dependency (uppercased) and `` is the name of the dependency. (As with other Cargo environment variables, dependency names are converted to uppercase, with dashes replaced by underscores.) This is the directory containing all the artifacts from the dependency. - If your manifest [renames the dependency](https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#renaming-dependencies-in-cargotoml), `` corresponds to the name you specify, not the original package name. -- `CARGO__FILE__`, where `` is the `artifact` specified for the dependency (uppercased as above), `` is the package of the crate being depended on (transformed as above), and `` is the name of the artifact from the dependency. This is the full path to the artifact. +- `CARGO__FILE__`, where `` is the `artifact` specified for the dependency (uppercased as above), `` is the package of the crate being depended on (transformed as above), and `` is the name of the artifact from the dependency. This is the full path to the artifact. - Note that `` is *not* modified in any way from the `name` specified in the crate supplying the artifact, or the crate name if not specified; for instance, it may be in lowercase, or contain dashes. - For convenience, if the artifact name matches the original package name, cargo additionally supplies a copy of this variable with the `_` suffix omitted. For instance, if the `cmake` crate supplies a binary named `cmake`, Cargo supplies both `CARGO_BIN_FILE_CMAKE` and `CARGO_BIN_FILE_CMAKE_cmake`. From c9470ad3d7f36c2889fe1d11e1a39c4098d71abc Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 16 Dec 2020 22:45:41 -0800 Subject: [PATCH 30/34] Document alternative syntaxes for specifying dependencies on specific binaries --- text/0000-cargo-binary-dependencies.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index 3d927ccf9a4..5beb7c48348 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -144,6 +144,10 @@ This RFC does not preclude future support in Cargo for more "native" handling of In place of `lib = true`, we could rename `artifact` and have a `"lib"` or similar value for that field. This would provide simpler syntax (with a single list of dependency types), but could potentially conflate different dependency types (since a `"lib"` dependency type would express a normal dependency on a Rust library, while `"bin"` would express an artifact dependency). +Instead of `artifact = ["bin:binary-name", "bin:another-binary"]` to specify dependencies on specific binaries, we could use a separate field `bins = ["binary-name", "another-binary"]`. This seems unnecessarily verbose, and separates the indication of an artifact dependency from the list of binaries. + +As another alternative to specify dependencies on specific binaries, we could use table-based structures, such as: `artifact = [{bin = ["binary-name", "another-binary"]}, "cdylib"]`. This would avoid parsing values like `bin:binary-name`, but it seems excessively complex and excessively nested. Other variations on this theme seem similarly complex. The proposed syntax feels like the right balance. + # Prior art [prior-art]: #prior-art From a42feade82d12ad55070747fc8d143e01408b4fc Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 16 Dec 2020 23:19:02 -0800 Subject: [PATCH 31/34] Propose specific artifact paths for the implementation --- text/0000-cargo-binary-dependencies.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index 5beb7c48348..44d181bc76e 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -115,6 +115,8 @@ Cargo does not take the specified `artifact` values into account when resolving Until this feature is stabilized, it will require specifying the nightly-only option `-Z bindeps` to `cargo`. If `cargo` encounters an artifact dependency and does not have this option specified, it will emit an error and immediately stop building. +The placement of artifact directories is an implementation detail of Cargo, and subject to change. The proposed implementation will place the artifact directory for each crate in `target//artifact/-/`, where `` is the target triple the artifact dependency is built for (which may be the target triple of the host), `` is the name of the crate, `` is the usual hash that Cargo appends to crate-related file and directory names to ensure that changing properties (such as features) that affect the build of the crate will build into different paths, and `` is the artifact type (`bin`, `cdylib`, or `staticlib`). + # Drawbacks [drawbacks]: #drawbacks From 98e1476a1851803f538985cece17f9c15d01ca5f Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 16 Dec 2020 23:25:48 -0800 Subject: [PATCH 32/34] Document the profile settings used to build artifact dependencies --- text/0000-cargo-binary-dependencies.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index 44d181bc76e..de62863f18d 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -113,6 +113,8 @@ Cargo will unify features and versions across all kinds of dependencies, includi Cargo does not take the specified `artifact` values into account when resolving a crate's version; it will resolve the version as normal, and then produce an error if that version does not support all the specified `artifact` values. Similarly, Cargo will produce an error if that version does not build all the binary artifacts required by `"bin:name"` values. Removing a crate type or an artifact is a semver-incompatible change. (Any further semver requirements on the interface provided by a binary or library depend on the nature of the binary or library in question.) +As with other kinds of dependencies, you can specify profile settings used to build artifact dependencies using [overrides](https://doc.rust-lang.org/cargo/reference/profiles.html#overrides). If not overridden, artifact dependencies in `build-dependencies` compiled for the host will build using the [`build-override` settings](https://doc.rust-lang.org/cargo/reference/profiles.html#build-dependencies), and all other artifact dependencies will inherit the same profile settings being used to build the crate depending on them. + Until this feature is stabilized, it will require specifying the nightly-only option `-Z bindeps` to `cargo`. If `cargo` encounters an artifact dependency and does not have this option specified, it will emit an error and immediately stop building. The placement of artifact directories is an implementation detail of Cargo, and subject to change. The proposed implementation will place the artifact directory for each crate in `target//artifact/-/`, where `` is the target triple the artifact dependency is built for (which may be the target triple of the host), `` is the name of the crate, `` is the usual hash that Cargo appends to crate-related file and directory names to ensure that changing properties (such as features) that affect the build of the crate will build into different paths, and `` is the artifact type (`bin`, `cdylib`, or `staticlib`). From 2ddc224f80e8dffd5b9e3b0b934fa9b5e6b6de45 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Thu, 24 Dec 2020 12:19:04 -0800 Subject: [PATCH 33/34] Document what happens when building for multiple targets at once --- text/0000-cargo-binary-dependencies.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/text/0000-cargo-binary-dependencies.md b/text/0000-cargo-binary-dependencies.md index de62863f18d..ab045bfc1b5 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/0000-cargo-binary-dependencies.md @@ -119,6 +119,8 @@ Until this feature is stabilized, it will require specifying the nightly-only op The placement of artifact directories is an implementation detail of Cargo, and subject to change. The proposed implementation will place the artifact directory for each crate in `target//artifact/-/`, where `` is the target triple the artifact dependency is built for (which may be the target triple of the host), `` is the name of the crate, `` is the usual hash that Cargo appends to crate-related file and directory names to ensure that changing properties (such as features) that affect the build of the crate will build into different paths, and `` is the artifact type (`bin`, `cdylib`, or `staticlib`). +If Cargo needs to build a crate for multiple targets, and that crate has an artifact dependency with `target="target"`, Cargo will build the artifact dependency for each target and supply it to the corresponding build of the depending crate. + # Drawbacks [drawbacks]: #drawbacks From 278de5af529bab534b3420b54375c457632661f4 Mon Sep 17 00:00:00 2001 From: Eric Huss Date: Sat, 23 Jan 2021 07:33:10 -0800 Subject: [PATCH 34/34] Merging RFC 3028 --- ...binary-dependencies.md => 3028-cargo-binary-dependencies.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename text/{0000-cargo-binary-dependencies.md => 3028-cargo-binary-dependencies.md} (99%) diff --git a/text/0000-cargo-binary-dependencies.md b/text/3028-cargo-binary-dependencies.md similarity index 99% rename from text/0000-cargo-binary-dependencies.md rename to text/3028-cargo-binary-dependencies.md index ab045bfc1b5..bdf272b1ada 100644 --- a/text/0000-cargo-binary-dependencies.md +++ b/text/3028-cargo-binary-dependencies.md @@ -1,7 +1,7 @@ - Feature Name: (`bindeps`) - Start Date: 2020-11-30 - RFC PR: [rust-lang/rfcs#3028](https://github.com/rust-lang/rfcs/pull/3028) -- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000) +- Tracking Issue: [rust-lang/cargo#9096](https://github.com/rust-lang/cargo/issues/9096) # Summary [summary]: #summary