Tracking issue for -Z features=host_dep #7915

ehuss · 2020-02-21T19:02:39Z

Implementation: #7820
Documentation: https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#features

Summary
-Z features=host_dep causes the feature resolver to prevent features enabled on build dependencies or proc-macros from being enabled for normal dependencies. For example:

[dependencies]
log = "0.4"

[build-dependencies]
log = {version = "0.4", features=['std']}

When building the build script, the log crate will be built with the std feature. When building the library of your package, it will not enable the feature.

Unresolved issues

Update cargo metadata
Does this cause existing projects to fail?
- Projects can add missing features as a backwards-compatible workaround. However, if it causes problems for too many projects, we may need to find some way to opt-in.
Does this significantly increase build times?
- This can cause common dependencies to be built multiple times, when they were previously shared (when --target is not used). Projects can add features to re-unify if they want, but it may not be clear or obvious what to add and when. We may need something that identifies these duplicate dependencies, and give suggestions on how to unify them. Or, we may need to make this opt-in somehow so the user needs to make this an explicit decision.
Does the feature resolver need to support decoupling of shared proc-macro dependencies?
- Currently this isn't possible because Cargo does not have the necessary information to know which dependencies are proc-macros during resolution time. Either we would need to add the proc-macro status to the index, or try to do feature resolution while dependencies are downloaded. The latter is very difficult due to the way downloading works. The former isn't too hard. Nobody has asked for this ability, so it is unclear how important it is. Presumably that is because proc-macros are relatively new, and tend to only use proc-macro-specific dependencies like syn/quote.
- Implemented in Re-implement proc-macro feature decoupling. #8028.

The text was updated successfully, but these errors were encountered:

Amanieu · 2020-02-24T17:10:04Z

Does the feature resolver need to support decoupling of shared proc-macro dependencies?

Yes please. I've encountered an issue related to this already when a proc-macro enabled many features in the rand crate which caused my build to fail since my target does not have std. See rust-lang/hashbrown#124 for details.

valarauca · 2020-02-24T23:26:44Z

Does the feature resolver need to support decoupling of shared proc-macro dependencies?

Seeing as #4866 was about proc-macro's it does seem to be in demand. Also could #4866 be re-opened as #7820 doesn't address it?

tarcieri · 2020-02-27T04:23:00Z

UPDATE: never mind, this is a red herring. the proc macro itself was referencing std, because I forgot to disable its std feature 😅. With the std feature it works, but I can see how this could come up in dependency-laden proc macros that activate the std feature in dependencies shared by the proc macro.

I just ran into the proc macro problem trying to use displaydoc in a no_std project:

$ cargo build -Z features=build_dep --no-default-features --target thumbv7em-none-eabihf --release
error[E0433]: failed to resolve: use of undeclared type or module `std`
 --> rust/src/error.rs:7:30
  |
7 | #[derive(Copy, Clone, Debug, Display, Eq, PartialEq)]
  |                              ^^^^^^^ use of undeclared type or module `std`

It would definitely be super duper handy to be able to use proc macros that link against std in no_std targets, and also to ensure there's some open issue tracking it until it's fixed.

Seeing as #4866 was about proc-macro...

@valarauca it looks like some people discussed proc macros in the comments, but #4866 itself was about build-dependencies

Perhaps it's worth opening a new issue?

ia0 · 2020-02-29T17:46:31Z

I would also be interested in having -Z features=build_dep work with proc-macro. I would use it for the data-encoding-macro crate.

Add proc-macro to index, and new feature resolver. This adds the "pm" field to the index so that Cargo can detect which packages contain a proc-macro without downloading the package. The second commit builds on that to support proc-macros in the new "de-unification" of the new feature resolver. This prevents dependencies shared between proc-macros and other dependency kinds from having features unified. cc #7915

ehuss · 2020-03-27T13:57:51Z

proc-macro support has been added in #8028, and should be in the latest nightly (2020-03-27). proc-macros and build dependencies are unified (since they are all "host" artifacts), but shouldn't affect anything else. Also, the option has been renamed to -Zfeatures=host_dep to reflect this change (the old build_dep option is an alias). I'd appreciate it if people could take it for a test drive and let me know how it works. This resulted in a major change in how crates are downloaded, so any testing is appreciated.

This is a complete overhaul of the way platform-specific dependencies are handled. Based on my experimentation and reading the Cargo source code, I believe that this is now correct. Doing so also required feature graph construction to be updated, so the feature graph construction is ready for the new feature resolver, and is now platform-sensitive as well. The APIs for the feature graph are still to come, but making the data model be full-fidelity allows for both the current and new feature resolvers to be implemented. For more about the new feature resolver, see: * https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#features * rust-lang/cargo#7914 * rust-lang/cargo#7915 * rust-lang/cargo#7916

This is a complete overhaul of the way platform-specific dependencies are handled. Based on my experimentation and reading the Cargo source code, I believe that this is now correct. Doing so also required feature graph construction to be updated, so the feature graph construction is ready for the new feature resolver, and is now platform-sensitive as well. The APIs for the feature graph are still to come, but making the data model be full-fidelity allows for both the current and new feature resolvers to be implemented. Now that target logic is internalized in guppy, cargo-guppy no longer needs to do its own evaluation. For more about the new feature resolver, see: * https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#features * rust-lang/cargo#7914 * rust-lang/cargo#7915 * rust-lang/cargo#7916

valarauca · 2020-03-30T19:31:25Z

I created this "project" as a slightly contrived example.

real_project configures fake-dep (via fake-dev-dep) to panic. see: link
real_project requires fake-dev-dep to panic . The real_project/build.rs file will exit(1) if fake-dep does not panic). see: link
real_project by default expects fake-dep to not panic within its test link

Results:

On stable running cargo test within real_project results in test failure (test::crate_worked_successfully panicked at hello world)
On nightly-2020-03-18 running cargo test within real_project results in a test failure (test::crate_worked_successfully panicked at hello world)
On 2020-03-27 running cargo test -Zfeatures=host_dep the test passes!

I hope this is a valid tests. The idea is build-dependencies and dependencies should have different, and conflicting features.

The results seem positive. Thanks for the work on this!

This is a complete overhaul of the way platform-specific dependencies are handled. Based on my experimentation and reading the Cargo source code, I believe that this is now correct. Doing so also required feature graph construction to be updated, so the feature graph construction is ready for the new feature resolver, and is now platform-sensitive as well. The APIs for the feature graph are still to come, but making the data model be full-fidelity allows for both the current and new feature resolvers to be implemented. Now that target logic is internalized in guppy, cargo-guppy no longer needs to do its own evaluation. For more about the new feature resolver, see: * https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#features * rust-lang/cargo#7914 * rust-lang/cargo#7915 * rust-lang/cargo#7916

BrandonArp · 2020-07-18T09:53:56Z

This feature is working great for an embedded project I'm working on. Like others, I was hitting no_std and bindgen conflict. It's really great to see this feature! Thanks!

blaind · 2020-08-29T14:03:42Z

Is there a way to enable this via .cargo/config or Cargo.toml, or do I need to pass -Z features=host_dep manually to Cargo every time?

Probably a bit offtopic, but I opened #8127 to propose a general solution to this problem.

Seems like this has been merged. Had to search a bit how to use it, so for anyone else who's using this - here's a way to enable the flag in .cargo/config

[unstable]
features = ["host_dep"]

Updates to use the `group` crate. See: RustCrypto/traits#287. This crate has a hard `rand_core` dependency so this commit gets rid of the `rand` features across the board and makes them mandatory. (Even if we don't end up shipping the `group` crate this release, that's probably for the best to keep the number of features down) This commit additionally splits out `no_std` build testing into `tests/*_no_std` Cargo projects. This is a workaround until the Cargo resolver is fixed upstream: rust-lang/cargo#7915 rust-lang/cargo#7916

h4x3rotab · 2020-09-11T05:11:21Z

When stable?

ctron · 2020-10-13T16:10:37Z

Works great. Would like to see this too.

jonasbb · 2021-01-08T14:59:54Z

Hi @ehuss,

three days ago (2021-01-05) I noticed some unstable CI build where I use -Zfeatures=dev_dep,host_dep. I could reproduce it locally only using host_dep, which is why I post here. I never noticed it without the feature.

Error Log

error[E0053]: method `from_list` has an incompatible type for trait
Error:    --> serde_with_macros\src\lib.rs:431:14
    |
431 |     #[derive(FromMeta, Debug)]
    |              ^^^^^^^^ expected enum `syn::attr::NestedMeta`, found enum `NestedMeta`
    |
    = note: expected fn pointer `fn(&[syn::attr::NestedMeta]) -> std::result::Result<_, _>`
               found fn pointer `fn(&[NestedMeta]) -> std::result::Result<_, _>`
    = note: perhaps two different versions of crate `syn` are being used?
    = note: this error originates in a derive macro (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0053]: method `from_field` has an incompatible type for trait
Error:    --> serde_with_macros\src\lib.rs:465:14
    |
465 |     #[derive(FromField, Debug)]
    |              ^^^^^^^^^ expected struct `syn::data::Field`, found struct `syn::Field`
    |
    = note: expected fn pointer `fn(&syn::data::Field) -> std::result::Result<_, _>`
               found fn pointer `fn(&syn::Field) -> std::result::Result<_, _>`
    = note: perhaps two different versions of crate `syn` are being used?
    = note: this error originates in a derive macro (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0053]: method `from_field` has an incompatible type for trait
Error:    --> serde_with_macros\src\lib.rs:482:14
    |
482 |     #[derive(FromField, Debug)]
    |              ^^^^^^^^^ expected struct `syn::data::Field`, found struct `syn::Field`
    |
    = note: expected fn pointer `fn(&syn::data::Field) -> std::result::Result<_, _>`
               found fn pointer `fn(&syn::Field) -> std::result::Result<_, _>`
    = note: perhaps two different versions of crate `syn` are being used?
    = note: this error originates in a derive macro (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0053]: method `from_derive_input` has an incompatible type for trait
Error:   --> serde_with_macros\src\utils.rs:26:10
   |
26 | #[derive(FromDeriveInput, Debug)]
   |          ^^^^^^^^^^^^^^^ expected struct `syn::derive::DeriveInput`, found struct `DeriveInput`
   |
   = note: expected fn pointer `fn(&syn::derive::DeriveInput) -> std::result::Result<_, _>`
              found fn pointer `fn(&DeriveInput) -> std::result::Result<_, _>`
   = note: perhaps two different versions of crate `syn` are being used?
   = note: this error originates in a derive macro (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0308]: mismatched types
Error:    --> serde_with_macros\src\lib.rs:438:68
    |
438 |     let container_options = match SerdeContainerOptions::from_list(&args) {
    |                                                                    ^^^^^ expected slice, found struct `Vec`
    |
    = note: expected reference `&[syn::attr::NestedMeta]`
               found reference `&Vec<NestedMeta>`

error[E0308]: mismatched types
Error:    --> serde_with_macros\src\lib.rs:431:14
    |
431 |     #[derive(FromMeta, Debug)]
    |              ^^^^^^^^ expected enum `syn::attr::Meta`, found enum `syn::Meta`
    |
    = note: expected reference `&syn::attr::Meta`
               found reference `&syn::Meta`
    = note: perhaps two different versions of crate `syn` are being used?
    = note: this error originates in a derive macro (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0308]: mismatched types
Error:    --> serde_with_macros\src\lib.rs:499:55
    |
499 |     let serde_as_options = SerdeAsOptions::from_field(field)?;
    |                                                       ^^^^^ expected struct `syn::data::Field`, found struct `syn::Field`
    |
    = note:      expected reference `&syn::data::Field`
            found mutable reference `&mut syn::Field`
    = note: perhaps two different versions of crate `syn` are being used?

error[E0308]: mismatched types
Error:    --> serde_with_macros\src\lib.rs:500:59
    |
500 |     let serde_with_options = SerdeWithOptions::from_field(field)?;
    |                                                           ^^^^^ expected struct `syn::data::Field`, found struct `syn::Field`
    |
    = note:      expected reference `&syn::data::Field`
            found mutable reference `&mut syn::Field`
    = note: perhaps two different versions of crate `syn` are being used?

error[E0308]: mismatched types
Error:    --> serde_with_macros\src\lib.rs:465:14
    |
465 |     #[derive(FromField, Debug)]
    |              ^^^^^^^^^ expected struct `syn::attr::Attribute`, found struct `Attribute`
    |
    = note: expected reference `&syn::attr::Attribute`
               found reference `&Attribute`
    = note: perhaps two different versions of crate `syn` are being used?
    = note: this error originates in a derive macro (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0308]: mismatched types
Error:    --> serde_with_macros\src\lib.rs:465:14
    |
465 |     #[derive(FromField, Debug)]
    |              ^^^^^^^^^
    |              |
    |              expected enum `syn::attr::NestedMeta`, found enum `NestedMeta`
    |              this expression has type `syn::attr::NestedMeta`
    |
    = note: perhaps two different versions of crate `syn` are being used?
    = note: this error originates in a derive macro (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0308]: mismatched types
Error:    --> serde_with_macros\src\lib.rs:465:14
    |
465 |     #[derive(FromField, Debug)]
    |              ^^^^^^^^^ expected enum `syn::attr::Meta`, found enum `syn::Meta`
    |
    = note: expected reference `&syn::attr::Meta`
               found reference `&syn::Meta`
    = note: perhaps two different versions of crate `syn` are being used?
    = note: this error originates in a derive macro (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0308]: mismatched types
Error:    --> serde_with_macros\src\lib.rs:482:14
    |
482 |     #[derive(FromField, Debug)]
    |              ^^^^^^^^^ expected struct `syn::attr::Attribute`, found struct `Attribute`
    |
    = note: expected reference `&syn::attr::Attribute`
               found reference `&Attribute`
    = note: perhaps two different versions of crate `syn` are being used?
    = note: this error originates in a derive macro (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0308]: mismatched types
Error:    --> serde_with_macros\src\lib.rs:482:14
    |
482 |     #[derive(FromField, Debug)]
    |              ^^^^^^^^^
    |              |
    |              expected enum `syn::attr::NestedMeta`, found enum `NestedMeta`
    |              this expression has type `syn::attr::NestedMeta`
    |
    = note: perhaps two different versions of crate `syn` are being used?
    = note: this error originates in a derive macro (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0308]: mismatched types
Error:    --> serde_with_macros\src\lib.rs:482:14
    |
482 |     #[derive(FromField, Debug)]
    |              ^^^^^^^^^ expected enum `syn::attr::Meta`, found enum `syn::Meta`
    |
    = note: expected reference `&syn::attr::Meta`
               found reference `&syn::Meta`
    = note: perhaps two different versions of crate `syn` are being used?
    = note: this error originates in a derive macro (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0308]: mismatched types
Error:   --> serde_with_macros\src\utils.rs:26:10
   |
26 | #[derive(FromDeriveInput, Debug)]
   |          ^^^^^^^^^^^^^^^ expected struct `syn::attr::Attribute`, found struct `Attribute`
   |
   = note: expected reference `&syn::attr::Attribute`
              found reference `&Attribute`
   = note: perhaps two different versions of crate `syn` are being used?
   = note: this error originates in a derive macro (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0308]: mismatched types
Error:   --> serde_with_macros\src\utils.rs:26:10
   |
26 | #[derive(FromDeriveInput, Debug)]
   |          ^^^^^^^^^^^^^^^
   |          |
   |          expected enum `syn::attr::NestedMeta`, found enum `NestedMeta`
   |          this expression has type `syn::attr::NestedMeta`
   |
   = note: perhaps two different versions of crate `syn` are being used?
   = note: this error originates in a derive macro (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0308]: mismatched types
Error:   --> serde_with_macros\src\utils.rs:26:10
   |
26 | #[derive(FromDeriveInput, Debug)]
   |          ^^^^^^^^^^^^^^^ expected enum `syn::attr::Meta`, found enum `syn::Meta`
   |
   = note: expected reference `&syn::attr::Meta`
              found reference `&syn::Meta`
   = note: perhaps two different versions of crate `syn` are being used?
   = note: this error originates in a derive macro (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0277]: the trait bound `syn::Path: FromMeta` is not satisfied
Error:   --> serde_with_macros\src\utils.rs:26:10
   |
26 | #[derive(FromDeriveInput, Debug)]
   |          ^^^^^^^^^^^^^^^ the trait `FromMeta` is not implemented for `syn::Path`
   |
   = note: required because of the requirements on the impl of `FromMeta` for `Option<syn::Path>`
   = note: required by `from_meta`
   = note: this error originates in a derive macro (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0308]: mismatched types
Error:   --> serde_with_macros\src\utils.rs:36:60
   |
36 |         match <Self as FromDeriveInput>::from_derive_input(&input) {
   |                                                            ^^^^^^ expected struct `syn::derive::DeriveInput`, found `&DeriveInput`
   |
   = note: expected reference `&syn::derive::DeriveInput`
              found reference `&&DeriveInput`

error: aborting due to 19 previous errors
Error: aborting due to 19 previous errors
Some errors have detailed explanations: E0053, E0277, E0308.
For more information about an error, try `rustc --explain E0053`.
error: could not compile `serde_with_macros`
Error: could not compile `serde_with_macros`
To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
Warning: error: build failed
Error: The process 'C:\Rust\.cargo\bin\cargo.exe' failed with exit code 101

Instance 1: https://github.com/jonasbb/serde_with/runs/1669122665#step:5:139
Instance 2: https://github.com/jonasbb/serde_with/runs/1658483317#step:5:139

The error occurs on both Ubuntu and Windows, but is non-deterministic. I think the first PR where I noticed that is jonasbb/serde_with#238 but I re-run the failed tests and now I cannot find the old logs. Before that point in time I haven't seen the issue before.

ehuss · 2021-01-09T23:37:40Z

Thanks for the report @jonasbb! I have posted a fix at #9059.

ehuss added C-tracking-issue Category: A tracking issue for something unstable. A-features2 Area: issues specifically related to the v2 feature resolver labels Feb 21, 2020

u1f408 mentioned this issue Feb 26, 2020

Build script using std can't compile rust-osdev/cargo-xbuild#10

Open

metajack mentioned this issue Feb 26, 2020

mime_guess 1.8.7 broken under new feature resolver -Z features=build_dep abonander/mime_guess#56

Closed

ehuss mentioned this issue Mar 15, 2020

Add proc-macro to index, and new feature resolver. #8003

Merged

ehuss changed the title ~~Tracking issue for -Z features=build_dep~~ Tracking issue for -Z features=host_dep Mar 27, 2020

sunshowers mentioned this issue Mar 28, 2020

handle platform dependencies + update feature graph construction facebookarchive/cargo-guppy#82

Merged

tomaka mentioned this issue Mar 29, 2020

Put everything in the same workspace tomaka/redshirt#369

Closed

jamwaffles mentioned this issue Jul 15, 2020

Cargo readme embedded-graphics/embedded-graphics#381

Merged

5 tasks

ehuss mentioned this issue Aug 5, 2020

A build dependency can force runtime features to be enabled in the final artifact #8592

Closed

chemicstry mentioned this issue Sep 22, 2020

Update riscv dependency version rust-embedded-community/async-on-embedded#10

Merged

ehuss mentioned this issue Oct 8, 2020

A dependency in both [dependencies] and [build-dependencies] has both feature sets combined #8759

Closed

wigy-opensource-developer mentioned this issue Oct 28, 2020

WIP: feedback needed - no_std multiformats/rust-multibase#25

Closed

This was referenced Oct 31, 2020

force proc-macro to be no_std compatable ia0/data-encoding#35

Merged

Macros Won't Build in no_std ia0/data-encoding#40

Closed

vmx mentioned this issue Nov 6, 2020

no_std Compatibility multiformats/rust-multibase#30

Merged

bytesnake mentioned this issue Nov 27, 2020

Errors Building Linfa Clustering and Linfa Reduction rust-ml/linfa#63

Closed

tarcieri mentioned this issue Nov 30, 2020

Crypto_Box doesn't compile on No_Std target RustCrypto/AEADs#242

Closed

ehuss mentioned this issue Dec 18, 2020

Stabilize -Zfeatures and -Zpackage-features. #8997

Merged

bors closed this as completed in 4aa5223 Jan 5, 2021

jaboatman mentioned this issue Jan 8, 2021

Upgrade to Tokio 1.0 importcjj/mobc#49

Merged

ryan-summers mentioned this issue Jan 20, 2021

Adding support for reporting service information quartiq/booster#131

Merged

ia0 mentioned this issue May 26, 2021

feature std leaking when using macro in no_std env ia0/data-encoding#47

Closed

clangenb mentioned this issue May 29, 2021

'sp-version' breaks no_std builds that are not wasm paritytech/substrate#8891

Closed

benma mentioned this issue Nov 11, 2021

prost: Pre-compile the tool BitBoxSwiss/bitbox02-firmware#829

Merged

tsoutsman mentioned this issue Jun 20, 2022

no_std support with serde feature bincode-org/bincode#556

Closed

haata mentioned this issue May 1, 2023

macros: convert to no_std twitchyliquid64/usbd-hid#53

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tracking issue for -Z features=host_dep #7915

Tracking issue for -Z features=host_dep #7915

ehuss commented Feb 21, 2020 •

edited

Loading

Amanieu commented Feb 24, 2020

valarauca commented Feb 24, 2020 •

edited

Loading

tarcieri commented Feb 27, 2020 •

edited

Loading

ia0 commented Feb 29, 2020

ehuss commented Mar 27, 2020

valarauca commented Mar 30, 2020 •

edited

Loading

BrandonArp commented Jul 18, 2020

blaind commented Aug 29, 2020

h4x3rotab commented Sep 11, 2020

ctron commented Oct 13, 2020

jonasbb commented Jan 8, 2021 •

edited

Loading

ehuss commented Jan 9, 2021

Tracking issue for -Z features=host_dep #7915

Tracking issue for -Z features=host_dep #7915

Comments

ehuss commented Feb 21, 2020 • edited Loading

Amanieu commented Feb 24, 2020

valarauca commented Feb 24, 2020 • edited Loading

tarcieri commented Feb 27, 2020 • edited Loading

ia0 commented Feb 29, 2020

ehuss commented Mar 27, 2020

valarauca commented Mar 30, 2020 • edited Loading

BrandonArp commented Jul 18, 2020

blaind commented Aug 29, 2020

h4x3rotab commented Sep 11, 2020

ctron commented Oct 13, 2020

jonasbb commented Jan 8, 2021 • edited Loading

ehuss commented Jan 9, 2021

ehuss commented Feb 21, 2020 •

edited

Loading

valarauca commented Feb 24, 2020 •

edited

Loading

tarcieri commented Feb 27, 2020 •

edited

Loading

valarauca commented Mar 30, 2020 •

edited

Loading

jonasbb commented Jan 8, 2021 •

edited

Loading