Inventory doesn't work with Rust >= 1.54

[phorward]

Hey there!

It seems inventory v0.1.10 does not work with Rust 1.54 as with Rust 1.53 before. I tested it with my project, when I run with cargo +1.53 run -- "1" it works as expected, when I try to run it with just cargo run -- "1" on a rustc 1.54 stable toolchain, no plugins are registered anymore.

Your example code from here runs as expected.

In my project in this mod.rs the for-loop at the end is never executed, as the inventory::iter::<Builtin> always returns None as first call to next(). I found out, that when I copy the plugins from the modules _std, string and token (first three mod lines) directly into the mod.rs, it works as expected. So inventory seems to not register plugins from other modules with Rust 1.54 anymore?

Can you give me a hint?

[dtolnay]

Nothing in this crate has changed since December. Can you try bisecting this to the first failing rustc nightly? https://github.com/rust-lang/cargo-bisect-rustc

[sdleffler]

I just hit this same issue; will see if I can find the time to bisect.

[sdleffler]

Unfortunately in my case whatever causes the failure appears to have occurred before nightly-2021-02-14, which is making me question whether the compiler is actually at fault. I just started this project a few days ago, so the dependencies start failing to resolve before about February 1st, 2021, and I can't bisect back to before then without updating dependencies. Will try that and report back.

[sdleffler]

I have a feeling this is not due to rustc but rather due to a bug in a dependency? I have bisected all the way back to nightly-2020-12-01 with some adjustments to unrelated dependencies and it is still present.

[phorward]

Hello @sdleffler, thanks for your effort and further reporting. I've created a test project inventory-test.tar.gz that reproduces the problem. It also contains a test.sh for cargo-bisect-rustc.

rustc 1.53.0 (53cb7b09b 2021-06-17)

$ cargo +1.53 run
   Compiling proc-macro2 v1.0.28
   Compiling unicode-xid v0.2.2
   Compiling syn v1.0.74
   Compiling inventory v0.1.10
   Compiling quote v1.0.9
   Compiling inventory-impl v0.1.10
   Compiling ctor v0.1.20
   Compiling ghost v0.1.2
   Compiling inventory-test v0.1.0 (/tmp/inventory-test)
    Finished dev [unoptimized + debuginfo] target(s) in 8.47s
     Running `target/debug/inventory-test`
-e, --error
-v, --verbose

rustc 1.54.0 (a178d0322 2021-07-26)

$ cargo run
   Compiling proc-macro2 v1.0.28
   Compiling unicode-xid v0.2.2
   Compiling syn v1.0.74
   Compiling inventory v0.1.10
   Compiling quote v1.0.9
   Compiling inventory-impl v0.1.10
   Compiling ctor v0.1.20
   Compiling ghost v0.1.2
   Compiling inventory-test v0.1.0 (/tmp/inventory-test)
    Finished dev [unoptimized + debuginfo] target(s) in 7.75s
     Running `target/debug/inventory-test`
-v, --verbose

I did several test-runs with cargo-bisect-rustc, but as they all run on rust-nightly, I'm unsure how to find out a regression between stable versions.

[sdleffler]

I did several test-runs with cargo-bisect-rustc, but as they all run on rust-nightly, I'm unsure how to find out a regression between stable versions.

I don't think we really mind that much here, Rust does sometimes have changes in nightly which accidentally make their way into stable and break things; it's more valuable to know exactly what spot in nightly broke the underlying ctor machinery. I'm going to take a stab at it with that project you've supplied myself and see what I can scrape up!

[sdleffler]

Something is really, really strange here. I cannot fathom why the 2021-01-01 nightly, which should predate stable 1.53 by nearly half a year, is clearly showing the same problem; but, after a cargo clean, cargo +1.53 run works as it should; I won't show the output as it's exactly like already shown above. This was not using bisect-rustc but rather using rustup to install the new year's nightly, and then cargo +nightly-2021-01-01 run to actually test.

What could possibly be the difference between a nightly which is this early yet still shows the issue, and the 1.53 stable? Is there some kind of difference in what libc it's using??

[tyranron]

@sdleffler can it be related to re-enabled incremental somehow?

[sdleffler]

Incremental compilation is a thought!! I'll give it a try, but I can tell one thing for sure now - it's the linker.

It's eliminating statics from different crates by eliminating the whole damn crate. If I pull in anything at all from one of the crates in question, all of the submitted inventory items from that crate show up.

[m-ou-se]

Setting codegen-units = 1 makes the issue disappear. With multiple codegen units, different modules end up in different .o files, which can be thrown out by the linker as a whole.

[phorward]

@m-ou-se thanks for the hint, I can confirm that my testcase works as expected on Rust 1.54 with RUSTFLAGS="-C codegen-units=1" cargo run.

[Hoverbear]

We met this today while working on a pgx feature and can confirm setting codegen-units = 1 resolves the issue. Prior to setting this only some types using inventory were present on iter, others appeared empty. This appears to confirm that the linker might be throwing out .o files.

[phorward]

Hey, I can confirm the same issue is reproducible with Rust 1.55. Is there still anything we can do here, or should I report to upstream?

[jerel]

I just ran into this as well. It doesn't seem to be related to a recent Rust version as I first encountered it on 1.51 and it behaves the same in 1.54 and 1.56. As far as I can tell the only condition needed for this bug is to have a large codegen-units value and incremental compilation enabled. The quantity of code and whether the compiler thinks a type is unused may also affect it as it worked when the project was small but then broke as the code quantity increased.

A real life example that I observed, below. (my_macro outputs submit!{ Plugin(meta) } to register a plugin that stores meta info)

// data.rs
#[my_macro]
#[derive(Serialize, Deserialize)]
enum One {
  A
}

#[derive(Serialize, Deserialize)]
struct Two {
  one: One
}

// mod.rs
pub fn example() -> data::Two {
  data::Two { one: data::One::A }
}

In this case inventory::iter::<Plugin> is always empty. If codegen-units is set to 1 then it works as intended or (strangely) if Debug is added to the derive of One and Two (it doesn't have to be printed, just have the Debug). Setting incremental = false also causes iter::<Plugin> to work. I find this strange because One is used. So the compiler seems to be losing the inventory init fn from the module but keeping the enum which is right below it.

Is there anything we can do to fix this in inventory or is it a compiler bug/feature?

[phorward]

I can confirm that this issue is similar to linkme/issues#31, and might have its origin in rust-lang/rust#47384.

[dtolnay]

This is fixed in rustc 1.62.0 / inventory 0.3.0.