undefined reference to linker error when using dylibs
#82151
Comments
alexkornitzer
added
C-bug
jonas-schievink
added
A-linkage
|
Hi @jonas-schievink, could you let me know how small the example needs to be? I have managed to shrink it to just reqwest & tokio, but might be able to get it much smaller assuming I dont need intimate compiler knowledge in order to guess what the issue could potentially be. |
|
Ideally it would not contain any external dependencies |
|
Okay let me see if I can isolate it down further, but it might be beyond me. MusingsNot managed to reduce it down yet but I have observed the following which seems interesting, when using but if we drop to and if we try and bring but not with |
|
Hey @jonas-schievink, Right got it, so its looks like the issue is when you have the following: where ---bar
pub struct Bar;
pub fn bar() -> Bar {
Bar
}
---foo
pub use bar::bar;
pub struct Foo;
pub fn foo() -> Foo {
Foo
}
--shared
pub struct Test;
impl Test {
pub fn new() -> Self {
let _ = foo::foo();
let _ = foo::bar();
Self
}
}
--app
fn main() {
let _ = shared::Test::new();
}I have a minimal example here: https://github.com/alexkornitzer/dylib-errors/tree/error/nested |
jonas-schievink
added
I-prioritize
|
Bisected, but this goes back to at least |
|
@rustbot label I-nominated We would like the compiler team to have a closer look at this (tomorrow during the meeting) to help assessing the implicatons (Zulip discussion) |
nagisa
removed
the
I-ICE
|
Assigning @rustbot label -I-prioritize +P-high |
rustbot
added
P-high
|
Potentially related to #56784 |
|
looking at aforementioned example from comment above ...
... It seems like something is different between stable/beta and nightly here, because here's what I see on this reduced test case: Click to see output(I also think the nightly behavior seems better. The diagnostic we see with those nightly runs is not itself new though; it dates from PR #64324 in 2019... so something must have changed elsewhere to make rustc's control flow start flowing into that code path. |
|
Wow, that is actually a very recent change, introduced in between nightlies for 2021-03-14 and 2021-03-15 ...! |
|
cargo-bisect-rustc result: searched toolchains acca818 through d6eaea1 Regression in 1381dcf searched nightlies: from nightly-2021-03-13 to nightly-2021-03-15 bisected with cargo-bisect-rustc v0.6.0Host triple: x86_64-unknown-linux-gnu cargo bisect-rustc --access github --start 2021-03-13 --end 2021-03-15 --prompt |
|
Hmm well now I'm wondering if a change to at least, my attempts to investigate via local builds of Click to see a pair of invocations, both (attempting to?) override the rustc version to keep it fixed while also selecting different `cargo` version to drive the build |
|
@alexkornitzer It looks to me like the nightly version of While I imagine that this outcome is not ideal for you, I am still curious whether it at least gives you something you can act on? (In other words, I'm trying to figure out how to triage this issue, given the changes in its behavior after the cargo change.) |
|
1I will say the fact that two of the messages are: does seem unhelpful. At least, without knowing why |
|
@pnkfelix thanks for looking into this. I think what you have discovered with the new cargo nightly is what the For my current use case, being able to compile into a single shared library would solve the problem for me. As with the As to solving the use case of having |
|
I think the linker error is caused by rustc thinking that the cdylib of hyper is a dylib because both have the same extension Only dylib exports the required functions for linking to succeed. Rustc currently doesn't check that a dynamic library contains crate metadata if there is an rmeta or rlib of the same name: rust/compiler/rustc_metadata/src/locator.rs Lines 526 to 529 in 00ce166 |
|
We may be able to fix that "arguably a bug", because dylibs are rare in practice now, most of crate searches will not encounter a dylib candidate. |
This reverts commit 5727de3. The upgrade seems to be blocked by rust-lang/rust#82151
|
In our project we use |
|
@fpoli I am not too familiar with this issue, but I am wondering if @alexkornitzer workaround in alexkornitzer/hyper@1199048 would help (assuming is it ok for you to vendor |
Thanks! So, that would solve the issue in |
Currently, the `hyper` crate (dependency of `reqwest`) causes linker errors when you attempt to compile a consumer of `steven_protocol` as a dylib. Compiling as a dylib is not uncommon for quick iteration; notably popularized by the Bevy game engine. See rust-lang/rust#82151 (comment).
|
from conversation above
It sounds like this might be the right next step. I'm already spending some amount of time looking at linker issues (and this is already assigned to me), so I can continue down this path. (We might want an MCP for the suggested change, in any case.) |
|
Opened #113695 with a fix for what I am pretty sure is the root cause here. (The test I added fails with the same linker issue before that PR at least.) |
…nkfelix,petrochenkov Verify that all crate sources are in sync This ensures that rustc will not attempt to link against a cdylib as if it is a rust dylib when an rlib for the same crate is available. Previously rustc didn't actually check if any further formats of a crate which has been loaded are of the same version and if they are actually valid. This caused a cdylib to be interpreted as rust dylib as soon as the corresponding rlib was loaded. As cdylibs don't export any rust symbols, linking would fail if rustc decides to link against the cdylib rather than the rlib. Two crates depended on the previous behavior by separately compiling a test crate as both rlib and dylib. These have been changed to capture their original spirit to the best of my ability while still working when rustc verifies that all crates are in sync. It is unlikely that build systems depend on the current behavior and in any case we are taking a lot of measures to ensure that any change to either the source or the compilation options (including crate type) results in rustc rejecting it as incompatible. We merely didn't do this check here for now obsolete perf reasons. Fixes rust-lang/rust#10786 Fixes rust-lang/rust#82151 Fixes rust-lang/rust#82972 Closes bevy-cheatbook/bevy-cheatbook#114
When including
mongodb(2.0.0-alpha) into a dylib'd crate that is included in an application, the linker is failing to resolve all references. It is fine when usingmongodb(1.0) which makes me think this is something that tokio 1.0 is bringing to light.This seems similar to #67276 except the workarounds it suggested at the time do not work.
Code
Due to the nature of this making a truly small example is quite challenging, I assume isolating tokio 1.2 out might work? But anyway for now, an repo for the example is here:
https://github.com/alexkornitzer/dylib-errors/tree/error/mongodb
Meta
rustc --version --verbose:Error output
Click to see (large) error output
The text was updated successfully, but these errors were encountered: