Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compiling to Shared Library doesn't always expose reexported symbols #50007

Closed
CryZe opened this issue Apr 16, 2018 · 11 comments · Fixed by #95604
Closed

Compiling to Shared Library doesn't always expose reexported symbols #50007

CryZe opened this issue Apr 16, 2018 · 11 comments · Fixed by #95604
Labels
A-linkage Area: linking into static, shared libraries and binaries C-bug Category: This is a bug. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@CryZe
Copy link
Contributor

CryZe commented Apr 16, 2018

Consider the following:

We have 2 crates, rlib helper and the shared library foo that depends on the helper crate. The helper crate's lib.rs looks like this:

#[no_mangle]
pub extern fn bla() -> i32 {
    5
}

The shared library's lib.rs looks like this:

extern crate helper;

pub use helper::bla;

The code is then compiled to a shared library. The bla symbol is not exported. That does not seem intentional.

@dbrgn
Copy link
Contributor

dbrgn commented May 15, 2018

Maybe a duplicate of #36342?

@XAMPPRocky XAMPPRocky added A-linkage Area: linking into static, shared libraries and binaries T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. C-bug Category: This is a bug. labels Aug 27, 2018
@thomcc
Copy link
Member

thomcc commented Nov 8, 2018

It's not a duplicate of #36342, which is about reexporting functions defined in C AFAICT. We've been struggling with this in https://github.com/mozilla/application-services.

This doesn't seem to happen with all targets. In particular, x86_64-apple-darwin seems unaffected, while i686-linux-android is effected (really all android targets seem to be affected).

I have a simple reproducable case at https://github.com/thomcc/megazord/tree/simple-bug-repro (e.g. the simple-bug-repro branch of that repository). An explanation for how to use it to reproduce the issue is in the README.md.

It's worth noting that if you turn on LTO, the issue does not manifest (all symbols are present), presumably due to it being a single compilation unit at that point (thanks to the reporter for pointing this out to me, I did not realize it initially).

@ncalexan
Copy link

ncalexan commented Nov 8, 2018

This doesn't seem to happen with all targets. In particular, x86_64-apple-darwin seems unaffected, while i686-linux-android is effected (really all android targets seem to be affected).

For more context: the Android toolchains are all specifying a stand-alone Android NDK toolchain, in particular a clang-based toolchain. I believe our Desktop targets (like x86_64-apple-darwin) are GCC based. So we may just be seeing a clang-ism as it passes linking through to an Android NDK toolchain GNU ld. I haven't tried lld to get to the bottom of this.

@Mark-Simulacrum
Copy link
Member

The right fix here might be that the relevant extern blocks should get kind = "static" on the link attributes, per #62088 (comment).

@thomcc
Copy link
Member

thomcc commented Aug 10, 2019

@Mark-Simulacrum This issue is about no_mangle symbols from another crate that get brought in by use. There are no extern blocks.

@kvark
Copy link
Contributor

kvark commented Feb 12, 2020

Heads up - this hits us in https://github.com/gfx-rs/wgpu as well. Very annoying and inconvenient, would very much appreciate any progress here!

@hansihe
Copy link

hansihe commented May 14, 2020

Hitting this issue as well.

My use-case is exporting some functions from my executable, then loading a dynamic library that needs to link to those symbols.

ryzhyk added a commit to Kixiron/differential-datalog that referenced this issue Dec 22, 2020
After `record.rs` was split into multiple modules, the C API disappeared
from the compiled .so library.  This is likely due to this compiler bug:
rust-lang/rust#50007

As a workaround until the bug is fixed, we move the API from `record/c_api.rs`
to `record/mod.rs`.
ryzhyk added a commit to vmware/differential-datalog that referenced this issue Dec 22, 2020
After `record.rs` was split into multiple modules, the C API disappeared
from the compiled .so library.  This is likely due to this compiler bug:
rust-lang/rust#50007

As a workaround until the bug is fixed, we move the API from `record/c_api.rs`
to `record/mod.rs`.
badboy added a commit to badboy/glean that referenced this issue Apr 26, 2021
Workaround to force a re-export of the `no_mangle` symbols from `glean_ffi`

Due to how linking works and hides symbols the symbols from `glean_ffi` might not be
re-exported and thus not usable.
By forcing use of _at least one_ symbol in an exported function the functions will also be
rexported.
This is only required for debug builds (and `debug_assertions` is the closest thing we have to
check that).
In release builds we rely on LTO builds to take care of it.
As the tests will use libglean_ffi functionality running the tests
should ensure this actually happens.

See rust-lang/rust#50007
badboy added a commit to badboy/glean that referenced this issue Apr 26, 2021
Workaround to force a re-export of the `no_mangle` symbols from `glean_ffi`

Due to how linking works and hides symbols the symbols from `glean_ffi` might not be
re-exported and thus not usable.
By forcing use of _at least one_ symbol in an exported function the functions will also be
rexported.
This is only required for debug builds (and `debug_assertions` is the closest thing we have to
check that).
In release builds we rely on LTO builds to take care of it.
As the tests will use libglean_ffi functionality running the tests
should ensure this actually happens.

See rust-lang/rust#50007
badboy added a commit to badboy/glean that referenced this issue May 7, 2021
Workaround to force a re-export of the `no_mangle` symbols from `glean_ffi`

Due to how linking works and hides symbols the symbols from `glean_ffi` might not be
re-exported and thus not usable.
By forcing use of _at least one_ symbol in an exported function the functions will also be
rexported.
This is only required for debug builds (and `debug_assertions` is the closest thing we have to
check that).
In release builds we rely on LTO builds to take care of it.
As the tests will use libglean_ffi functionality running the tests
should ensure this actually happens.

See rust-lang/rust#50007
badboy added a commit to mozilla/glean that referenced this issue May 17, 2021
Workaround to force a re-export of the `no_mangle` symbols from `glean_ffi`

Due to how linking works and hides symbols the symbols from `glean_ffi` might not be
re-exported and thus not usable.
By forcing use of _at least one_ symbol in an exported function the functions will also be
rexported.
This is only required for debug builds (and `debug_assertions` is the closest thing we have to
check that).
In release builds we rely on LTO builds to take care of it.
As the tests will use libglean_ffi functionality running the tests
should ensure this actually happens.

See rust-lang/rust#50007
badboy added a commit to badboy/glean that referenced this issue Jul 27, 2021
Workaround to force a re-export of the `no_mangle` symbols from `glean_ffi`

Due to how linking works and hides symbols the symbols from `glean_ffi` might not be
re-exported and thus not usable.
By forcing use of _at least one_ symbol in an exported function the functions will also be
rexported.
This is only required for debug builds (and `debug_assertions` is the closest thing we have to
check that).
In release builds we rely on LTO builds to take care of it.
As the tests will use libglean_ffi functionality running the tests
should ensure this actually happens.

See rust-lang/rust#50007
bendk added a commit to bendk/uniffi-rs that referenced this issue Mar 28, 2022
This can be used to work around
[rust-lang#50007](rust-lang/rust#50007), which
is getting to be more and more of an issue on the desktop JS project.
bendk added a commit to bendk/uniffi-rs that referenced this issue Mar 28, 2022
This can be used to work around
[rust-lang#50007](rust-lang/rust#50007), which
is getting to be more and more of an issue on the desktop JS project.
bendk added a commit to bendk/uniffi-rs that referenced this issue Mar 30, 2022
This can be used to work around
[rust-lang#50007](rust-lang/rust#50007), which
is getting to be more and more of an issue on the desktop JS project.

Updated `uniffi::interface` and some of the fixtures so that we can test this.
bendk added a commit to bendk/uniffi-rs that referenced this issue Mar 30, 2022
This can be used to work around
[rust-lang#50007](rust-lang/rust#50007), which
is getting to be more and more of an issue on the desktop JS project.

Updated `uniffi::interface` and some of the fixtures so that we can test this.
bendk added a commit to mozilla/uniffi-rs that referenced this issue Mar 31, 2022
This can be used to work around
[rust-lang#50007](rust-lang/rust#50007), which
is getting to be more and more of an issue on the desktop JS project.

Updated `uniffi::interface` and some of the fixtures so that we can test this.
Dylan-DPC added a commit to Dylan-DPC/rust that referenced this issue Apr 10, 2022
Generate synthetic object file to ensure all exported and used symbols participate in the linking

Fix rust-lang#50007 and rust-lang#47384

This is the synthetic object file approach that I described in rust-lang#95363 (comment), allowing all exported and used symbols to be linked while still allowing them to be GCed.

Related rust-lang#93791, rust-lang#95363

r? `@petrochenkov`
cc `@carbotaniuman`
Dylan-DPC added a commit to Dylan-DPC/rust that referenced this issue Apr 11, 2022
Generate synthetic object file to ensure all exported and used symbols participate in the linking

Fix rust-lang#50007 and rust-lang#47384

This is the synthetic object file approach that I described in rust-lang#95363 (comment), allowing all exported and used symbols to be linked while still allowing them to be GCed.

Related rust-lang#93791, rust-lang#95363

r? ``@petrochenkov``
cc ``@carbotaniuman``
Dylan-DPC added a commit to Dylan-DPC/rust that referenced this issue Apr 11, 2022
Generate synthetic object file to ensure all exported and used symbols participate in the linking

Fix rust-lang#50007 and rust-lang#47384

This is the synthetic object file approach that I described in rust-lang#95363 (comment), allowing all exported and used symbols to be linked while still allowing them to be GCed.

Related rust-lang#93791, rust-lang#95363

r? ```@petrochenkov```
cc ```@carbotaniuman```
Dylan-DPC added a commit to Dylan-DPC/rust that referenced this issue Apr 17, 2022
Generate synthetic object file to ensure all exported and used symbols participate in the linking

Fix rust-lang#50007 and rust-lang#47384

This is the synthetic object file approach that I described in rust-lang#95363 (comment), allowing all exported and used symbols to be linked while still allowing them to be GCed.

Related rust-lang#93791, rust-lang#95363

r? `@petrochenkov`
cc `@carbotaniuman`
Dylan-DPC added a commit to Dylan-DPC/rust that referenced this issue Apr 18, 2022
Generate synthetic object file to ensure all exported and used symbols participate in the linking

Fix rust-lang#50007 and rust-lang#47384

This is the synthetic object file approach that I described in rust-lang#95363 (comment), allowing all exported and used symbols to be linked while still allowing them to be GCed.

Related rust-lang#93791, rust-lang#95363

r? `@petrochenkov`
cc `@carbotaniuman`
@bors bors closed this as completed in 18b53ce Apr 25, 2022
JelteF added a commit to JelteF/pgrx that referenced this issue Feb 6, 2023
When changing something trivial in the hello world project created by
`cargo pgx new`, it would take ~20 seconds to compile a dev build of the
generated crate on my machine. This seemed way slower than it should be.
It turns out that the thin LTO setting greatly inrceases compilation
times. By disabling it, compilation time of the crate is only 1 second.

As far as I can tell this setting was enabled initially as a workaround
for this bug in Rust: rust-lang/rust#50007
Since this bug has been fixed since Rust 1.62, and current stable Rust
is version 1.67, this PR removes the workaround.
JelteF added a commit to JelteF/pgrx that referenced this issue Feb 6, 2023
When changing something trivial in the hello world project created by
`cargo pgx new`, it would take ~20 seconds to compile a dev build of the
generated crate on my machine. This seemed way slower than it should be.
It turns out that the thin LTO setting greatly inrceases compilation
times. By disabling it, compilation time of the crate is only 1 second.

As far as I can tell this setting was enabled initially as a workaround
for this bug in Rust: rust-lang/rust#50007
Since this bug has been fixed since Rust 1.62, and current stable Rust
is version 1.67, this PR removes the workaround.
JelteF added a commit to JelteF/pgrx that referenced this issue Feb 8, 2023
When changing something trivial in the hello world project created by
`cargo pgx new`, it would take ~20 seconds to compile a dev build of the
generated crate on my machine. This seemed way slower than it should be.
It turns out that the thin LTO setting greatly inrceases compilation
times. By disabling it, compilation time of the crate is only 1 second.

As far as I can tell this setting was enabled initially as a workaround
for this bug in Rust: rust-lang/rust#50007
Since this bug has been fixed since Rust 1.62, and current stable Rust
is version 1.67, this PR removes the workaround.
eeeebbbbrrrr pushed a commit to pgcentralfoundation/pgrx that referenced this issue Feb 16, 2023
When changing something trivial in the hello world project created by
`cargo pgx new`, it would take ~20 seconds to compile a dev build of the
generated crate on my machine. This seemed way slower than it should be.
It turns out that the thin LTO setting greatly inrceases compilation
times. By disabling it, compilation time of the crate is only 1 second.

As far as I can tell this setting was enabled initially as a workaround
for this bug in Rust: rust-lang/rust#50007
Since this bug has been fixed since Rust 1.62, and current stable Rust
is version 1.67, this PR removes the workaround.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-linkage Area: linking into static, shared libraries and binaries C-bug Category: This is a bug. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet