Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Build with clang/lld fails with undefined reference error when LTO is enabled #4596

Open
vashirov opened this issue Feb 5, 2021 · 6 comments · May be fixed by #6392
Open

Build with clang/lld fails with undefined reference error when LTO is enabled #4596

vashirov opened this issue Feb 5, 2021 · 6 comments · May be fixed by #6392
Assignees
@vashirov
Milestone
2.x backlog

Comments

@vashirov
Copy link
Member

vashirov commented Feb 5, 2021

Issue Description
In Fedora 33 LTO is enabled by default: https://fedoraproject.org/wiki/LTOByDefault
Build with clang/lld fails with undefined reference error.

Steps to Reproduce
On Fedora 33 with

lld-11.0.0-1.fc33.x86_64
clang-11.0.0-2.fc33.x86_64
llvm-libs-11.0.0-1.fc33.x86_64

$ sed -i -e 's/COCKPIT_ON = 1/COCKPIT_ON = 0/g' -e 's/CLANG_ON = 0/CLANG_ON = 1/g' rpm.mk
$ make -f rpm.mk rpms

/bin/sh ./libtool  --tag=CC   --mode=link clang -g -O2 -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1     -O2 -flto -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS --config /usr/lib/rpm/redhat/redhat-hardened-clang.cfg -fstack-protector-strong   -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -avoid-version -Wl,-z,relro -Wl,--as-needed  -Wl,-z,now  -flto -o libattr-unique-plugin.la -rpath /usr/lib64/dirsrv/plugins ldap/servers/plugins/uiduniq/libattr_unique_plugin_la-7bit.lo ldap/servers/plugins/uiduniq/libattr_unique_plugin_la-uid.lo ldap/servers/plugins/uiduniq/libattr_unique_plugin_la-utils.lo libslapd.la -lplds4 -lplc4 -lnspr4 -lpthread -ldl  
ld.lld: error: ./.libs/libslapd.so: undefined reference to __rust_probestack [--no-allow-shlib-undefined]
ld.lld: error: ./.libs/libslapd.so: undefined reference to __muloti4 [--no-allow-shlib-undefined]
clang-11: error: linker command failed with exit code 1 (use -v to see invocation)

If I disable LTO in the spec file, build succeeds:

%global _lto_cflags %nil
@vashirov vashirov added the needs triage The issue will be triaged during scrum label Feb 5, 2021
@Firstyear
Copy link
Contributor

@vashirov Should we try using lto in our make/configure so that all developers are testing this regularly? I have been using clang 11 + rust recently, so I can check this for clang too.

@Firstyear
Copy link
Contributor

Also if we turn on LTO in C, we should consider it for rust as well.

@vashirov
Copy link
Member Author

vashirov commented Feb 8, 2021

@vashirov Should we try using lto in our make/configure so that all developers are testing this regularly? I have been using clang 11 + rust recently, so I can check this for clang too.

LTO is on by default in Fedora 33 and OpenSUSE Tumbleweed when rpm build is used. I use rpms target in our GH actions (#4599), so it will be tested on each PR. I wanted to add clang variant to the pipeline, but encountered the issue above.

I don't have a strong opinion about adding it to make/configure though.

Also if we turn on LTO in C, we should consider it for rust as well.

A profile in Cargo.toml?

@Firstyear
Copy link
Contributor

Well it looks like SUSE is doing it by default as well, so it would be best for us to "develop how it's used", which means we should enable LTO by default for us developers too. That's why I suggest we make it default.

Yes, I think it's a profile flag according to https://doc.rust-lang.org/cargo/reference/profiles.html we probably want "thin".

@vashirov
Copy link
Member Author

vashirov commented Feb 9, 2021

OK, I will submit a PR with the changes.

@vashirov vashirov self-assigned this Feb 9, 2021
@mreynolds389 mreynolds389 removed the needs triage The issue will be triaged during scrum label Feb 11, 2021
@mreynolds389 mreynolds389 added this to the 2.x backlog milestone Feb 11, 2021
@vashirov vashirov mentioned this issue Sep 9, 2021
Closed
@vashirov
Copy link
Member Author

@Firstyear, I need your help regarding fixing Rust code with LTO. IIUC, error messages in the description should be fixed by using compiler-builtins crate. I looked at the Rust libs under src/, but I have a hard time understanding how they are related to each other and what exactly needs to be patched to solve this issue. We have a page on wiki describing some usage of Rust in DS, but I'd appreciate an overview of the code layout and how things are interconnected, some kind of high-level architecture overview doc.
Thanks!

@Firstyear Firstyear mentioned this issue Sep 13, 2021
Closed
Firstyear added a commit to Firstyear/389-ds-base that referenced this issue Sep 15, 2021
@Firstyear
Issue 4596 - BUG - lto linking issues
380d21d 
Bug Description: Fedora is introducing lto by default which
may affect our build especially with rust and asan

Fix Description: Adapt how we use lto and how we link in
our builds so that lto works for gcc.

fixes: 389ds#4596

Author: William Brown <william@blackhats.net.au>

Review by: ???
@vashirov vashirov mentioned this issue Sep 21, 2021
Closed
vashirov pushed a commit to vashirov/389-ds-base that referenced this issue Nov 1, 2024
@Firstyear @vashirov
Issue 4596 - BUG - lto linking issues
ab8b96a 
Bug Description: Fedora is introducing lto by default which
may affect our build especially with rust and asan

Fix Description: Adapt how we use lto and how we link in
our builds so that lto works for gcc.

fixes: 389ds#4596

Author: William Brown <william@blackhats.net.au>

Review by: ???
vashirov added a commit to vashirov/389-ds-base that referenced this issue Nov 1, 2024
@vashirov
Issue 4596 - Build with clang/lld fails when LTO enabled
4277f27 
Bug Description:
Build with clang/lld fails with undefined reference error.
```
ld.lld: error: ./.libs/libslapd.so: undefined reference to __rust_probestack [--no-allow-shlib-undefined]
ld.lld: error: ./.libs/libslapd.so: undefined reference to __muloti4 [--no-allow-shlib-undefined]
```

Fix Description:
* Disabled GCC security flags when building with clang.
* lld by default uses xxhash for build ids, which is too small for rpm
(it requires it between 16 and 64 bytes in size), use sha1 instead.
* Switch debug CFLAGS and LDFLAGS to use DWARF4 instead of DWARF5.
* Disable LTO for clang rpm build.

Fixes: 389ds#4596

Reviewed by: ???
@vashirov vashirov linked a pull request Nov 1, 2024 that will close this issue
Open
vashirov added a commit to vashirov/389-ds-base that referenced this issue Nov 1, 2024
@vashirov
Issue 4596 - Build with clang/lld fails when LTO enabled
0e11219 
Bug Description:
Build with clang/lld fails with undefined reference error.
```
ld.lld: error: ./.libs/libslapd.so: undefined reference to __rust_probestack [--no-allow-shlib-undefined]
ld.lld: error: ./.libs/libslapd.so: undefined reference to __muloti4 [--no-allow-shlib-undefined]
```

Fix Description:
* Disabled GCC security flags when building with clang.
* lld by default uses xxhash for build ids, which is too small for rpm
(it requires it between 16 and 64 bytes in size), use sha1 instead.
* Switch debug CFLAGS and LDFLAGS to use DWARF4 instead of DWARF5.
* Disable LTO for clang rpm build.

Fixes: 389ds#4596

Reviewed by: ???
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vashirov vashirov

Labels
None yet
Projects
None yet
Milestone
2.x backlog
3 participants
@Firstyear @vashirov @mreynolds389