Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically detect the appropriate linker to use when cross-compiling #4133

Open
infinity0 opened this issue Jun 7, 2017 · 11 comments
Open
Labels
A-cross-compiling Area: using --target flag for other platforms A-linkage Area: linker issues, dylib, cdylib, shared libraries, so C-feature-request Category: proposal for a feature. Before PR, ping rust-lang/cargo if this is not `Feature accepted` Command-build S-triage Status: This issue is waiting on initial triage.

Comments

@infinity0
Copy link
Contributor

In order to cross-compile the Debian cargo package I have to explicitly set RUSTFLAGS += -C linker=$(DEB_HOST_GNU_TYPE)-gcc, see here. I originally didn't think too much of this, however rustbuild does not require me to set this sort of thing when cross-compiling rustc, it automatically detects the appropriate linker to use. I wonder if it's possible to add similar detection logic to Cargo?

@alexcrichton
Copy link
Member

I've definitely often wanted to do this but Cargo's not really in a position to do so. It's never been quite clear where this logic would go.

@lucab
Copy link
Contributor

lucab commented Jun 9, 2017

I just spent some time debugging some obscure build failure, before realizing that this was the root cause (a non-native x86_64-unknown-linux-musl build using host cc instead of musl-gcc when linking).
This manifested as relocation errors with ring: briansmith/ring#544.

But it looks like other people previously stepped on this, again with musl: rust-lang/rust#40049 (comment).

@briansmith
Copy link

I've definitely often wanted to do this but Cargo's not really in a position to do so. It's never been quite clear where this logic would go.

I think it is OK to put it in Cargo if we can't identify any better place to put it. The kind of issue it would fix has tripped me up multiple times. Also, people tend to blame ring when things fail to link because often the first linker errors reported are in ring.

@carols10cents carols10cents added A-cross-compiling Area: using --target flag for other platforms A-linkage Area: linker issues, dylib, cdylib, shared libraries, so C-feature-request Category: proposal for a feature. Before PR, ping rust-lang/cargo if this is not `Feature accepted` Command-build labels Aug 30, 2017
@infinity0
Copy link
Contributor Author

What about adding the logic to rustc itself? Basically, what's already in cc-rs here. Since rustc is the one calling the linker, it could select the right linker when it's being told to cross-compile.

@EdSchouten
Copy link

If this is implemented, could we please default to ${target}-cc, as opposed to ${target}-gcc? People who use Clang-based cross compiler toolchains obviously don't have the latter.

@infinity0
Copy link
Contributor Author

Since Clang uses LLVM does it not have a --target flag to support cross-compilation out-of-the-box via one single cc binary? This situation of different ccs is only needed because of GCC's build system where you can only target one architecture for a given output compiler.

@elcritch
Copy link

elcritch commented Aug 20, 2019

This issue comes into play when cross compiling Cargo projects using scripted cross compile build systems. Most C Makefile or CMake projects seem to rely primarily on $CC and $LD to find the correct compiler and linker to use. This makes it easy to cross compile most projects just by setting those flags.

I tried using CARGO_TARGET_$(TRIPLET)_LINKER=$(LD) in a recent attempt to get Cargo to properly cross compile using a specific set of cross compiler and linker but it's pretty clumsy. The option was tricky to find, and it still resulted in a broken binary that wouldn't run.

IMHO, Cargo does seem the correct place for something like this if it's already going to support --target=xyz and automatically find the C cross compiler (for dependencies like backtrace-sys and nix), it should find the linker as well. Though the RUSTFLAGS option is probably better than the CARGO_TARGET_$(TRIPLET)_LINKER=$(LD) method I tried.

@jonnius
Copy link

jonnius commented Oct 13, 2021

Any new thoughts on this? I am currently specifying the corresponding gcc linker in the .cargo/config for each project. But this does not work for others not using gcc.

@heroin-moose
Copy link

Since cargo already has --target option it makes a lot of sense to make $(TRIPLET)-cc the default linker, as it's pretty obvious that the default one will not work.

@samliddicott
Copy link

samliddicott commented Oct 7, 2022

I think this question is relevant: https://users.rust-lang.org/t/same-arch-cross-compile-linker-mixup/82337

When cross-compiling to the host arch but different libs (CC, LD env vars set appropriately) cargo/rust tries to use the same linker for build_script_build targets as it does for the final target which is wrong. It ignores $(LD) unless but something like --config target.$triple.linker= can be used (no arguments allowed, grr) but that linker is also used for build_script_build binaries which run on the host instead of the target system (and so linking or running may fail).

I think that --config target.$triple.linker=... should apply only to the target and the usual linker should apply for build_script_build binaries -- the same as it does for differing-arch cross-compiles.

How to detect a cross-compile? I thought it was by specifying --target, but it seems that isn't sufficient.

My workaround was a linker wrapper with heuristics to guess which context it was being called in.

(I'm using plain cargo without any build script commands of my own)

@samliddicott
Copy link

samliddicott commented Oct 31, 2022

In case it helps, this is the linker wrapper that I am using, and then pass it to cargo with: --config target.$(RUST_TARGET).linker=\"my-hacky-linker-cc\"

#! /bin/bash

# When cross-building for same host arch, cargo uses the same linker
# for the build tools as it does for the target which is wrong
# So we detect why it is being called, if an argument includes the
# words build_script_build then we are building build toos for the host
# and use host gcc, otherwise we use target CC

grep -q 'build_script_main\|build_script_build' <<<"$*" && exec gcc "$@" || exec ${TARGET_CC:-${CC}} "$@"

github-merge-queue bot pushed a commit to stellar/rs-soroban-sdk that referenced this issue Aug 25, 2023
Fix #1011 by avoiding
the use of `crate-type = ["cdylib"]` in native builds.

It was causing some linker errors when cross-compiling from x64-linux to
aarch64-linux, I believe the root cause is
rust-lang/cargo#4133 i.e. cargo failing to
select the right target linker during cross builds.

The `cdylib` crate type is only relevant when building wasms, so this
change just lowers that functionality into the specific commands that
build wasms.

---------

Co-authored-by: Leigh McCulloch <351529+leighmcculloch@users.noreply.github.com>
@epage epage added the S-triage Status: This issue is waiting on initial triage. label Oct 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-cross-compiling Area: using --target flag for other platforms A-linkage Area: linker issues, dylib, cdylib, shared libraries, so C-feature-request Category: proposal for a feature. Before PR, ping rust-lang/cargo if this is not `Feature accepted` Command-build S-triage Status: This issue is waiting on initial triage.
Projects
None yet
Development

No branches or pull requests