Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

musl-cross-make cross compiling and specifying "--with-libc=/custom/path/to/libc.so" is dismissed #97842

Open
tlaurion opened this issue Oct 4, 2022 · 3 comments
Labels
build The build process and cross-build type-bug An unexpected behavior, bug, or error

Comments

@tlaurion
Copy link

tlaurion commented Oct 4, 2022

Bug report

Trying to integrate CPython 3.9.2 under Heads firmware (coreboot+linux payload). This is followup of old integration issue at linuxboot/heads#689

CPython 3.9.2 (as all other modules under Heads but coreboot) is built by first building musl-cross-make and then having modules being build with musl-cross-make to have reproducible build environment, so that coreboot payload' linux kernel's initrd contains binaries needed in linux embedded environment (recovery shell and associated tools), are reproducible (this is the goal).

Your environment

  • CPython versions tested on: 3.9.2 installed to build target 3.9.2 python
  • Operating system and architecture: debian-11 host build environement

I'm experimenting with the following build recipe, and did a lot of variation without success https://github.com/osresearch/heads/blob/df50c5f7d242b502fc443157fd592b638da87af0/modules/python#L13-L33.

This results in configure script producing the following config.log output:
config.log

So above build recipe produces:

  • a correct ./configure callm specifying ./configure --enable-optimizations --with-cxx-main=/home/user/heads/crossgcc/x86/bin/x86_64-linux-musl-gcc --with-libc=/home/user/heads/crossgcc/x86/x86_64-linux-musl/lib/libc.so --host x86_64-linux-musl --build x86_64 --disable-ipv6 --prefix= --exec-prefix=
  • --with-libc="$(musl-cross_libraries)" translates into config.log, correctly, into configure:14435: result: set LIBC="/home/user/heads/crossgcc/x86/x86_64-linux-musl/lib/libc.so". This is the libc.so we need to rely on
  • Unfortunately, the build fails (See in CircleCI logs here) with libc being completely dismissed. This results into error : ImportError: /usr/lib/x86_64-linux-gnu/libc.so: invalid ELF header. This is the wrong libc.so of the host, not of the cross built environment.

Expected behavior:

  • Specifying --with-libc=/home/user/heads/crossgcc/x86/x86_64-linux-musl/lib/libc.so should result in libc.so used from /home/user/heads/crossgcc/x86/x86_64-linux-musl/lib/libc.so not /usr/lib/x86_64-linux-gnu/libc.so.

Why is the host libc used even when libc is specified and found at configure?


Was gonna investigate and adapt OpenWrt patches at https://github.com/openwrt/packages/tree/master/lang/python/python3/patches but reading about cross build issues it would be better to have this fixed upstream then downstream into all projects trying to cross build for their embedded systems.

Any insight?

@tlaurion tlaurion added the type-bug An unexpected behavior, bug, or error label Oct 4, 2022
@tlaurion
Copy link
Author

tlaurion commented Oct 4, 2022

Already tried to apply OpenWrt patches, but they won't work, since they do the builds under buildroot, but the resulting patch behavior might work in their use case since no other libc would be found under buildroot path discovered. Here, we want to specify CXX, CC LIBC and have those respected and used.

In the current issue, we try to force configure to point to the right libc, and the builds should not deviate from what is configured through ./configure --enable-optimizations --with-cxx-main=/home/user/heads/crossgcc/x86/bin/x86_64-linux-musl-gcc --with-libc=/home/user/heads/crossgcc/x86/x86_64-linux-musl/lib/libc.so --host x86_64-linux-musl --build x86_64 --disable-ipv6 --prefix= --exec-prefix=

Again, pointing to config.log directly if that helps for quicker response.

@tlaurion
Copy link
Author

tlaurion commented Oct 4, 2022

@tiran I have looked into other musl related issues like #95855 and #87278 which are similar and for which fixes should be included into 3.9.2 as I understood it. Yet, the issue is still present and configure scripts are doing the wrong thing.

@erlend-aasland
Copy link
Contributor

@tlaurion, FYI: #107221 has now been merged into main; it will not be backported, though.

@iritkatriel iritkatriel added the build The build process and cross-build label Nov 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build The build process and cross-build type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

3 participants