Skip to content

Native C (bindgen) dependent crates do not compile? #53

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
brainstorm opened this issue Jan 30, 2020 · 11 comments
Closed

Native C (bindgen) dependent crates do not compile? #53

brainstorm opened this issue Jan 30, 2020 · 11 comments

Comments

@brainstorm
Copy link

I wrongly filed this issue on rust-htslib: rust-bio/rust-htslib#179

After a bit more digging, I now believe it belongs here. I've put together a small reproducer using the serverless-aws-rust-multi template, just adding an additional lambda function called htslib:

https://github.com/brainstorm/serverless-aws-rust-multi-fail-native-crate

There's a full explanation and error log on the README from the repo above, but TL;DR the lamda-rust docker container doesn't seem to compile/link the rust-htslib crate with C bindings well:

  = note: /code/target/lambda/release/deps/librust_htslib-fc86895fccfb6420.rlib(rust_htslib-fc86895fccfb6420.rust_htslib.25lsx2fx-cgu.11.rcgu.o): In function `rust_htslib::bam::Reader::new::h7805fb2441edac2e':
          rust_htslib.25lsx2fx-cgu.11:(.text._ZN11rust_htslib3bam6Reader3new17h7805fb2441edac2eE+0x4c): undefined reference to `sam_hdr_read'
          /code/target/lambda/release/deps/librust_htslib-fc86895fccfb6420.rlib(rust_htslib-fc86895fccfb6420.rust_htslib.25lsx2fx-cgu.11.rcgu.o): In function `_$LT$rust_htslib..bam..Reader$u20$as$u20$core..ops..drop..Drop$GT$::drop::hc749ab5efb9aea68':
          rust_htslib.25lsx2fx-cgu.11:(.text._ZN66_$LT$rust_htslib..bam..Reader$u20$as$u20$core..ops..drop..Drop$GT$4drop17hc749ab5efb9aea68E+0x5): undefined reference to `hts_close'
          /code/target/lambda/release/deps/librust_htslib-fc86895fccfb6420.rlib(rust_htslib-fc86895fccfb6420.rust_htslib.25lsx2fx-cgu.11.rcgu.o): In function `rust_htslib::bam::hts_open::hac1f0f64c9261566':
          rust_htslib.25lsx2fx-cgu.11:(.text._ZN11rust_htslib3bam8hts_open17hac1f0f64c9261566E+0xc3): undefined reference to `hts_open'
          /code/target/lambda/release/deps/librust_htslib-fc86895fccfb6420.rlib(rust_htslib-fc86895fccfb6420.rust_htslib.25lsx2fx-cgu.11.rcgu.o): In function `_$LT$rust_htslib..bam..HeaderView$u20$as$u20$core..ops..drop..Drop$GT$::drop::h06ec3494656e04e9':
          rust_htslib.25lsx2fx-cgu.11:(.text._ZN70_$LT$rust_htslib..bam..HeaderView$u20$as$u20$core..ops..drop..Drop$GT$4drop17h06ec3494656e04e9E+0xb): undefined reference to `bam_hdr_destroy'
          collect2: error: ld returned 1 exit status
@brainstorm
Copy link
Author

@softprops trying to figure out this as I go...

After jumping into your container bypassing the entrypoint to interactively try to figure this out:

$ docker run --entrypoint /bin/bash -it --rm -v "${PWD}":/code -v "${HOME}"/.cargo/registry:/root/.cargo/registry -v "${HOME}"/.cargo/git:/root/.cargo/git softprops/lambda-rust

And also installing musl-gcc from source and defining it on the .cargo/config file, also making sure $PATH, $LD_LIBRARY_PATH and the appropriate rust musl toolchain is installed:

bash-4.2# cat .cargo/config
[target.x86_64-unknown-linux-musl]
linker = "musl-gcc"

bash-4.2# echo $LD_LIBRARY_PATH
/lib64:/usr/lib64:/var/runtime:/var/runtime/lib:/var/task:/var/task/lib:/opt/lib:/usr/local/musl/lib/

bash-4.2# echo $PATH
/root/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/local/musl/bin/

bash-4.2# rustup show
Default host: x86_64-unknown-linux-gnu
rustup home:  /root/.rustup

installed targets for active toolchain
--------------------------------------

x86_64-unknown-linux-gnu
x86_64-unknown-linux-musl

active toolchain
----------------

1.39.0-x86_64-unknown-linux-gnu (default)
rustc 1.39.0 (4560ea788 2019-11-04)

I'm still having compilation issues right after calling /usr/local/bin/build.sh:

   Compiling native-crate v0.1.0 (/code)
     Running `rustc --edition=2018 --crate-name bootstrap src/main.rs --color always --crate-type bin --emit=dep-info,link -C opt-level=3 -C metadata=8ddd4fa8dae4d2e9 -C extra-filename=-8ddd4fa8dae4d2e9 --out-dir /code/target/lambda/release/deps -L dependency=/code/target/lambda/release/deps --extern lambda_runtime=/code/target/lambda/release/deps/liblambda_runtime-27fab337c1bdc137.rlib --extern rust_htslib=/code/target/lambda/release/deps/librust_htslib-bcc50205832ee1f9.rlib --extern serde_json=/code/target/lambda/release/deps/libserde_json-2c4c2dff3bd38626.rlib -L native=/code/target/lambda/release/build/backtrace-sys-fc8bac11650b2558/out -L native=/code/target/lambda/release/build/rust-htslib-fa5f6feb29f349fb/out -L native=/code/target/lambda/release/build/bzip2-sys-6de061f326041bbb/out/lib -L native=/usr/lib64 -L native=/usr/lib64`
error: linking with `cc` failed: exit code: 1
  |
  = note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-L" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "/code/target/lambda/release/deps/bootstrap-8ddd4fa8dae4d2e9.bootstrap.54d9wc2c-cgu.0.rcgu.o" "/code/target/lambda/release/deps/bootstrap-8ddd4fa8dae4d2e9.bootstrap.54d9wc2c-cgu.1.rcgu.o" "/code/target/lambda/release/deps/bootstrap-8ddd4fa8dae4d2e9.bootstrap.54d9wc2c-cgu.10.rcgu.o" "/code/target/lambda/release/deps/bootstrap-8ddd4fa8dae4d2e9.bootstrap.54d9wc2c-cgu.11.rcgu.o" "/code/target/lambda/release/deps/bootstrap-8ddd4fa8dae4d2e9.bootstrap.54d9wc2c-cgu.12.rcgu.o" "/code/target/lambda/release/deps/bootstrap-8ddd4fa8dae4d2e9.bootstrap.54d9wc2c-cgu.13.rcgu.o" "/code/target/lambda/release/deps/bootstrap-8ddd4fa8dae4d2e9.bootstrap.54d9wc2c-cgu.14.rcgu.o" "/code/target/lambda/release/deps/bootstrap-8ddd4fa8dae4d2e9.bootstrap.54d9wc2c-cgu.15.rcgu.o" "/code/target/lambda/release/deps/bootstrap-8ddd4fa8dae4d2e9.bootstrap.54d9wc2c-cgu.2.rcgu.o" "/code/target/lambda/release/deps/bootstrap-8ddd4fa8dae4d2e9.bootstrap.54d9wc2c-cgu.3.rcgu.o" "/code/target/lambda/release/deps/bootstrap-8ddd4fa8dae4d2e9.bootstrap.54d9wc2c-cgu.4.rcgu.o" "/code/target/lambda/release/deps/bootstrap-8ddd4fa8dae4d2e9.bootstrap.54d9wc2c-cgu.5.rcgu.o" "/code/target/lambda/release/deps/bootstrap-8ddd4fa8dae4d2e9.bootstrap.54d9wc2c-cgu.6.rcgu.o" "/code/target/lambda/release/deps/bootstrap-8ddd4fa8dae4d2e9.bootstrap.54d9wc2c-cgu.7.rcgu.o" "/code/target/lambda/release/deps/bootstrap-8ddd4fa8dae4d2e9.bootstrap.54d9wc2c-cgu.8.rcgu.o" "/code/target/lambda/release/deps/bootstrap-8ddd4fa8dae4d2e9.bootstrap.54d9wc2c-cgu.9.rcgu.o" "-o" "/code/target/lambda/release/deps/bootstrap-8ddd4fa8dae4d2e9" "/code/target/lambda/release/deps/bootstrap-8ddd4fa8dae4d2e9.25qdi9xkjjurrkc6.rcgu.o" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro" "-Wl,-znow" "-Wl,-O1" "-nodefaultlibs" "-L" "/code/target/lambda/release/deps" "-L" "/code/target/lambda/release/build/backtrace-sys-fc8bac11650b2558/out" "-L" "/code/target/lambda/release/build/rust-htslib-fa5f6feb29f349fb/out" "-L" "/code/target/lambda/release/build/bzip2-sys-6de061f326041bbb/out/lib" "-L" "/usr/lib64" "-L" "/usr/lib64" "-L" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/code/target/lambda/release/deps/librust_htslib-bcc50205832ee1f9.rlib" "/code/target/lambda/release/deps/libitertools-f3835078cb76931d.rlib" "/code/target/lambda/release/deps/libieee754-31e358538504232c.rlib" "/code/target/lambda/release/deps/liburl-179761af4a843e2e.rlib" "/code/target/lambda/release/deps/libpercent_encoding-d985814ec59e21ba.rlib" "/code/target/lambda/release/deps/libidna-09885cc799774535.rlib" "/code/target/lambda/release/deps/libunicode_normalization-d3bc7c76467693c2.rlib" "/code/target/lambda/release/deps/libsmallvec-86eeb486a55644d2.rlib" "/code/target/lambda/release/deps/libunicode_bidi-127b6dea613aa070.rlib" "/code/target/lambda/release/deps/libmatches-656cd79ac1f4dbf3.rlib" "/code/target/lambda/release/deps/libbio_types-b85dae10730c49ea.rlib" "/code/target/lambda/release/deps/libquick_error-b99603485db0852f.rlib" "/code/target/lambda/release/deps/libregex-665dd09806e4b8a1.rlib" "/code/target/lambda/release/deps/libthread_local-17d2289b727a4eaf.rlib" "/code/target/lambda/release/deps/libregex_syntax-37aa91eae5563f01.rlib" "/code/target/lambda/release/deps/libaho_corasick-acfc8bae7e8559a6.rlib" "/code/target/lambda/release/deps/libmemchr-d011937b0e4fb3d0.rlib" "/code/target/lambda/release/deps/liblinear_map-1f78a0ead911ef1d.rlib" "/code/target/lambda/release/deps/libsnafu-9e5d9767888a08f9.rlib" "/code/target/lambda/release/deps/libdoc_comment-c167071d06a64bad.rlib" "/code/target/lambda/release/deps/liblzma_sys-e276e8159637c177.rlib" "/code/target/lambda/release/deps/libbzip2_sys-4d39fc001894c114.rlib" "/code/target/lambda/release/deps/liblibz_sys-9312e74b5a83b9ef.rlib" "/code/target/lambda/release/deps/libnewtype_derive-3845a05c4ec64475.rlib" "/code/target/lambda/release/deps/libcustom_derive-ef5f5fe80dbc7d21.rlib" "/code/target/lambda/release/deps/liblambda_runtime-27fab337c1bdc137.rlib" "/code/target/lambda/release/deps/liblambda_runtime_core-df16ac691298aa33.rlib" "/code/target/lambda/release/deps/liblambda_runtime_client-73f0441cfb6cdffc.rlib" "/code/target/lambda/release/deps/liblambda_runtime_errors-8c2c068a0e221d52.rlib" "/code/target/lambda/release/deps/libserde_json-2c4c2dff3bd38626.rlib" "/code/target/lambda/release/deps/libryu-250abc56353e4f1a.rlib" "/code/target/lambda/release/deps/libserde-9f4f577279d3e721.rlib" "/code/target/lambda/release/deps/libhyper-125f447ab210c65c.rlib" "/code/target/lambda/release/deps/libwant-15b092e4a3d6a406.rlib" "/code/target/lambda/release/deps/libtry_lock-c3ab9ae9b8d10427.rlib" "/code/target/lambda/release/deps/libtokio-51eb75322ff49ed5.rlib" "/code/target/lambda/release/deps/libtokio_uds-ac3d1375e77cdab9.rlib" "/code/target/lambda/release/deps/libmio_uds-122f123f3493ba50.rlib" "/code/target/lambda/release/deps/libtokio_udp-68c9ee31ef46c7ca.rlib" "/code/target/lambda/release/deps/libtokio_timer-fd26a1d177e6bc24.rlib" "/code/target/lambda/release/deps/libtokio_tcp-a3a69ae7fadc0dc1.rlib" "/code/target/lambda/release/deps/libtokio_reactor-f887653bad0767b2.rlib" "/code/target/lambda/release/deps/libtokio_sync-6584ad6d9169f707.rlib" "/code/target/lambda/release/deps/libparking_lot-d4f4cb497402b6e0.rlib" "/code/target/lambda/release/deps/libparking_lot_core-199ecaa26af352ff.rlib" "/code/target/lambda/release/deps/libsmallvec-3ab86593794904ed.rlib" "/code/target/lambda/release/deps/libmaybe_uninit-97fb638f02759584.rlib" "/code/target/lambda/release/deps/liblock_api-961f64fbadb40ac0.rlib" "/code/target/lambda/release/deps/libtokio_fs-757693a9b6aec707.rlib" "/code/target/lambda/release/deps/libtokio_threadpool-eaee2c7ff5c16f7d.rlib" "/code/target/lambda/release/deps/libcrossbeam_queue-55b75d73ff737e2c.rlib" "/code/target/lambda/release/deps/libcrossbeam_deque-b50237f16fd358a6.rlib" "/code/target/lambda/release/deps/libcrossbeam_epoch-3f4a3d0aaca987b4.rlib" "/code/target/lambda/release/deps/libscopeguard-fbfb32c1cb4c3e17.rlib" "/code/target/lambda/release/deps/libmemoffset-3e9f6971dbc8f79d.rlib" "/code/target/lambda/release/deps/libcrossbeam_utils-2dd24291656fd3d0.rlib" "/code/target/lambda/release/deps/libtokio_current_thread-6336e318f5198a48.rlib" "/code/target/lambda/release/deps/libtokio_executor-61f9fe3a74555f64.rlib" "/code/target/lambda/release/deps/libcrossbeam_utils-a3643d7f61cb12cf.rlib" "/code/target/lambda/release/deps/liblazy_static-9927bdb5a51d8866.rlib" "/code/target/lambda/release/deps/libtokio_codec-2a3f0b825288ebc9.rlib" "/code/target/lambda/release/deps/libmio-b298f150c206468c.rlib" "/code/target/lambda/release/deps/libnet2-2589aab84683a3d2.rlib" "/code/target/lambda/release/deps/libhttparse-b16dabd259454e4e.rlib" "/code/target/lambda/release/deps/libhttp_body-761e17f7607fe575.rlib" "/code/target/lambda/release/deps/libtokio_buf-07ee66150e59350e.rlib" "/code/target/lambda/release/deps/libh2-de4820524e8e71f0.rlib" "/code/target/lambda/release/deps/libindexmap-2850a2884328a439.rlib" "/code/target/lambda/release/deps/libstring-23f38e319ee741ea.rlib" "/code/target/lambda/release/deps/libslab-aca7457c7f2adc8c.rlib" "/code/target/lambda/release/deps/libhttp-1086d8ded4dc06f2.rlib" "/code/target/lambda/release/deps/libitoa-8b90c0416aaa1224.rlib" "/code/target/lambda/release/deps/libfnv-e24d8f06a947b9e8.rlib" "/code/target/lambda/release/deps/libtokio_io-01eb7221776e4529.rlib" "/code/target/lambda/release/deps/liblog-5f075c2f94d34499.rlib" "/code/target/lambda/release/deps/libfutures_cpupool-7f433ef6768b85f9.rlib" "/code/target/lambda/release/deps/libnum_cpus-d4e5e60783eef22d.rlib" "/code/target/lambda/release/deps/libfutures-d6d90b26a1db9345.rlib" "/code/target/lambda/release/deps/libbytes-984ca03b6bac9032.rlib" "/code/target/lambda/release/deps/libeither-49105ea47ad0d133.rlib" "/code/target/lambda/release/deps/libiovec-761b16d27e507b1c.rlib" "/code/target/lambda/release/deps/libbyteorder-8dbacdfd1cfce70a.rlib" "/code/target/lambda/release/deps/libchrono-a8a297d27436a7c5.rlib" "/code/target/lambda/release/deps/libnum_integer-1a510640678c8078.rlib" "/code/target/lambda/release/deps/libnum_traits-e9f7da71647d3cfe.rlib" "/code/target/lambda/release/deps/libtime-6ce88697e34eb71f.rlib" "/code/target/lambda/release/deps/libfailure-e25c0d9e0fc5e625.rlib" "/code/target/lambda/release/deps/libbacktrace-a5dd1de208b875e9.rlib" "/code/target/lambda/release/deps/libbacktrace_sys-797c63915e78100b.rlib" "/code/target/lambda/release/deps/liblibc-c1af9ba80fb01a25.rlib" "/code/target/lambda/release/deps/libcfg_if-f51889ee12a5a31e.rlib" "/code/target/lambda/release/deps/librustc_demangle-2518b9cc69d3ccce.rlib" "-Wl,--start-group" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-fae576517123aa4e.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-a72070139220275e.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-093434daf7d99801.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-24daf38551b7a03b.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libbacktrace-36d70d9746402ce9.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libbacktrace_sys-7acfc843240167a8.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-eb2e0f5fe057b8b3.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-75e9ddd83715a368.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-af51e7c6fd7d1248.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-27f2a77b2995d98c.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-ad10152c26711a1e.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-291bd2456cb6c9fe.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-fc6e9071307a3016.rlib" "-Wl,--end-group" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-ebe4001ded7f33e7.rlib" "-Wl,-Bdynamic" "-llzma" "-lz" "-lutil" "-lutil" "-ldl" "-lrt" "-lpthread" "-lgcc_s" "-lc" "-lm" "-lrt" "-lpthread" "-lutil" "-lutil"
  = note: /code/target/lambda/release/deps/librust_htslib-bcc50205832ee1f9.rlib(rust_htslib-bcc50205832ee1f9.rust_htslib.ei7zbp53-cgu.11.rcgu.o): In function `rust_htslib::bam::Reader::new::h022909085da44515':
          rust_htslib.ei7zbp53-cgu.11:(.text._ZN11rust_htslib3bam6Reader3new17h022909085da44515E+0x4c): undefined reference to `sam_hdr_read'
          /code/target/lambda/release/deps/librust_htslib-bcc50205832ee1f9.rlib(rust_htslib-bcc50205832ee1f9.rust_htslib.ei7zbp53-cgu.11.rcgu.o): In function `_$LT$rust_htslib..bam..Reader$u20$as$u20$core..ops..drop..Drop$GT$::drop::h865ba254bac3c94f':
          rust_htslib.ei7zbp53-cgu.11:(.text._ZN66_$LT$rust_htslib..bam..Reader$u20$as$u20$core..ops..drop..Drop$GT$4drop17h865ba254bac3c94fE+0x5): undefined reference to `hts_close'
          /code/target/lambda/release/deps/librust_htslib-bcc50205832ee1f9.rlib(rust_htslib-bcc50205832ee1f9.rust_htslib.ei7zbp53-cgu.11.rcgu.o): In function `rust_htslib::bam::hts_open::h1c03eca264c6dd16':
          rust_htslib.ei7zbp53-cgu.11:(.text._ZN11rust_htslib3bam8hts_open17h1c03eca264c6dd16E+0xc3): undefined reference to `hts_open'
          /code/target/lambda/release/deps/librust_htslib-bcc50205832ee1f9.rlib(rust_htslib-bcc50205832ee1f9.rust_htslib.ei7zbp53-cgu.11.rcgu.o): In function `_$LT$rust_htslib..bam..HeaderView$u20$as$u20$core..ops..drop..Drop$GT$::drop::h920a039de3d51cf0':
          rust_htslib.ei7zbp53-cgu.11:(.text._ZN70_$LT$rust_htslib..bam..HeaderView$u20$as$u20$core..ops..drop..Drop$GT$4drop17h920a039de3d51cf0E+0xb): undefined reference to `bam_hdr_destroy'
          collect2: error: ld returned 1 exit status


error: aborting due to previous error

error: could not compile `native-crate`.

Caused by:
  process didn't exit successfully: `rustc --edition=2018 --crate-name bootstrap src/main.rs --color always --crate-type bin --emit=dep-info,link -C opt-level=3 -C metadata=8ddd4fa8dae4d2e9 -C extra-filename=-8ddd4fa8dae4d2e9 --out-dir /code/target/lambda/release/deps -L dependency=/code/target/lambda/release/deps --extern lambda_runtime=/code/target/lambda/release/deps/liblambda_runtime-27fab337c1bdc137.rlib --extern rust_htslib=/code/target/lambda/release/deps/librust_htslib-bcc50205832ee1f9.rlib --extern serde_json=/code/target/lambda/release/deps/libserde_json-2c4c2dff3bd38626.rlib -L native=/code/target/lambda/release/build/backtrace-sys-fc8bac11650b2558/out -L native=/code/target/lambda/release/build/rust-htslib-fa5f6feb29f349fb/out -L native=/code/target/lambda/release/build/bzip2-sys-6de061f326041bbb/out/lib -L native=/usr/lib64 -L native=/usr/lib64` (exit code: 1)

I also wonder how does this container build for musl since I cannot seem to find a compiler in there:

bash-4.2# find / -iname "musl*"
/root/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.48/src/unix/notbsd/linux/musl
/root/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.58/src/unix/notbsd/linux/musl
/root/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.62/src/unix/linux_like/linux/musl
/root/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.66/src/unix/linux_like/linux/musl

So far I have been unable to reproduce my host musl setup where this bindgen crate compiles successfully with the accompanying musl rust toolchain. Even after following the experience and fixing from Andrew Thorburn step by step :/

Any ideas on how to fix this are much appreciated!

@brainstorm
Copy link
Author

In a nutshell, after getting into the container I do:

bash-4.2# wget https://www.musl-libc.org/releases/musl-1.1.24.tar.gz
bash-4.2# tar xvfz musl-1.1.24.tar.gz && cd musl-1.1.24 && ./configure --prefix=/usr
bash-4.2# make install
bash-4.2# . $HOME/.cargo/env
bash-4.2# rustup target add x86_64-unknown-linux-musl

Minimal changes to your build.sh script to see what's going on:

diff --git a/build.sh b/build.sh
index 39320c3..b72d1f0 100755
--- a/build.sh
+++ b/build.sh
@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/bin/bash -x
 # build and pack a rust lambda library
 # https://aws.amazon.com/blogs/opensource/rust-runtime-for-aws-lambda/

@@ -23,7 +23,7 @@ export CARGO_TARGET_DIR=$PWD/target/lambda
     # cargo only supports --release flag for release
     # profiles. dev is implicit
     if [ "${PROFILE}" == "release" ]; then
-        cargo build ${CARGO_FLAGS:-} --${PROFILE}
+        cargo build ${CARGO_FLAGS:-} --${PROFILE} --verbose --target x86_64-unknown-linux-musl
     else
         cargo build ${CARGO_FLAGS:-}
     fi

Also slightly modifying the .cargo/config like so:

bash-4.2# cat .cargo/config
[target.x86_64-unknown-linux-musl]
linker = "musl-gcc"

bash-4.2# musl-gcc -v
Using built-in specs.
COLLECT_GCC=musl-gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-amazon-linux/4.8.5/lto-wrapper
Target: x86_64-amazon-linux
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-linker-hash-style=gnu --enable-languages=c,c++,fortran,ada,lto --enable-plugin --enable-initfini-array --disable-libgcj --with-isl=/builddir/build/BUILD/gcc-4.8.5-20150702/obj-x86_64-amazon-linux/isl-install --with-cloog=/builddir/build/BUILD/gcc-4.8.5-20150702/obj-x86_64-amazon-linux/cloog-install --enable-gnu-indirect-function --with-tune=generic --with-arch_32=x86-64 --build=x86_64-amazon-linux
Thread model: posix
gcc version 4.8.5 20150623 (Red Hat 4.8.5-28) (GCC)

But still I'm getting the same linker error:

bash-4.2# /usr/local/bin/build.sh
(...)
   Compiling native-crate v0.1.0 (/code)
     Running `rustc --edition=2018 --crate-name bootstrap src/main.rs --color always --crate-type bin --emit=dep-info,link -C opt-level=3 -C metadata=c09e81091113ceee -C extra-filename=-c09e81091113ceee --out-dir /code/target/lambda/x86_64-unknown-linux-musl/release/deps --target x86_64-unknown-linux-musl -C linker=musl-gcc -L dependency=/code/target/lambda/x86_64-unknown-linux-musl/release/deps -L dependency=/code/target/lambda/release/deps --extern lambda_runtime=/code/target/lambda/x86_64-unknown-linux-musl/release/deps/liblambda_runtime-f67913905ac57255.rlib --extern rust_htslib=/code/target/lambda/x86_64-unknown-linux-musl/release/deps/librust_htslib-b637035c49b1a5be.rlib --extern serde_json=/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libserde_json-4aaf4c8a05fff3e3.rlib -L native=/code/target/lambda/x86_64-unknown-linux-musl/release/build/backtrace-sys-b90cf17db8f96223/out -L native=/code/target/lambda/x86_64-unknown-linux-musl/release/build/rust-htslib-9d1845320f5d5251/out -L native=/code/target/lambda/x86_64-unknown-linux-musl/release/build/bzip2-sys-d3dbc0e0edc3367f/out/lib -L native=/code/target/lambda/x86_64-unknown-linux-musl/release/build/libz-sys-e4cd506607550e41/out/build -L native=/code/target/lambda/x86_64-unknown-linux-musl/release/build/lzma-sys-ecc564711531c7ab/out`
error: linking with `musl-gcc` failed: exit code: 1
  |
  = note: "musl-gcc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-Wl,--eh-frame-hdr" "-m64" "-nostdlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/crt1.o" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/crti.o" "-L" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/bootstrap-c09e81091113ceee.bootstrap.350xf6zk-cgu.0.rcgu.o" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/bootstrap-c09e81091113ceee.bootstrap.350xf6zk-cgu.1.rcgu.o" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/bootstrap-c09e81091113ceee.bootstrap.350xf6zk-cgu.10.rcgu.o" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/bootstrap-c09e81091113ceee.bootstrap.350xf6zk-cgu.11.rcgu.o" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/bootstrap-c09e81091113ceee.bootstrap.350xf6zk-cgu.12.rcgu.o" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/bootstrap-c09e81091113ceee.bootstrap.350xf6zk-cgu.13.rcgu.o" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/bootstrap-c09e81091113ceee.bootstrap.350xf6zk-cgu.14.rcgu.o" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/bootstrap-c09e81091113ceee.bootstrap.350xf6zk-cgu.15.rcgu.o" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/bootstrap-c09e81091113ceee.bootstrap.350xf6zk-cgu.2.rcgu.o" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/bootstrap-c09e81091113ceee.bootstrap.350xf6zk-cgu.3.rcgu.o" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/bootstrap-c09e81091113ceee.bootstrap.350xf6zk-cgu.4.rcgu.o" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/bootstrap-c09e81091113ceee.bootstrap.350xf6zk-cgu.5.rcgu.o" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/bootstrap-c09e81091113ceee.bootstrap.350xf6zk-cgu.6.rcgu.o" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/bootstrap-c09e81091113ceee.bootstrap.350xf6zk-cgu.7.rcgu.o" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/bootstrap-c09e81091113ceee.bootstrap.350xf6zk-cgu.8.rcgu.o" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/bootstrap-c09e81091113ceee.bootstrap.350xf6zk-cgu.9.rcgu.o" "-o" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/bootstrap-c09e81091113ceee" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/bootstrap-c09e81091113ceee.18ps1a3vyeatpeo1.rcgu.o" "-Wl,--gc-sections" "-Wl,-zrelro" "-Wl,-znow" "-Wl,-O1" "-nodefaultlibs" "-L" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps" "-L" "/code/target/lambda/release/deps" "-L" "/code/target/lambda/x86_64-unknown-linux-musl/release/build/backtrace-sys-b90cf17db8f96223/out" "-L" "/code/target/lambda/x86_64-unknown-linux-musl/release/build/rust-htslib-9d1845320f5d5251/out" "-L" "/code/target/lambda/x86_64-unknown-linux-musl/release/build/bzip2-sys-d3dbc0e0edc3367f/out/lib" "-L" "/code/target/lambda/x86_64-unknown-linux-musl/release/build/libz-sys-e4cd506607550e41/out/build" "-L" "/code/target/lambda/x86_64-unknown-linux-musl/release/build/lzma-sys-ecc564711531c7ab/out" "-L" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib" "-Wl,-Bstatic" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/librust_htslib-b637035c49b1a5be.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libitertools-0adbc7df7ba9e123.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libieee754-5385444ee034100e.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/liburl-718a7a699a45c951.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libpercent_encoding-1d02589b6fa1539b.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libidna-644bc09fb2eef7a5.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libunicode_normalization-f2c5a5a59dd05fa8.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libsmallvec-be68e2e066da690a.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libunicode_bidi-e0e8ce70249f0737.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libmatches-eb6284dd5ab65a1a.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libbio_types-e3916382872f4b4e.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libquick_error-f93e48fab40521fb.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libregex-917c8ad7f095040d.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libthread_local-513e66e11ab87e34.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libregex_syntax-98d8fe806225a73d.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libaho_corasick-1e69bb7e2f0d9a0f.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libmemchr-ab8e3b78bbe2ef0c.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/liblinear_map-5920f1fbce864418.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libsnafu-1be79f2c0a29c0c4.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libdoc_comment-05c590f2380d6e56.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/liblzma_sys-e77ea915e8374fc4.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libbzip2_sys-39a2d3c44ca007f8.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/liblibz_sys-01e3689ec7f22ad1.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libnewtype_derive-1df4bcbbd4fa1247.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libcustom_derive-6b8f32da9a9c4370.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/liblambda_runtime-f67913905ac57255.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/liblambda_runtime_core-41f20cdf63ddf4ea.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/liblambda_runtime_client-fc4587683e4b07ea.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/liblambda_runtime_errors-7c089b84aa4275a7.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libserde_json-4aaf4c8a05fff3e3.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libryu-987ef2dda5c6f778.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libserde-98e01ca354f09109.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libhyper-18b922604e83e035.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libwant-4f15c6558de19741.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libtry_lock-934813bed979e45d.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libtokio-1076b3b928ad1de2.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libtokio_uds-98b2c70c7d70d3bd.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libmio_uds-45c6deb0003b1fb1.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libtokio_udp-78e96e191fdb8ba5.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libtokio_timer-5f509f98990d834d.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libtokio_tcp-a4cb5534a2b66863.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libtokio_reactor-bf9c57b45dad280c.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libtokio_sync-94b2a49ac7f86adb.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libparking_lot-d47c79816b7edfc2.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libparking_lot_core-7d2517817f8b48e4.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libsmallvec-7a1227ea644e34ac.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libmaybe_uninit-06867833f5552452.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/liblock_api-1d2190fb34295118.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libtokio_fs-db922e1a0f1cd4f3.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libtokio_threadpool-15a60e948291663a.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libcrossbeam_queue-76e1a06118769a10.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libcrossbeam_deque-17ef97ca7b7a18f9.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libcrossbeam_epoch-10ea9d0cfab31c58.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libscopeguard-6471871a291a00e0.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libmemoffset-6c2846be1364f0d5.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libcrossbeam_utils-7a31f82dafc9b1d9.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libtokio_current_thread-20db4bd0c4bd2ffe.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libtokio_executor-7c7995e38bac15f7.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libcrossbeam_utils-81e5934fca514bde.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/liblazy_static-1f4d2eef09ed0ef1.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libtokio_codec-b4ce414f1cc5e324.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libmio-35c5d665a3eea53b.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libnet2-1b63793e76f45a0e.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libhttparse-5d6a0a834e87223a.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libhttp_body-04e2c1a918f27550.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libtokio_buf-55d317dfd4008a0f.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libh2-b736bad7ff3f4c7e.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libindexmap-e43c7100c42db5fb.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libstring-9d55d6eed7b9933c.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libslab-aa99ddab00ce326d.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libhttp-e3b4012509ec4098.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libitoa-adfe67026705b0ca.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libfnv-6e971795abcf6dcf.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libtokio_io-66bea987e85433e7.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/liblog-99f30788bf397d66.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libfutures_cpupool-1869c953122299be.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libnum_cpus-18c0fa0adc89e949.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libfutures-cb8d3190919b79fd.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libbytes-e9f5a04d94067fde.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libeither-04b311798f8b4fed.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libiovec-cfe638311ced4040.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libbyteorder-921a841a7c18fa8a.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libchrono-46d42887a8d74a8e.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libnum_integer-06884dfb52a07e27.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libnum_traits-d5e621b703df68b8.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libtime-6e47603988718b8e.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libfailure-7d4747bba423ab3c.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libbacktrace-da7b1cb6efd46fd6.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libbacktrace_sys-88da53095bf9b393.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/liblibc-f9604e2813c04637.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libcfg_if-e8d51f745108570b.rlib" "/code/target/lambda/x86_64-unknown-linux-musl/release/deps/librustc_demangle-9347e7181e80d080.rlib" "-Wl,--start-group" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libstd-7fd334f8b7e9cf07.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libpanic_unwind-3282d6e06289cb43.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libhashbrown-34b1c084e3a06420.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/librustc_std_workspace_alloc-c2838718aeef790e.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libbacktrace-eec5c4608c035255.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libbacktrace_sys-47a1cb5b4581d258.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/librustc_demangle-0e599b39e6834088.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libunwind-11110c4ac3057173.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libcfg_if-9ebcdce602679acd.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/liblibc-704cdfabaaa7cafb.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/liballoc-750cac2ce57672f5.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/librustc_std_workspace_core-d25378b74b806141.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libcore-2a42e14cbaeecc7d.rlib" "-Wl,--end-group" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libcompiler_builtins-a60d2b4f619641d7.rlib" "-static" "-Wl,-Bdynamic" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/crtn.o"
  = note: /code/target/lambda/x86_64-unknown-linux-musl/release/deps/librust_htslib-b637035c49b1a5be.rlib(rust_htslib-b637035c49b1a5be.rust_htslib.6guecgfh-cgu.11.rcgu.o): In function `rust_htslib::bam::Reader::new::ha66f4c29a21a332d':
          rust_htslib.6guecgfh-cgu.11:(.text._ZN11rust_htslib3bam6Reader3new17ha66f4c29a21a332dE+0x4c): undefined reference to `sam_hdr_read'
          /code/target/lambda/x86_64-unknown-linux-musl/release/deps/librust_htslib-b637035c49b1a5be.rlib(rust_htslib-b637035c49b1a5be.rust_htslib.6guecgfh-cgu.11.rcgu.o): In function `_$LT$rust_htslib..bam..Reader$u20$as$u20$core..ops..drop..Drop$GT$::drop::h270694aecc1be41f':
          rust_htslib.6guecgfh-cgu.11:(.text._ZN66_$LT$rust_htslib..bam..Reader$u20$as$u20$core..ops..drop..Drop$GT$4drop17h270694aecc1be41fE+0x5): undefined reference to `hts_close'
          /code/target/lambda/x86_64-unknown-linux-musl/release/deps/librust_htslib-b637035c49b1a5be.rlib(rust_htslib-b637035c49b1a5be.rust_htslib.6guecgfh-cgu.11.rcgu.o): In function `rust_htslib::bam::hts_open::h7c0946902255d455':
          rust_htslib.6guecgfh-cgu.11:(.text._ZN11rust_htslib3bam8hts_open17h7c0946902255d455E+0xc3): undefined reference to `hts_open'
          /code/target/lambda/x86_64-unknown-linux-musl/release/deps/librust_htslib-b637035c49b1a5be.rlib(rust_htslib-b637035c49b1a5be.rust_htslib.6guecgfh-cgu.11.rcgu.o): In function `_$LT$rust_htslib..bam..HeaderView$u20$as$u20$core..ops..drop..Drop$GT$::drop::h7184d16dc054274c':
          rust_htslib.6guecgfh-cgu.11:(.text._ZN70_$LT$rust_htslib..bam..HeaderView$u20$as$u20$core..ops..drop..Drop$GT$4drop17h7184d16dc054274cE+0xb): undefined reference to `bam_hdr_destroy'
          collect2: error: ld returned 1 exit status


error: aborting due to previous error

error: could not compile `native-crate`.

Caused by:
  process didn't exit successfully: `rustc --edition=2018 --crate-name bootstrap src/main.rs --color always --crate-type bin --emit=dep-info,link -C opt-level=3 -C metadata=c09e81091113ceee -C extra-filename=-c09e81091113ceee --out-dir /code/target/lambda/x86_64-unknown-linux-musl/release/deps --target x86_64-unknown-linux-musl -C linker=musl-gcc -L dependency=/code/target/lambda/x86_64-unknown-linux-musl/release/deps -L dependency=/code/target/lambda/release/deps --extern lambda_runtime=/code/target/lambda/x86_64-unknown-linux-musl/release/deps/liblambda_runtime-f67913905ac57255.rlib --extern rust_htslib=/code/target/lambda/x86_64-unknown-linux-musl/release/deps/librust_htslib-b637035c49b1a5be.rlib --extern serde_json=/code/target/lambda/x86_64-unknown-linux-musl/release/deps/libserde_json-4aaf4c8a05fff3e3.rlib -L native=/code/target/lambda/x86_64-unknown-linux-musl/release/build/backtrace-sys-b90cf17db8f96223/out -L native=/code/target/lambda/x86_64-unknown-linux-musl/release/build/rust-htslib-9d1845320f5d5251/out -L native=/code/target/lambda/x86_64-unknown-linux-musl/release/build/bzip2-sys-d3dbc0e0edc3367f/out/lib -L native=/code/target/lambda/x86_64-unknown-linux-musl/release/build/libz-sys-e4cd506607550e41/out/build -L native=/code/target/lambda/x86_64-unknown-linux-musl/release/build/lzma-sys-ecc564711531c7ab/out` (exit code: 1)

What am I doing wrong? :/

@brainstorm
Copy link
Author

brainstorm commented Feb 1, 2020

Hello @nlhepler!

I just noticed your recently merged hts-sys packaging for rust-htslib PR and while reading the cargo sys section I was wondering if rust-htslib needs a links= parameter for the crate above to compile successfully?

Also @clux refers to other *-sys crates on a related issue... is there an easy way to determine what rust-htslib requires from the error message above? I seriously doubt that LLVM/clang is required since I've compiled the native samtools htslib in the past with GCC...?

I'm new to rust-htslib and bindgen and *-sys crates and its packaging intricacies, so any directions/suggestions on how to fix this are very appreciated :)

@nlhepler
Copy link

nlhepler commented Feb 1, 2020

Hi @brainstorm!

We did! Exciting times. I'm also very new to these things, but I was also trying to follow the guidelines. My understanding is that the -sys library that links the system library needs to specify links=, and indeed, the hts-sys package does specify links: https://github.com/rust-bio/rust-htslib/blob/master/hts-sys/Cargo.toml#L6. My understanding is that rust-htslib does not need links=, it only needs to depend on an hts-sys.

@nlhepler
Copy link

nlhepler commented Feb 1, 2020

LLVM being required may be part of the linker rust uses. I am not entirely sure, I have seen these errors from time to time myself, but I've always had llvm-config to add to PATH.

@brainstorm
Copy link
Author

Thanks @nlhepler for the pointers! Yesterday I managed to fix the muslrust container that was also failing for rust-htslib with a single apt-get install clang line:

clux/muslrust#66 (comment)

I'll try to import the fix to this lambda-rust container as it's better integrated with serverless.com framework.

@brainstorm
Copy link
Author

brainstorm commented Feb 2, 2020

Aaand, problem solved! I just had to do this inside the container:

yum remove clang && yum install clang6.0

Since the installed clang/llvm is 3.6 but htslib seems to require clang 3.9 minimum.

Next up is pullrequest and a testcase, thanks everyone!

@johansmitsnl
Copy link

For me it failed here:

thread 'main' panicked at 'failed to execute ["musl-gcc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "include" "-Wall" "-Wextra" "-pedantic" "-pedantic-errors" "-Wall" "-Wextra" "-Wcast-align" "-Wcast-qual" "-Wconversion" "-Wenum-compare" "-Wfloat-equal" "-Wformat=2" "-Winline" "-Winvalid-pch" "-Wmissing-field-initializers" "-Wmissing-include-dirs" "-Wredundant-decls" "-Wshadow" "-Wsign-compare" "-Wsign-conversion" "-Wundef" "-Wuninitialized" "-Wwrite-strings" "-fno-strict-aliasing" "-fvisibility=hidden" "-fstack-protector" "-g3" "-U_FORTIFY_SOURCE" "-DNDEBUG" "-c" "-o/builds/race-passport/serverless/target/x86_64-unknown-linux-musl/release/build/ring-8813b0c2d3b38690/out/aesni-x86_64-elf.o" "/root/.cargo/registry/src/github.com-1ecc6299db9ec823/ring-0.16.15/pregenerated/aesni-x86_64-elf.S"]: No such file or directory (os error 2)', /root/.cargo/registry/src/github.com-1ecc6299db9ec823/ring-0.16.15/build.rs:661:9

Is this the same issue or a other one?

@brainstorm
Copy link
Author

@johansmitsnl Check ring-0.16.15/build.rs:661:9, but it doesn't seem to be related to this issue :-S

@johansmitsnl
Copy link

I found that musl-gcc is not installed in the image. Installing makes it build. Why is musl-gcc not available?

@softprops
Copy link
Owner

Why is musl-gcc not available

The goal here is to leverage a build environment that aligns as closly with the runtime as possible.

We could add musl-gcc to the build environment but we might start walking the path of assuming environmental dependencies that won't exist in the actual lambda runtime.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants