Using x86_64-unknown-linux-musl as a target for a module

[chipsenkbeil]

This is just an open question and I think the answer is no. Is it possible in some way to build a Lua module with musl as the target? If you try to build a cdylib, you get the immediate notification that the musl target doesn't support that crate type.

You can use a staticlib crate type and it builds fine, but to my knowledge you cannot import *.a or *.lib (windows) using module require(...) in Lua. I've seen scattered mentioning of baking static libraries into programs that also embed lua like Lua EPICS module, but no mention for Lua 5.1+ or luajit about using them.

With my lack of knowledge in the space, I'm assuming it doesn't make sense to link to a static library. The only reason this popped up for me is that some musl-based systems like Alpine linux don't have the needed libraries like ld-linux-x86-64.so for Lua to be able to load the module when built with the gnu target. This isn't a dealbreaker for me, but is a limitation of my move to a Lua module for distant.nvim.

[khvzak]

Hmm, it should be possible. You probably need to build it inside docker on alpine image.
Resulted library will be dynamically linked to a musl c runtime.
Or with cross compilation you need musl gcc toolchain. The error message you got probably means that you cannot produce musl dynamic library using glibc toolchain (I can be wrong).

rust-lang/rust#40113

[chipsenkbeil]

Looks like it should be possible with this feature enabled, but it still requires libgcc as well: rust-lang/rust#82521

export RUSTFLAGS="-C target-feature=-crt-static"
cargo build --release --target x86_64-unknown-linux-musl

[chipsenkbeil]

It compiles and loads, which is great! I'm getting runtime errors right now, but I'm going to assume that this is user error on my part and am closing this out. If I dig in deeper and this turns out to be something else, I'll reopen.

In case this does work just fine, you might want to add a note in the readme that you'll need the rust flags above as otherwise cargo rejects trying to build the module with a crate type of cdylib.

[khvzak]

Yeah, I have plan to publish even more comprehensive doc than readme/docs.rs

[chipsenkbeil]

@khvzak this may still not be compatible without installing a full-blown glibc on a musl-based system like Alpine Linux.

On Alpine, I installed libc6-compat to get around ld-linux-x86-64.so.2 missing. I am still missing the symbol __register_atfork. All of the lua ones I'm assuming are fine because of the nature of the lua module, but reading online I'm not sure how __register_atfork comes into play but it seems to be a glibc feature, unless I'm mistaken.

~/app $ ldd lua/distant_lua.so
        /lib/ld-musl-x86_64.so.1 (0x4000000000)
        libc.so.6 => /lib/ld-musl-x86_64.so.1 (0x4000000000)
        libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x4001fe2000)
Error relocating lua/distant_lua.so: __register_atfork: symbol not found
Error relocating lua/distant_lua.so: lua_getmetatable: symbol not found
Error relocating lua/distant_lua.so: lua_isstring: symbol not found
// ... bunch more lua symbols not found

[chipsenkbeil]

Okay, actually, it looks like if you build on the native machine - or in my case within the alpine docker container - the __register_atfork is not required nor is ld-linux-x86-64.so.2. So it does load and work in that case.

So something about the cross build using target x86_64-unknown-linux-musl from a GNU system bakes in unneeded symbols and libraries. I don't know if this is an mlua issue or a rustc issue. It seems like it would be rust itself doing this and not something that mlua would contribute.

[khvzak]

@chipsenkbeil I think you still should be able to build a Lua module on x86_64-glibc machine, but using musl toolchain.
You might need to export CC env variable and/or set linker program: https://doc.rust-lang.org/cargo/reference/config.html#targettriplelinker

Toolchain: https://musl.cc/x86_64-linux-musl-cross.tgz

[chipsenkbeil]

@chipsenkbeil I think you still should be able to build a Lua module on x86_64-glibc machine, but using musl toolchain. You might need to export CC env variable and/or set linker program: https://doc.rust-lang.org/cargo/reference/config.html#targettriplelinker

Toolchain: https://musl.cc/x86_64-linux-musl-cross.tgz

I've give that a try. I'm building both the gnu and musl Lua modules on the same Github runner. I had assumed that the change in target would also change the cc and linker used!

[khvzak]

@chipsenkbeil
Just tried the following command on x86_64 ubuntu

RUSTFLAGS="-C target-feature=-crt-static -C linker=musl-cc" cargo build --target x86_64-unknown-linux-musl

and it produces library without requiring ld-linux-x86-64.so.2

Unfortunately by default Rust uses just cc linker which is gnu 😞

Could be related to rust-lang/cargo#4133

[chipsenkbeil]

Closing this out as I think I finally got my deployment working with the extra flags per your rec. This is also out of scope for mlua and is actually a rust/cargo challenge. :) Appreciate all of your help here even though this wasn't an mlua problem!