Builds fail with useless errors if user's global Cargo target-dir is changed

[alerque]

I just spent an unreasonable amount of time debugging a little app that includes Lua bindings that I was adding some trivial new features to. For whatever reason I kept getting successful builds but unusable Lua modules.

Eventually I realized that my last stable tested tag was also not building for me, but that it had built fine and even was still building fine in CI. That finally made me stop looking at what changes I'd just made to the project and look for other local changes.

Eventually I discovered that a change I made a long time back to my user's default Cargo config (in ~/.cargo/config.toml) was the culprit. It wasn't the default change to clang + mold for linking, it was the target-dir change:

[build]
target-dir = ".local/share/cargo"

This causes a bizarre error condition. The build process actually doesn't throw an error: luarocks make builds and installs an .so module as usual. The .so file has execute permissions, and looking at it with file mymodule.so shows it looks about right:

lua_modules/lib/lua/5.4/mymodule.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=45aa8d9f4722fd71bf873b361fa1b0b7d379a32c, not stripped

Trying to load and use the module however is a failure:

lua: error loading module 'mymodule' from file '/home/caleb/projects/mymodule/lua_modules/lib/lua/5.4/mymodule.so':
        /home/caleb/projects/mymodule/lua_modules/lib/lua/5.4/mymodule.so: undefined symbol: luaopen_mymodule
stack traceback:
        [C]: in ?
        [C]: in function 'require'
        (command line):1: in main chunk
        [C]: in ?

[alerque]

Having dug into this a big farther is apparent that the actual cargo build being run is using the currently configured build dir, but the copy operation being triggered on luarocks make is copying an old version out of my old cache location from some unfinished state when I didn't have things working properly.

I this the install routine needs to take into account that the cargo build dir may not be the default. Using a global target directory saves a lot of disk space and compile time when you work with a lot of Rust projects and is not an entirely uncommon way to handle CI caching either.

[khvzak]

You can specify a custom target path when configuring luarocks-build-rust-mlua via rockspec file.
Option is called target_path

[alerque]

First, sorry this is on the wrong repository. I think you have a move button in the sidebar and you can move it to the luarocks-build-rust-mlua repo.

Second, thanks for the tip, but that's a pretty unsatisfactory resolution. Of course I could configure my own project to override all user and system settings and hard code a target location, but that wouldn't be particularly friendly to other users. Worse, it isn't just my project it turns out I can't build anything that uses the rust-mlua builder on this system. I have to either change the global Cargo settings or hand-tweak all the rockspecs to override it.

I looked into a solution and it looks like this is another case of Cargo not playing nice with other build systems because it assumes it is in ultimate control of a project. There is a long running issue about being able to determine the current target directory with to clear solutions.

The current artifact build directory cat be had with some effort:

$ cargo build --quiet --message-format=json | jq -sr 'map(select(.reason == "build-script-executed")) | last | .out_dir'

... but that isn't the same as the target directory which seems to be even harder to locate.

A temporary solution might be to override the default target in the build. Instead of just using the target_path as a place to expect the output, the cargo build called could be passed a target argument to make sure it is there in the first place. Thay way Cargo is forced to build in a project-relative target directory for rockspecs that don't otherwise specify. That shouldn't break things any more than they already are and will solve most build breakage. It won't still allow user preferences to have a global target directory, but that can be fixed if/when Cargo makes the location discoverable.