build: Fail to build `libmultiprocess` for the `arm-linux-gnueabihf` target

[hebasto]

On master (a6c3da1), building of libmultiprocess fails for the arm-linux-gnueabihf target:

$ make -C depends libmultiprocess MULTIPROCESS=1 HOST=arm-linux-gnueabihf
make: Entering directory '/home/hebasto/GitHub/bitcoin/depends'
Building libmultiprocess...
make[1]: Entering directory '/home/hebasto/GitHub/bitcoin/depends/work/build/arm-linux-gnueabihf/libmultiprocess/d576d975debdc9090bd2582f83f49c76c0061698-e5741e5a04d'
make[2]: Entering directory '/home/hebasto/GitHub/bitcoin/depends/work/build/arm-linux-gnueabihf/libmultiprocess/d576d975debdc9090bd2582f83f49c76c0061698-e5741e5a04d'
make[3]: Entering directory '/home/hebasto/GitHub/bitcoin/depends/work/build/arm-linux-gnueabihf/libmultiprocess/d576d975debdc9090bd2582f83f49c76c0061698-e5741e5a04d'
[ 14%] Compiling Cap'n Proto schema include/mp/proxy.capnp
/lib/ld-linux-armhf.so.3: No such file or directory
make[3]: *** [CMakeFiles/multiprocess.dir/build.make:62: include/mp/proxy.capnp.c++] Error 255
make[3]: Leaving directory '/home/hebasto/GitHub/bitcoin/depends/work/build/arm-linux-gnueabihf/libmultiprocess/d576d975debdc9090bd2582f83f49c76c0061698-e5741e5a04d'
make[2]: *** [CMakeFiles/Makefile2:153: CMakeFiles/multiprocess.dir/all] Error 2
make[2]: Leaving directory '/home/hebasto/GitHub/bitcoin/depends/work/build/arm-linux-gnueabihf/libmultiprocess/d576d975debdc9090bd2582f83f49c76c0061698-e5741e5a04d'
make[1]: *** [Makefile:130: all] Error 2
make[1]: Leaving directory '/home/hebasto/GitHub/bitcoin/depends/work/build/arm-linux-gnueabihf/libmultiprocess/d576d975debdc9090bd2582f83f49c76c0061698-e5741e5a04d'
make: *** [funcs.mk:282: /home/hebasto/GitHub/bitcoin/depends/work/build/arm-linux-gnueabihf/libmultiprocess/d576d975debdc9090bd2582f83f49c76c0061698-e5741e5a04d/./.stamp_built] Error 2
make: Leaving directory '/home/hebasto/GitHub/bitcoin/depends'

[fanquake]

This is likely better opened upstream.

[hebasto]

This is likely better opened upstream.

It seems a makefile recipe calls target binary depends/arm-linux-gnueabihf/bin/capnp instead of native depends/arm-linux-gnueabihf/native/bin/capnp one.

[ryanofsky]

I think I'm seeing something a little different when I try to reproduce this with make -C depends libmultiprocess MULTIPROCESS=1 HOST=arm-linux-gnueabihf. The build succeeds at HEAD 3a618c1 but seems to create a package in arm-linux-gnueabihf/libmultiprocess-(...).tar.gz that has an x86-64 binary instead of an arm one.

This is likely better opened upstream.

It seems a makefile recipe calls target binary depends/arm-linux-gnueabihf/bin/capnp instead of native depends/arm-linux-gnueabihf/native/bin/capnp one.

I'm happy to help with any issues (either here or if you want to move it upstream https://github.com/chaincodelabs/libmultiprocess/issues/new)

Is there an easy to way to get depends not to delete the build/staging directories when it finishes building something? So far I couldn't check what is happening with the build because all the directories are deleted.

[hebasto]

The build succeeds at HEAD 3a618c1...

Confirming the same error when building at HEAD 3a618c1.

Is there an easy to way to get depends not to delete the build/staging directories when it finishes building something?

You could use a ..._built "subtarget" to get a building directory depends/work/build/arm-linux-gnueabihf/native_capnp/... available:

$ make -C depends native_capnp_built MULTIPROCESS=1 HOST=arm-linux-gnueabihf

and a ..._staged one to get a staging directory depends/work/staging/arm-linux-gnueabihf/native_capnp/...:

$ make -C depends native_capnp_staged MULTIPROCESS=1 HOST=arm-linux-gnueabihf

Also: https://github.com/bitcoin/bitcoin/blob/master/depends/packages.md#build-targets

[ryanofsky]

Ok. I am making some progress debugging this but I still can't reproduce the failure, and I've pretty confused about a number of things. My earlier build seemed to succeed only because I didn't have a cross compiler installed, so for some unknown reason the depends build was using the native compiler and appearing to succeed but actually building the wrong thing.

Now I've set up the cross compiler...

(details)

I'm running nix-shell --pure with shell.nix

{ nixpkgs ? import <nixpkgs> }:

let
  pkgs = nixpkgs { };
  crossPkgs = nixpkgs {
    crossSystem = {
      config = "aarch64-unknown-linux-gnu";
    };
  };
in
crossPkgs.callPackage
  ({ mkShell, cmake, curl, gcc, pkg-config }:
    mkShell {
      buildInputs = [ ];
      nativeBuildInputs = [
        cmake
        curl
        pkg-config
        pkgs.gcc
      ];

      SSL_CERT_FILE = "/etc/ssl/certs/ca-bundle.crt";
    }
  )
{ }

...and build seems to be succeeding and producing ARM output, so I can't reproduce the original error. But I also don't understand why it's succeeding. Specifically I'm seeing cross-compiled libmultiprocess package build somehow detecting the native capnproto package (in CMakeCache.txt file):

CapnProto_DIR:PATH=/.../depends/arm-linux-gnueabihf/native/lib/cmake/CapnProto

despite the fact that depends build is explicitly setting non-native lib/cmake path (in make V=1 output):

CMAKE_MODULE_PATH=/.../depends/arm-linux-gnueabihf/lib/cmake

It's a mystery to me how it is detecting the native package when CMAKE_MODULE_PATH does not include /native/

Even more strangely. I don't understand how the CMakeCache.txt also contains lines like CapnProto_kj_IMPORTED_LOCATION:FILEPATH=.../depends/arm-linux-gnueabihf/lib/libkj-0.7.0.so which point at non-native libraries. So it is somehow finding the right non-native libraries despite using the wrong native package.

Ideally the cross-compiled non-native multiprocess package should be depending on the cross-compiled non-native capnp package, so it uses the right capnp libraries. This appears to be happening in the original bug report, but is not happening for me.

In any case the depends build does appear to be buggy. I think packages/libmultiprocess.mk probably needs to be passing a -DCAPNP_EXECUTABLE=.../depends/arm-linux-gnueabihf/native/bin/capnp argument to cmake. Also the package should have dependencies on both capnp and native_capnp. I don't understand why it currently has a dependency on native_$(package). That may be causing it to work indirectly, but doesn't seem to make much sense.

It'd be helpful to know what build environment / operating system the original bug report was happening on so I can try to reproduce and fix the problem on a system that isn't mysteriously working like mine.