Indirect shared libraries built by vcpkg are not found on Linux

[Fedr]

I use vcpkg to build a number of packages on Linux as shared libraries (.so-files) using the setting

set(VCPKG_LIBRARY_LINKAGE dynamic)

in the triplet.

It works fine, but there is a problem with shared libraries that are not directly referenced by my application. For example, my program uses libblosc library, which depends on liblz4 library. So when starting the program I get a dynamic linker error. And not surprisingly, because ldd output is as follow:

libblosc.so.1 => /home/fed/vcpkg/installed/x64-linux-dynamic/lib/libblosc.so.1 (0x00007f4ada176000)
liblz4.so => not found

And libz4.so is present in the same folder with libblosc.so.1, but it is not searched there because it is not the direct dependency of the application.

Is there a way to install dynamic packages in vcpkg on Linux and make sure that just built application will find them?

[JackBoosY]

1. I think you should add a new triplet file x64-linux-dynamic.cmake and put it into VCPKG_ROOT/triplets/community, then build every port using ./vcpkg install PORT_NAME:x64-linux-dynamic:

set(VCPKG_TARGET_ARCHITECTURE x64)
set(VCPKG_CRT_LINKAGE dynamic)
set(VCPKG_LIBRARY_LINKAGE dynamic)

set(VCPKG_CMAKE_SYSTEM_NAME Linux)

2. Or add the path where the library installed(VCPKG_ROOT/installed/x64-linux/lib) to /etc/ld.so.conf then run command ldconfig .
   That's why we set to build static by default in Linux and OSX.

[Fedr]

@JackBoosY

1. Thank you, I used the same triplet before asking the initial question. And not-direct dependent .so-files could not be found without your point 2.
2. Thanks again, this method works fine. But what if I want to build both Release and Debug configurations of my program. If I add in /etc/ld.so.conf both Release and Debug folders with vcpkg built .so-files (vcpkg/installed/x64-linux-dynamic/lib and vcpkg/installed/x64-linux-dynamic/debug/lib), which in many cases have exactly same names, then I guess .so-files from incorrect configuration can be used sometimes. How this issue can be solved?

[JackBoosY]

@Fedr For question 2, if the generated debug and release library names are the same, the compiler will find the first path when adding debug/lib and lib to ldconfig. So I think this cannot be solved at present.

[Fedr]

@JackBoosY
I see. Personally I very like how vcpkg behaves on Windows, where it automatically copies all required dlls from proper configuration in the output folder of the application. It works great for both Release and Debug. Could you plan to do something similar on Linux as well?

[JackBoosY]

@Fedr There are some differences between Windows and Linux, which makes us temporarily unable to unify the behavior of the two platforms.

[JackBoosY]

And I remember @BillyONeal mentioned about this.

[BillyONeal]

Is there a way to install dynamic packages in vcpkg on Linux and make sure that just built application will find them?

In the general case, the platform fights you here; the whole system must agree on the version of any given .so installed in any of the default library paths, because under POSIX all symbols are smashed into a single process-wide symbol table. Only one entity on the system gets to say what vector::vector does, for example. That makes "follow your code" systems like vcpkg and conan unable to provide a fantastic experience for deploying shared libraries. That's why there's no triplet, even under community status, which delivers dynamic libraries on Linux right now; most of the time you have to use what comes with your distro there.

This is why the system-wide package managers dnf/yum/apt/etc. took off on Linux: without them getting to a functional system is effectively impossible.

(Contrast this with Windows where any program can choose the versions of their dependencies by copying DLLs next to their EXE; you can get much the same effect by statically linking everything which is what we do on the POSIX-ish platforms)

[Fedr]

Thanks for the detailed explanation. The reason why we use shared libraries is because our software package contains a few executables. And if all executables had been statically linked, each of them would be huge in size. So, we are forced to use shared libraries even despite the difficulties discussed here.

[Osyotr]

I wonder if I can create a small wrapper app with something like this in main(): system("LD_LIBRARY_PATH=. ./my_app")
Will it work?

[BillyONeal]

Thanks for the detailed explanation. The reason why we use shared libraries is because our software package contains a few executables. And if all executables had been statically linked, each of them would be huge in size. So, we are forced to use shared libraries even despite the difficulties discussed here.

I know this is what you want, but the POSIX platforms are designed very explicitly to not give you that.

I wonder if I can create a small wrapper app with something like this in main(): system("LD_LIBRARY_PATH=. ./my_app")
Will it work?

Using current directory would be exploitable and bad. Using relative-to-current-executable may work; I don't know the detailed nitty gritty on when dependencies are resolved in ELF land.

[Osyotr]

We could also try to use BUILD_RPATH_USE_ORIGIN cmake target property in our custom x64-linux-dynamic triplet and in project's CMakeLists.txt.
As per documentation,

On platforms that support runtime paths (RPATH) with the $ORIGIN token, setting this property to TRUE enables relative paths in the build RPATH for executables and shared libraries that point to shared libraries in the same build tree.

This is exactly what we need - relocatable and reproducible builds, same as on Windows.

[BillyONeal]

We could also try to use BUILD_RPATH_USE_ORIGIN cmake target property in our custom x64-linux-dynamic triplet and in project's CMakeLists.txt.
As per documentation,

On platforms that support runtime paths (RPATH) with the $ORIGIN token, setting this property to TRUE enables relative paths in the build RPATH for executables and shared libraries that point to shared libraries in the same build tree.
This is exactly what we need - relocatable and reproducible builds, same as on Windows.

To my understanding that only works if the relative path between your exe and the libraries is the same relative path on the build machine and on the customer box, which isn't going to be true here (since we build the libraries in different directories then you deploy them).

You might be able to use something like patchelf on deployment #15134

[Fedr]

I would like to try setting rpath=$ORIGIN in all .so-files built by vcpkg. Could you please suggest how to specify this setting in the triplet?

[BillyONeal]

I would like to try setting rpath=$ORIGIN in all .so-files built by vcpkg. Could you please suggest how to specify this setting in the triplet?

We don't have a ready-to-hand option for this. You can try yourself to see if it would work by using patchelf on the resulting binaries though.

[Osyotr]

I would like to try setting rpath=$ORIGIN in all .so-files built by vcpkg. Could you please suggest how to specify this setting in the triplet?

Not sure if you can specify it in triplet, but take a look at the toolchain file:
https://github.com/microsoft/vcpkg/blob/master/scripts/toolchains/linux.cmake

[Fedr]

I mean if the triplet is as follows:

set(VCPKG_TARGET_ARCHITECTURE x64)
set(VCPKG_CRT_LINKAGE dynamic)
set(VCPKG_LIBRARY_LINKAGE dynamic)

set(VCPKG_CMAKE_SYSTEM_NAME Linux)

set(VCPKG_CXX_FLAGS "-Wl,-rpath,'$ORIGIN'")
set(VCPKG_C_FLAGS "-Wl,-rpath,'$ORIGIN'")
set(VCPKG_LINKER_FLAGS "-Wl,-rpath,'$ORIGIN'")

Then all .so-files produced by vcpkg should search its dependencies in the same folder:

$ objdump -p installed/x64-linux-dynamic/lib/liblz4.so | grep RUNPATH
  RUNPATH              :$ORIGIN

and potentially solve the issue with indirect .so-dependencies. But probably I miss something?

[ras0219]

You can try it and report back; like @BillyONeal mentioned, this is a thorny issue that is counter to the way software is typically built and distributed for these platforms.

[Osyotr]

This is fixed in #23035