linkers: don't include absolue RPATH on cross-compiling

[Ansuel]

There is currently a reproducible problem when cross-compiling with the inclusion of external shared library RPATH entry. Meson normally includes RPATH entry to permit the usage of the tool in the build process and later removes it on the intall phase. This might be ok and permits creating reproducible build to some degree when building on host (as we can expect the shared library are always placed on a standard directory path and have a consistent RPATH)

This doesn't apply for cross-compilation scenario where the shared library might be provided from an arbritrary directory to be later packed in the final system (for example a squashfs image)

On top of this on cross-compilation on 99% of the scenario, it's not really possible to run the just built tool for build usage as it probably target a different arch.

To permit building REAL reproducible binary, add extra logic to skip the inclusion of such library path in RPATH if we detect a cross-compilation scenario and limit the inclusion of library path in RPATH only to relative path (expected to be the ones specific to the building binary/internal shared library)

[eli-schwartz]

On top of this on cross-compilation on 99% of the scenario, it's not really possible to run the just built tool for build usage as it probably target a different arch.

Assuming that nobody falls into the 1% case is an absolute deal breaker. At minimum, this PR cannot even be considered unless it continues to use rpaths when cross compiling but the resulting executables are runnable.

More generally, Meson was explicitly designed to support this use case via exe_wrapper set in your cross file, and APIs such as meson.can_run_host_binaries().

It is absolutely vital to have such support, if one wishes to allow users to for example run testsuites during cross builds. It also allows things such as running cross-built compiletests (cc.run() and suchlike).

While I would like to aid reproducibility too, it is NEVER acceptable to prevent a subset of users from being able to build at all, just for the sake of a different subset benefiting from reproducible builds.

[eli-schwartz]

The environment object tracks both the need for, and the presence of, an exe wrapper. See e.g.

meson/mesonbuild/interpreter/mesonmain.py

Lines 267 to 275 in 019d960

	def can_run_host_binaries_method(self, args: T.List['TYPE_var'], kwargs: 'TYPE_kwargs') -> bool:
	return self._can_run_host_binaries_impl()
	def _can_run_host_binaries_impl(self) -> bool:
	return not (
	self.build.environment.is_cross_build() and
	self.build.environment.need_exe_wrapper() and
	self.build.environment.exe_wrapper is None
	)

We could use that to narrow the impact of this change to an acceptable level. I'm still interested in opinions from other team members regarding whether it feels right to use different compile arguments based on this, though.

[Ansuel]

@eli-schwartz thanks a lot for the comments. The alternative might be to introduce a toggle to disable adding the RPATH in build stage but I don't want to follow that path...

Thanks for pointing out _can_run_host_binaries_impl

In the meantime I will update this to use that instead of the simple is_cross_build().

I honestly think it's a good compromise for this.

[eli-schwartz]

@eli-schwartz thanks a lot for the comments. The alternative might be to introduce a toggle to disable adding the RPATH in build stage but I don't want to follow that path...

Right, this would allow every person building something to make a personal decision "do I want tests to work or do I want to avoid build time rpaths". But the downside is that people have to choose. :D

I know that @jpakkane is nervous to add too many built-in options as it tends to clutter up the ./configure --help / meson configure sourcedir/ UX, so that path might be a bit of a far reach anyway...

[Ansuel]

@eli-schwartz I updated the thing to use the helper (I also had to move it) What do you think?

[Rondom]

Just because I do not have an exe wrapper and just because I am cross compiling does not mean that I am not interested in running the binaries as is during development. With binfmt_misc I can run all kinds of binaries without any issues, especially on a distribution that supports multi-arch like Debian.

[dnicolodi]

There is something that I don't grok. The usual way to build binary packages is to run DESTDIR=dist/$package-$version meson install and archive the content of dist/$package-$version. Meson removes the build RPATH entries when running meson install. Therefore, there should be no reproducibility problems due to build RPATH. This works just the same for cross builds, I believe. Where is the problem that this patch tries to solve coming from?

[eli-schwartz]

Just because I do not have an exe wrapper and just because I am cross compiling does not mean that I am not interested in running the binaries as is during development. With binfmt_misc I can run all kinds of binaries without any issues, especially on a distribution that supports multi-arch like Debian.

Per the discussion above, my point about supporting cross exe_wrapper was specifically about "APIs such as can_run_host_binaries".

This is explicitly designed to respect the union of "is cross AND needs exe_wrapper AND NOT has exe_wrapper". Your use of binfmt_misc aligns with point two, so you should not be affected by this PR as a result of the fact that I requested to transition from "is cross" to "same impl as the DSL function meson.can_run_host_binaries()".

Does that answer your concern?

[Rondom]

Does that answer your concern?

Yes, thanks.

[eli-schwartz]

Meson removes the build RPATH entries when running meson install. Therefore, there should be no reproducibility problems due to build RPATH. This works just the same for cross builds, I believe. Where is the problem that this patch tries to solve coming from?

The linker produces different output (sometimes) depending on the rpath, for example because it sees it can deduplicate string constants that appear in two places. Consequently, when we strip the build rpath we try to be unambitious, and zero out only enough to mark the rest of the space as unused, rather than zeroing out the whole thing.

@Ansuel had another PR already to try to zero out a bit more smartly, which should make binaries a bit more reproducible though still not guaranteed.

[Ansuel]

@eli-schwartz you want me to rebase this so the ci all runs green ? (the macos failing is unrelated)