RFE: introduce an rpm-controlled per-build directory to builds

[pmatilai]

The current build directory setup is suboptimal in many ways: %_builddir can be shared among multiple packages so there's a possibility for conflicts, cleanup is not straightforward, and OTOH %buildsubdir is dependent on %setup and so rpm cannot rely on it existing at all times.

We should introduce a fully rpm controlled intermediate directory between these two. This gives us a place to safely unpack multiple sources, put per-build temporary rpm-level data without disturbing the pristine sources in the build directory (there's a ticket on that somewhere), including but not limited to eg debuginfo file manifests, gemspec files, place spec parts (thinking dynamic spec generation) and whatnot other context, such as the buildroot directory. It's also trivial to clean up. It would be also easier to share for remote execution (cross-compilation etc), especially if we wrote the build scriptlets to the build directory instead of %_tmpddir.

I know I'm missing a known use-case or two in the above, will add when I remember. Easier to augment an existing ticket...

[voxik]

This ticket was triggered by the .gemspec file in #2076 and therefore I'd like to share here additional issues we typically face packaging rubygem- packages:

1. Using %setup, we typically expand not just the "source" gem package, but also additional sources, such as test suite. We don't want to expand the sources into %buildsubdir for similar reasons as the .gemspec is not expanded there. On top of that, we don't want to mangle the source code in any way to be on the safe side, because sometimes the .gemspec might include files by wildcards and we want to prevent including random files.

2. %build section is calling gem install command and that installs the sources under the %buildsubdir. This is again unfortunate, but we don't have better place. Ages ago, the gem install was called in %install section installing directly into %{buildroot}, but that was abandoned, because the installation can involve also building of sources, which would not be appropriate for %install section.

3. I think that wherever feasible, the out-of-source build should be used, to keep the sources pristine. RPM provides little help to this. There is %{_vpath_builddir}, but it is subdirectory of %buildsubdir if I am not mistaken. IOW if the build output was split from the sources, that would be very nice.

[ffesti]

The question here is whether we can just point %_builddir to this new directory during build or if we need a new name. The former will keep all spec files working unchanged and just move very package to a save place. Otoh specs relying on having a save name space might cause problems when being build with an older rpm version.

[pmatilai]

This keeps coming up in various contexts. It's time we actually do it.

[Conan-Kudo]

Couldn't we just make %setup -c the default behavior and then strip the top level directory automatically when extracting an archive? (Yes, I realize this is a variation of #2664, but it's a relevant solution!)

[ffesti]

See also https://bugzilla.redhat.com/show_bug.cgi?id=1479198 which is a similar issue also including debuginfo not working outside of %buildsubdir

[pmatilai]

Couldn't we just make %setup -c the default behavior and then strip the top level directory automatically when extracting an archive?

This directory needs to be set up regardless of whether a spec uses %setup or not. And, extracting the sources to the topmost directory would defeat some of the reasons we're doing this to begin with, as explained in the ticket description.

[pmatilai]

Added buildroot itself to the description, there's no reason whatsoever to scatter the built content around multiple directories if we have that one top-level directory to rule it all.

[pmatilai]

Vpath builds added...

[pmatilai]

Making remote execution (cross-compilation) easier added (inspired by #2884)

[hobbes1069]

I found my way here from the Fedora Developers Matrix chat... After reading through everything a couple of times I get the need for the change at a high level, but it completely broke quilt for me as it doesn't know about the change. Any pointers?