build: add ability to interactively run guix builds, use container that has unprivileged user

[kwvg]

Additional Notes

• Not everything can be retained on volumes, /gnu/store and /var/guix are pre-populated when extracting the Guix archive, which is done during the image building process.
  • Guix also highly cautions against modifying the contents of the store by any means outside manipulating it through the Guix daemon (see https://guix.gnu.org/en/manual/en/guix.html#The-Store)
  • This is enforced by means of bind-mounting /gnu/store as read-only for anything that isn't the daemon
• This container also has the Xcode SDK archive hardcoded in (see guix-start) and will need to be updated if the SDK version is updated
• The container runs as privileged process and all operations are being done as the root user (no unprivileged user exists within the container due to its barebones nature) No longer true, as we're not inheriting the Alpine container. An unprivileged user is used for git-specific operations with a configurable UID/GID pair which should be matched with the host user.
• This container implicitly trusts https://bordeaux.guix.gnu.org and https://ci.guix.gnu.org (see entrypoint) as it fetches substitutions from it to avoid re-building unpatched/unmodified packages by fetching them from a remote server.
  • This can be disabled at the expense of significantly longer build times (to note, a lot of build outputs are discarded in between instances of this container as not everything is or can be cached, so this is recommended against)

Motivation

@PastaPastaPasta was interested in the possibility of being able to run Guix builds as it is run by CI interactively with the fewest commands possible. Since my personal devenv is entirely defined using Docker, I had some experience with interactive containers.

We initially tried using the Alpine container (and earlier versions of this PR built upon this container) already available in develop but two problems arose:

• build-push-action@v2 is not particularly cooperative with edrevo/dockerfile-plus (which is how we get the INCLUDE+ macro).
  • All Docker builds are context-dependent (when building a container, you need to define a "base" context from which all relative paths are interpreted), dockerfile-plus doubly so. Your context directory is loaded into the builder and therefore, it is imperative to keep the context as minimal as possible, at the risk of increased resource consumption and longer build times.
  • build-push-action would not cooperate despite efforts accepting the tradeoff and passing the repository root as the context
• Simultaneously, we realized permissions issues were starting to arise with backport: merge bitcoin#25643, #23583, #23618, #23603, #23817, #24520, #24508, #25639, #26018, #25437, #25484, #23585, #24549, #24733, #21991, #22526, #25633, #24597, #24955, #25099, #24552, #21851, #25389, #25357, partial #22318 (guix backports: part 3) #5426, after backporting guix: bump time-machine to 998eda3067c7d21e0d9bb3310d2f5a14b8f1c681 bitcoin/bitcoin#25099, which is an important pre-requisite for subsequent backports that introduce Apple Silicon target support.
  • The Alpine container proved too barebones and introducing support for it would require changes to it that would future backports. Hence, it was decided to leave it as a vestigial entry while we use our own Docker container.
  • The container here intentionally does not inherit anything from our base containers to avoid using dockerfile-plus

The context problem still persisted after moving to the Ubuntu-based container (which matches the rest of our CI) but Docker implements something called "additional contexts" that allow you to do things like COPY operations with a different directory as the base.

[kwvg]

I rebased #5426 on top of this PR and it resolved the safe.directory problem (see build for more information)

[PastaPastaPasta]

Makes me sad this had to get moved over to ubuntu, instead of alpine, but probably fine. Started my build, let's make sure it works, seems good so far

[PastaPastaPasta]

For some reason my arm dash-qt binary doesn't match... everything else matches

b94461d685321ea0d6add4f2b07ae3e9da434f69ef4f5be66fecce819cb32d86  guix-build-01c022e6e244/distsrc-01c022e6e244-arm-linux-gnueabihf/src/qt/dash-qt

[PastaPastaPasta]

arm-linux-gnueabihf dash-qt is non-deterministic...

CI had

b94461d685321ea0d6add4f2b07ae3e9da434f69ef4f5be66fecce819cb32d86  guix-build-01c022e6e244/distsrc-01c022e6e244-arm-linux-gnueabihf/src/qt/dash-qt

I have (twice)

ca34664ebdcaec0915dc26900131eb2a1581f3e7fef7e116e7e9cc22bdd40b9c  guix-build-01c022e6e244/distsrc-01c022e6e244-arm-linux-gnueabihf/src/qt/dash-qt

[knst]

I got 2 binaries: b94461d685321ea0d6add4f2b07ae3e9da434f69ef4f5be66fecce819cb32d86 from my build and ca34664ebdcaec0915dc26900131eb2a1581f3e7fef7e116e7e9cc22bdd40b9c from pasta.

There's difference in disassembled code of function double_conversion::FastFixedDtoa(double, int, double_conversion::Vector<char>, int*, int*)

@kittywhiskers need to backport this one: https://github.com/bitcoin/bitcoin/pull/25643/files
Otherwise builds are non-deterministic now. @PastaPastaPasta be informed

note for myself (and for further cases): need to install gcc-arm-none-eabi to make disassembler arm-none-eabi-objdump available under amd64's linux.

[PastaPastaPasta]

We can probably move forward with this PR then and backport that in the next set of guix backports

[PastaPastaPasta]

utACK for squash merge

[UdjinM6]

utACK

[knst]

I still have couple extra comments that I accidentally put to wrong PR.

1. can you avoid code duplication between contrib/containers/guix/scripts/guix-check and .github/workflows/guix-build.yml?
2. instead hard-coded 32 jobs better to get environment variable: -e ADDITIONAL_GUIX_COMMON_FLAGS='--max-jobs=32' \ by one of the ways: you can use $(nproc --all)" or "$(grep -c ^processor /proc/cpuinfo)" or "$(getconf _NPROCESSORS_ONLN)"

[PastaPastaPasta]

Good thoughts, but I'm gonna merge this so we can move forward with guix part 3; feel free to do a PR with those changes

[kwvg]

The reason why we cannot de-duplicate is because both guix-check and guix-build are "run-anywhere" and so they cd to the workspace mount in the script, if not rely on absolute paths altogether. I feel like for an interactive instance which is supposed to be as-few-steps-as-possible (it's why I made those scripts and had a pseudo-motd printed out), we have to account for operator error and think one step ahead.

We could de-duplicate it but that would require the scripts to be modified to be relative, which defeats the point of embedding them into PATH since they won't "run-anywhere", which makes the interactive prompt harder to use.

As for binding ADDITIONAL_GUIX_COMMON_FLAGS, to nproc --all, I'm fine with it.