ci: build GCC 14, build nowallet depends and source with it, stage built packages in /opt, allowlist LLVM libc++

[kwvg]

Motivation

This PR is a spiritual continuation of dash#5375. Unlike upstream, Dash uses a development image for CI and therefore, is limited by the packages provided by the base distro (in our case, Ubuntu 22.04 jammy) unless additional efforts are placed to make those packages available, those efforts were expended for Clang and in this PR, it is being expanded to GCC to be able to catch issues like the ones resolved in dash#5064 within CI itself.

Additional Information

• Depends on ci: merge bitcoin#27314, #28954, #29765, introduce dependency options in GitHub Actions builds, fix multiprocess builds #6400

• Dependency for backport: merge bitcoin#23060, #24164, #23565, #24337, #27682, #29208, #29091, #29165, #29934, #30261, partial bitcoin#27662, #28210, #28348, #30263 (bump minimum compiler) #6389

• While building GCC ourselves increases the build time of the container, mitigations have been made to reduce the build time.

  The following alternatives were explored before settling for building:

  • Using the Ubuntu Toolchain PPA (which provides up to GCC 12 for jammy using the ppa branch, source, and GCC 13 using test, source)
  • Using a container base that comes with GCC 14 (requiring significant retooling for CI to support using multiple images)
  • Moving the development image base to noble as GCC 14 is available for it (source) (but means we aren't testing against the lowest supported LTS).
  • Downloading pre-built binaries from GNU (they don't provide them and their binaries page lists other sources and none of them are Linux, here)
  • Using a GCC Docker container as a base (they're not based on the same LTS we're using, source)

  The following mitigations were made to reduce build time:

  • Dropping support for Obj(C++) and Fortran as we aren't expecting macOS cross compilation and we don't have Fortran code to compile
  • Dropping libsanitizer (as tsan uses Clang and ubsan uses distro-supplied GCC), libstdcxx-debug, Native Language Support (we expect CI diagnostic information to remain in English) and ISL (used for loop optimizations, no significant performance regressions found)
  • Using distro's package manager for dependencies (GMP, MPC, MPFR) instead of fetching them and building them in-tree.
  • Reducing the packages installed in first RUN layer to the minimum needed for GCC and LLVM installations and placing GCC build and install to the second RUN layer.

• As of develop (a8e2316), packages built by the container are stored in /tmp, which is inadvisable as it is the same directory used to store functional test runs and it's not too difficult to delete /tmp's contents to save space in a long running develop container and then realize that both shellcheck and cppcheck are stored there and now you have to ditch the container you're working in and restart it.

  • To remedy this, packages are now built and stored in /opt in accordance with the FHS (source). /usr/local was a contender but it's pre-populated, meanwhile ls /opt would give you a very quick picture of what's built for the container.
  • /tmp will not be entirely empty because pypa/pip#10753 results in residual .pem files leaking into /tmp and pyenv stores its build log there and keeping it around has some debug value.

• Using LD_LIBRARY_PATH to point to our compiled GCC's library files (and extending that to LLVM's library files) are acceptable and will not interfere with executing binaries built using the distro's packaged compiler as it will eventually search default paths and find the libraries shipped with the distro (source).

  • depends built using GCC 11.4 (distro's GCC) can be used to build Dash Core using GCC 14.2 (compiled GCC) but the reverse doesn't hold true.

Breaking Changes

None expected.

Checklist

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas (note: N/A)
• I have added or updated relevant unit/integration/functional/e2e tests (note: N/A)
• I have made corresponding changes to the documentation (note: N/A)
• I have assigned this pull request to a milestone (for repository code-owners and collaborators only)

[kwvg]

GitHub Actions on 62d4636 works as expected on my fork, see https://github.com/kwvg/dash/actions/runs/11764281671/job/32769805675#step:6:1

[github-actions]

This pull request has conflicts, please rebase.

[kwvg]

GitHub Actions run for 3263879, https://github.com/kwvg/dash/actions/runs/11949630923.

[knst]

I am against of building gcc everytime. It's heavy process, it make things that supposed to be close to instant to be "1 hour long".

If you really need custom gcc build - let's build it, put somewhere as a binary, and only download it and verify hashes. It's not necessary should be a build that is done by gcc's team, but it can be 3rd party repo

[knst]

gcc is default compiler in my ubuntu 24.10, I don't see a reason to build it from scratch. It's not a bleeding-edge-night-build which we need.

[knst]

maybe their's? https://github.com/xpack-dev-tools/gcc-xpack

[kwvg]

It's heavy process, it make things that supposed to be close to instant to be "1 hour long".

It doesn't take an hour, it takes ~25 minutes (build) and efforts have been made to make it cacheable as possible (as mentioned in the PR description, see below, along with all the alternatives explored, see above). Builds without the GCC compile take ~5 minutes (build), so the slowdown impact relative to the current deployment in a worst-case scenario (cache-miss) is 20 minutes and the subsequent caching should should give us pull times comparable to regular cache hits.

• Reducing the packages installed in first RUN layer to the minimum needed for GCC and LLVM installations and placing GCC build and install to the second RUN layer.

I'm not against building them separately and downloading binaries for them. What about baking in the first two steps into their own Docker container and FROM'ing it? It should bypass cache misses and the rebuilds that it would attract.

---

gcc is default compiler in my ubuntu 24.10, I don't see a reason to build it from scratch. It's not a bleeding-edge-night-build which we need.

This has been addressed in the PR description (see below)

• Moving the development image base to noble as GCC 14 is available for it (source) (but means we aren't testing against the lowest supported LTS).

---

maybe their's? https://github.com/xpack-dev-tools/gcc-xpack

I had suggested this to Pasta and he seemed lukewarm towards the idea (@PastaPastaPasta, thoughts?)

[github-actions]

This pull request has conflicts, please rebase.