Get rid of historic bloat on the build pipeline

[antondlr]

Description

We currently do a lot of different builds (esp. on the docker front) which slows down CI pipelines without any clear benefit.
Additionally, we publish a lot of different docker image tags as part of the build pipeline.

Tangentially related is this issue about the docker pipeline being started more often than necessary, resulting in even more CI overhead.

Present Behaviour

Release binaries are typically built with portable and modern blst config, which has been redundant since august 2023, when we introduced runtime cpu feature detection to the portable builds.

On the docker front, another flavour in built for every spec (-dev), which only includes spec-minimal

This results on these builds being released:

Binaries:
- lighthouse-v5.1.1-x86_64-apple-darwin.tar.gz
- lighthouse-v5.1.1-x86_64-apple-darwin-portable.tar.gz
- lighthouse-v5.1.1-x86_64-unknown-linux-gnu.tar.gz
- lighthouse-v5.1.1-x86_64-unknown-linux-gnu-portable.tar.gz
- lighthouse-v5.1.1-aarch64-unknown-linux-gnu.tar.gz
- lighthouse-v5.1.1-aarch64-unknown-linux-gnu-portable.tar.gz
- lighthouse-v5.1.1-x86_64-windows.tar.gz
- lighthouse-v5.1.1-x86_64-windows-portable.tar.gz

Docker: 
- v5.1.1
- v5.1.1-modern
- v5.1.1-amd64
- v5.1.1-amd64-dev
- v5.1.1-amd64-modern
- v5.1.1-amd64-modern-dev
- v5.1.1-arm64
- v5.1.1-arm64-dev
- v5.1.1-arm64-modern
- v5.1.1-arm64-modern-dev

Expected Behaviour

• make portable the default and get rid of portable and modern tags, as this should always give you (very close to?) optimal performance, with fallback in place for older systems.
• add the spec-minimal profile to regular docker builds and get rid of -dev
• stop publishing architecture-specific docker images, only publish proper multiarch images
• (build and add an aarch64 lcli image to the lcli image manifest)

The releases would thus be cut down to:

Binaries:
- lighthouse-v5.1.1-x86_64-apple-darwin.tar.gz
- lighthouse-v5.1.1-x86_64-unknown-linux-gnu.tar.gz
- lighthouse-v5.1.1-aarch64-unknown-linux-gnu.tar.gz

Docker: 
- v5.1.1

Why now?

As we have just had a hard fork, this feels like the safest timing to push out changes to releases and docker images: missing out on an update or two because of hard-coded specificity (eg. latest-amd64-unstable-dev) won't be as impactful as it would be right before a fork.
However, we should still tread carefully, eg. by keeping an eye on actual downloads (releases) and pulls (docker images).

[sebastianst]

@antondlr Any progress on this? Would be great to have a multi-arch image ready that includes the spec-minimal! I'm currently trying to set up lighthouse in CI (ethereum-optimism/optimism#9117) and right now we custom-build the minimal spec lighthouse node like

FROM rust:1.77.2-bookworm AS builder
RUN apt-get update && apt-get -y upgrade && apt-get install -y cmake libclang-dev

RUN git clone --depth 1 --branch v5.1.3 https://github.com/sigp/lighthouse.git

ARG FEATURES=spec-minimal
ARG PROFILE=release
ARG CARGO_USE_GIT_CLI=true
ENV FEATURES $FEATURES
ENV PROFILE $PROFILE
ENV CARGO_NET_GIT_FETCH_WITH_CLI=$CARGO_USE_GIT_CLI
RUN cd lighthouse && make

FROM ubuntu:22.04
RUN apt-get update && apt-get -y upgrade && apt-get install -y --no-install-recommends \
  libssl-dev \
  ca-certificates \
  && apt-get clean \
  && rm -rf /var/lib/apt/lists/*
COPY --from=builder /usr/local/cargo/bin/lighthouse /usr/local/bin/lighthouse

COPY l1-lighthouse-bn-entrypoint.sh /entrypoint-bn.sh
COPY l1-lighthouse-vc-entrypoint.sh /entrypoint-vc.sh

VOLUME ["/db"]

Also tried

ARG TARGETARCH

FROM sigp/lighthouse:v5.1.3-${TARGETARCH}-modern-dev

COPY l1-lighthouse-bn-entrypoint.sh /entrypoint-bn.sh
COPY l1-lighthouse-vc-entrypoint.sh /entrypoint-vc.sh

VOLUME ["/db"]

but that didn't work well either.

[sebastianst]

I just saw #5614 I guess that would be fixing this issue?

[antondlr]

I just saw #5614 I guess that would be fixing this issue?

yeah I think merging #5614 would fix your issue.
in the meantime you could re-use our -dev images to build your own multiarch manifest from, but I think we're about ready to merge the above mentioned PR as our docker pipeline is currently broken and this should fix it.

[sebastianst]

Thanks @antondlr I see that #5614 is currently being merged. Are you gonna publish multi-arch docker images that include spec-minimal already later today once it got merged?

[michaelsproul]

@sebastianst Yeah it's out as :latest-unstable:

docker run sigp/lighthouse:latest-unstable lighthouse --version
Lighthouse v5.1.3-c8ffafb
BLS library: blst-portable
SHA256 hardware acceleration: false
Allocator: jemalloc
Profile: maxperf
Specs: mainnet (true), minimal (true), gnosis (true)

After the next release, the :latest tag will be v5.2.0 with the same features