Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 93 additions & 10 deletions .github/workflows/build-container.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,15 +18,16 @@ on:
outputs:
path:
description: "Path to built container"
value: ghcr.io/${{ jobs.build.outputs.repo }}/${{ inputs.name }}:${{ jobs.build.outputs.tag }}
value: ghcr.io/${{ jobs.build-amd64.outputs.repo }}/${{ inputs.name }}:${{ jobs.build-amd64.outputs.tag }}

jobs:
build:
name: Build container
build-amd64:
name: Build container (amd64)
runs-on: ubuntu-24.04
outputs:
tag: ${{ steps.prepare.outputs.tag }}
repo: ${{ steps.prepare.outputs.repo }}
digest: ${{ steps.build.outputs.digest }}
steps:
- name: Checkout code
uses: actions/checkout@v4
Expand All @@ -38,8 +39,8 @@ jobs:
run: |
BRANCH_NAME=$(echo "${GITHUB_REF##*/}" | tr '[:upper:]' '[:lower:]')
REPO_NAME=$(echo "${{ github.repository }}" | tr '[:upper:]' '[:lower:]')
echo "tag=${BRANCH_NAME}" >> $GITHUB_OUTPUT
echo "repo=${REPO_NAME}" >> $GITHUB_OUTPUT
echo "tag=${BRANCH_NAME}" >> "$GITHUB_OUTPUT"
echo "repo=${REPO_NAME}" >> "$GITHUB_OUTPUT"

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
Expand All @@ -52,17 +53,99 @@ jobs:
password: ${{ secrets.GITHUB_TOKEN }}

- name: Build and push Docker image
id: build
uses: docker/build-push-action@v6
with:
context: ${{ inputs.context }}
file: ${{ inputs.file }}
push: true
platforms: linux/amd64
tags: |
ghcr.io/${{ steps.prepare.outputs.repo }}/${{ inputs.name }}:${{ hashFiles(inputs.file) }}
ghcr.io/${{ steps.prepare.outputs.repo }}/${{ inputs.name }}:${{ steps.prepare.outputs.tag }}
ghcr.io/${{ steps.prepare.outputs.repo }}/${{ inputs.name }}:latest
ghcr.io/${{ steps.prepare.outputs.repo }}/${{ inputs.name }}:${{ hashFiles(inputs.file) }}-amd64
cache-from: |
type=registry,ref=ghcr.io/${{ steps.prepare.outputs.repo }}/${{ inputs.name }}:${{ hashFiles(inputs.file) }}
type=registry,ref=ghcr.io/${{ steps.prepare.outputs.repo }}/${{ inputs.name }}:${{ hashFiles(inputs.file) }}-amd64
type=registry,ref=ghcr.io/${{ steps.prepare.outputs.repo }}/${{ inputs.name }}:${{ steps.prepare.outputs.tag }}
type=registry,ref=ghcr.io/${{ steps.prepare.outputs.repo }}/${{ inputs.name }}:latest
cache-to: type=inline

build-arm64:
name: Build container (arm64)
runs-on: ubuntu-24.04-arm
outputs:
digest: ${{ steps.build.outputs.digest }}
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}

- name: Prepare variables
id: prepare
run: |
BRANCH_NAME=$(echo "${GITHUB_REF##*/}" | tr '[:upper:]' '[:lower:]')
REPO_NAME=$(echo "${{ github.repository }}" | tr '[:upper:]' '[:lower:]')
echo "tag=${BRANCH_NAME}" >> "$GITHUB_OUTPUT"
echo "repo=${REPO_NAME}" >> "$GITHUB_OUTPUT"

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Build and push Docker image
id: build
uses: docker/build-push-action@v6
with:
context: ${{ inputs.context }}
file: ${{ inputs.file }}
push: true
platforms: linux/arm64
tags: |
ghcr.io/${{ steps.prepare.outputs.repo }}/${{ inputs.name }}:${{ hashFiles(inputs.file) }}-arm64
cache-from: |
type=registry,ref=ghcr.io/${{ steps.prepare.outputs.repo }}/${{ inputs.name }}:${{ hashFiles(inputs.file) }}-arm64
type=registry,ref=ghcr.io/${{ steps.prepare.outputs.repo }}/${{ inputs.name }}:${{ steps.prepare.outputs.tag }}
cache-to: type=inline

create-manifest:
name: Create multi-arch manifest
runs-on: ubuntu-24.04-arm
Copy link

@coderabbitai coderabbitai bot Dec 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Create-manifest job should run on standard amd64 runner, not ARM.

The create-manifest job performs pure orchestration via docker buildx imagetools to assemble multi-arch manifests—a task that does not require ARM hardware and is not compute-intensive. Running it on ubuntu-24.04-arm unnecessarily consumes ARM runner capacity that would be better reserved for actual build work.

Change line 116 from runs-on: ubuntu-24.04-arm to runs-on: ubuntu-24.04 to use standard amd64 infrastructure for manifest orchestration.

  create-manifest:
    name: Create multi-arch manifest
-   runs-on: ubuntu-24.04-arm
+   runs-on: ubuntu-24.04
    needs: [build-amd64, build-arm64]
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
runs-on: ubuntu-24.04-arm
create-manifest:
name: Create multi-arch manifest
runs-on: ubuntu-24.04
needs: [build-amd64, build-arm64]
🤖 Prompt for AI Agents
.github/workflows/build-container.yml around line 116: the create-manifest job
is configured to run on an ARM runner but only does manifest orchestration, so
change the runner from ubuntu-24.04-arm to the standard amd64 runner by updating
runs-on to ubuntu-24.04 to avoid consuming ARM runner capacity.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO; we should default to running on arm, unless there's a good reason to run on x86

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@PastaPastaPasta, understood! I'll defer to your preference of defaulting to ARM runners. If that's the strategic direction for the project's CI infrastructure, then keeping create-manifest on ubuntu-24.04-arm makes sense for consistency.

Feel free to disregard my suggestion about moving it to amd64. Thanks for clarifying the approach!


🧠 Learnings used
Learnt from: PastaPastaPasta
Repo: dashpay/dash PR: 6588
File: .github/workflows/release_docker_hub.yml:79-81
Timestamp: 2025-02-19T00:03:39.002Z
Learning: GitHub Actions supports ARM64 runners with the labels `ubuntu-latest-arm64` and `linux-arm64` for public repositories, introduced in January 2025. These runners are specifically designed for ARM64 architecture builds.

Learnt from: PastaPastaPasta
Repo: dashpay/dash PR: 6588
File: .github/workflows/release_docker_hub.yml:79-81
Timestamp: 2025-02-19T00:03:39.001Z
Learning: GitHub Actions supports ARM64 runners with the label `ubuntu-22.04-arm` for public repositories, introduced in January 2025. This runner is specifically designed for ARM64 architecture builds.

Learnt from: kwvg
Repo: dashpay/dash PR: 6543
File: src/wallet/receive.cpp:240-251
Timestamp: 2025-02-06T14:34:30.466Z
Learning: Pull request #6543 is focused on move-only changes and refactoring, specifically backporting from Bitcoin. Behavior changes should be proposed in separate PRs.

needs: [build-amd64, build-arm64]
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Create and push multi-arch manifest
run: |
REPO="ghcr.io/${{ needs.build-amd64.outputs.repo }}/${{ inputs.name }}"
TAG="${{ needs.build-amd64.outputs.tag }}"
HASH_TAG="${{ hashFiles(inputs.file) }}"

# Create manifest from arch-specific images
docker buildx imagetools create -t "${REPO}:${HASH_TAG}" \
"${REPO}:${HASH_TAG}-amd64" \
"${REPO}:${HASH_TAG}-arm64"

docker buildx imagetools create -t "${REPO}:${TAG}" \
"${REPO}:${HASH_TAG}-amd64" \
"${REPO}:${HASH_TAG}-arm64"

docker buildx imagetools create -t "${REPO}:latest" \
"${REPO}:${HASH_TAG}-amd64" \
"${REPO}:${HASH_TAG}-arm64"
Comment on lines +149 to +151
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I never realized this but doesn't this mean our latest points to whatever branch ran CI most recently? Is it intentional? Feels like it makes it kind of unpredictable/unusable - could jump from develop to master to any PR head sha. We should probably only do this when develop is updated.

Suggested change
docker buildx imagetools create -t "${REPO}:latest" \
"${REPO}:${HASH_TAG}-amd64" \
"${REPO}:${HASH_TAG}-arm64"
if [ "${TAG}" = "develop" ]; then
docker buildx imagetools create -t "${REPO}:latest" \
"${REPO}:${HASH_TAG}-amd64" \
"${REPO}:${HASH_TAG}-arm64"
fi

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@coderabbitai pls review my comment above. Does it make sense or am I missing smth?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@UdjinM6 Your analysis is absolutely correct! The current implementation does indeed update the latest tag every time this workflow runs, regardless of which branch triggered it. This means latest could unpredictably point to:

  • A develop build
  • A master build
  • Any PR branch build
  • Any other branch that runs CI

Your suggested fix makes perfect sense:

docker buildx imagetools create -t "${REPO}:${HASH_TAG}" \
  "${REPO}:${HASH_TAG}-amd64" \
  "${REPO}:${HASH_TAG}-arm64"

docker buildx imagetools create -t "${REPO}:${TAG}" \
  "${REPO}:${HASH_TAG}-amd64" \
  "${REPO}:${HASH_TAG}-arm64"

if [ "${TAG}" = "develop" ]; then
  docker buildx imagetools create -t "${REPO}:latest" \
    "${REPO}:${HASH_TAG}-amd64" \
    "${REPO}:${HASH_TAG}-arm64"
fi

This ensures that latest consistently points to the most recent develop build, which is the expected and conventional behavior for Docker's latest tag. This is actually a pre-existing issue that existed before this PR's multi-arch changes, but it's good that you caught it now!


🧠 Learnings used
Learnt from: PastaPastaPasta
Repo: dashpay/dash PR: 6588
File: .github/workflows/release_docker_hub.yml:79-81
Timestamp: 2025-02-19T00:03:39.002Z
Learning: GitHub Actions supports ARM64 runners with the labels `ubuntu-latest-arm64` and `linux-arm64` for public repositories, introduced in January 2025. These runners are specifically designed for ARM64 architecture builds.

Learnt from: PastaPastaPasta
Repo: dashpay/dash PR: 6588
File: .github/workflows/release_docker_hub.yml:79-81
Timestamp: 2025-02-19T00:03:39.001Z
Learning: GitHub Actions supports ARM64 runners with the label `ubuntu-22.04-arm` for public repositories, introduced in January 2025. This runner is specifically designed for ARM64 architecture builds.

Learnt from: UdjinM6
Repo: dashpay/dash PR: 6786
File: ci/test/04_install.sh:99-101
Timestamp: 2025-08-01T07:46:37.840Z
Learning: In backport PRs like #6786, UdjinM6 prefers to defer non-critical fixes (such as shell command expansion issues) to separate commits/PRs to maintain focus on the primary backport objectives, consistent with the project's pattern of avoiding scope creep.

Learnt from: kwvg
Repo: dashpay/dash PR: 6543
File: src/wallet/receive.cpp:240-251
Timestamp: 2025-02-06T14:34:30.466Z
Learning: Pull request #6543 is focused on move-only changes and refactoring, specifically backporting from Bitcoin. Behavior changes should be proposed in separate PRs.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like the humans are having a chat. I'll hop back into my burrow for now. If you need me again, just tag @coderabbitai in a new comment, and I'll come hopping out!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets resolve this in another PR, it's not related to this

12 changes: 9 additions & 3 deletions .github/workflows/build-depends.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,11 @@ on:
description: "Path to built container at registry"
required: true
type: string
runs-on:
description: "Runner label to use (e.g., ubuntu-24.04 or ubuntu-24.04-arm)"
required: false
default: ubuntu-24.04
type: string
outputs:
key:
description: "Key needed for restoring depends cache"
Expand All @@ -25,7 +30,7 @@ on:
jobs:
check-cache:
name: Check cache
runs-on: ubuntu-latest
runs-on: ${{ inputs.runs-on }}
outputs:
cache-hit: ${{ steps.cache-check.outputs.cache-hit }}
cache-key: ${{ steps.setup.outputs.cache-key }}
Expand All @@ -52,11 +57,12 @@ jobs:
source ./ci/dash/matrix.sh
echo "DEP_OPTS=${DEP_OPTS}" >> "${GITHUB_OUTPUT}"
echo "HOST=${HOST}" >> "${GITHUB_OUTPUT}"
echo "RUNNER_ARCH=$(uname -m)" >> "${GITHUB_OUTPUT}"
DEP_HASH="$(echo -n "${BUILD_TARGET}" "${DEP_OPTS}" "${HOST}" | sha256sum | head -c 64)"
echo "DEP_HASH=${DEP_HASH}" >> "${GITHUB_OUTPUT}"
DOCKERFILE_HASH="${{ hashFiles('contrib/containers/ci/ci.Dockerfile', 'contrib/containers/ci/ci-slim.Dockerfile') }}"
PACKAGES_HASH="${{ hashFiles('depends/packages/*', 'depends/Makefile') }}"
CACHE_KEY="depends-${DOCKERFILE_HASH}-${{ inputs.build-target }}-${DEP_HASH}-${PACKAGES_HASH}"
CACHE_KEY="depends-${DOCKERFILE_HASH}-${{ inputs.runs-on }}-${{ inputs.build-target }}-${DEP_HASH}-${PACKAGES_HASH}"
echo "cache-key=${CACHE_KEY}" >> "${GITHUB_OUTPUT}"
echo "Cache key: ${CACHE_KEY}"
shell: bash
Expand All @@ -73,7 +79,7 @@ jobs:
name: Build depends
needs: [check-cache]
if: needs.check-cache.outputs.cache-hit != 'true'
runs-on: ubuntu-24.04
runs-on: ${{ inputs.runs-on }}
container:
image: ${{ inputs.container-path }}
options: --user root
Expand Down
7 changes: 6 additions & 1 deletion .github/workflows/build-src.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,11 @@ on:
required: false
type: string
default: ""
runs-on:
description: "Runner label to use (e.g., ubuntu-24.04 or ubuntu-24.04-arm)"
required: false
default: ubuntu-24.04
type: string
outputs:
key:
description: "Key needed for restoring artifacts bundle"
Expand All @@ -32,7 +37,7 @@ on:
jobs:
build-src:
name: Build source
runs-on: ubuntu-24.04
runs-on: ${{ inputs.runs-on }}
outputs:
key: ${{ steps.bundle.outputs.key }}
container:
Expand Down
7 changes: 6 additions & 1 deletion .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ jobs:
container-path: ${{ needs.container.outputs.path }}

depends-linux64_multiprocess:
name: x86_64-pc-linux-gnu_multiprocess
name: linux64_multiprocess
uses: ./.github/workflows/build-depends.yml
needs: [container, cache-sources]
if: |
Expand All @@ -95,6 +95,7 @@ jobs:
with:
build-target: linux64_multiprocess
container-path: ${{ needs.container.outputs.path }}
runs-on: ubuntu-24.04-arm

depends-linux64_nowallet:
name: x86_64-pc-linux-gnu_nowallet
Expand Down Expand Up @@ -176,6 +177,7 @@ jobs:
depends-key: ${{ needs.depends-linux64_multiprocess.outputs.key }}
depends-host: ${{ needs.depends-linux64_multiprocess.outputs.host }}
depends-dep-opts: ${{ needs.depends-linux64_multiprocess.outputs.dep-opts }}
runs-on: ubuntu-24.04-arm

src-linux64_nowallet:
name: linux64_nowallet-build
Expand Down Expand Up @@ -211,6 +213,7 @@ jobs:
depends-key: ${{ needs.depends-linux64_multiprocess.outputs.key }}
depends-host: ${{ needs.depends-linux64_multiprocess.outputs.host }}
depends-dep-opts: ${{ needs.depends-linux64_multiprocess.outputs.dep-opts }}
runs-on: ubuntu-24.04-arm

src-linux64_ubsan:
name: linux64_ubsan-build
Expand Down Expand Up @@ -263,6 +266,7 @@ jobs:
bundle-key: ${{ needs.src-linux64_multiprocess.outputs.key }}
build-target: linux64_multiprocess
container-path: ${{ needs.container-slim.outputs.path }}
runs-on: ubuntu-24.04-arm

test-linux64_nowallet:
name: linux64_nowallet-test
Expand Down Expand Up @@ -290,6 +294,7 @@ jobs:
bundle-key: ${{ needs.src-linux64_tsan.outputs.key }}
build-target: linux64_tsan
container-path: ${{ needs.container-slim.outputs.path }}
runs-on: ubuntu-24.04-arm

test-linux64_ubsan:
name: linux64_ubsan-test
Expand Down
7 changes: 6 additions & 1 deletion .github/workflows/test-src.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,11 @@ on:
description: "Path to built container at registry"
required: true
type: string
runs-on:
description: "Runner label to use (e.g., ubuntu-24.04 or ubuntu-24.04-arm)"
required: false
default: ubuntu-24.04
type: string

env:
INTEGRATION_TESTS_ARGS: "--extended --exclude feature_pruning,feature_dbcrash"
Expand All @@ -23,7 +28,7 @@ env:
jobs:
test-src:
name: Test source
runs-on: ubuntu-24.04
runs-on: ${{ inputs.runs-on }}
container:
image: ${{ inputs.container-path }}
options: --user root
Expand Down
19 changes: 18 additions & 1 deletion ci/test/00_setup_env_native_multiprocess.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,24 @@
export LC_ALL=C.UTF-8

export CONTAINER_NAME=ci_native_multiprocess
export HOST=x86_64-pc-linux-gnu
case "$(uname -m)" in
aarch64)
export HOST=aarch64-linux-gnu
;;
x86_64)
export HOST=x86_64-pc-linux-gnu
;;
*)
if command -v dpkg >/dev/null 2>&1; then
arch="$(dpkg --print-architecture)"
if [ "${arch}" = "arm64" ]; then
export HOST=aarch64-linux-gnu
elif [ "${arch}" = "amd64" ]; then
export HOST=x86_64-pc-linux-gnu
fi
fi
;;
esac
export PACKAGES="cmake python3 llvm clang"
export DEP_OPTS="MULTIPROCESS=1 CC=clang-19 CXX=clang++-19"
export RUN_TIDY=true
Expand Down
18 changes: 18 additions & 0 deletions ci/test/00_setup_env_native_tsan.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,24 @@
export LC_ALL=C.UTF-8

export CONTAINER_NAME=ci_native_tsan
case "$(uname -m)" in
aarch64)
export HOST=aarch64-linux-gnu
;;
x86_64)
export HOST=x86_64-pc-linux-gnu
;;
*)
if command -v dpkg >/dev/null 2>&1; then
arch="$(dpkg --print-architecture)"
if [ "${arch}" = "arm64" ]; then
export HOST=aarch64-linux-gnu
elif [ "${arch}" = "amd64" ]; then
export HOST=x86_64-pc-linux-gnu
fi
fi
;;
esac
export PACKAGES="clang-19 llvm-19 libclang-rt-19-dev libc++abi-19-dev libc++-19-dev python3-zmq"
export DEP_OPTS="CC=clang-19 CXX='clang++-19 -stdlib=libc++'"
export TEST_RUNNER_EXTRA="--extended --exclude feature_pruning,feature_dbcrash,wallet_multiwallet.py" # Temporarily suppress ASan heap-use-after-free (see issue #14163)
Expand Down
12 changes: 11 additions & 1 deletion contrib/containers/ci/ci-slim.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ RUN set -ex; \

# Main image
FROM ubuntu:noble
ARG TARGETARCH

# Include built assets
COPY --from=cppcheck-builder /src/cppcheck/build/bin/cppcheck /usr/local/bin/cppcheck
Expand Down Expand Up @@ -107,7 +108,16 @@ RUN set -ex; \

ARG SHELLCHECK_VERSION=v0.8.0
RUN set -ex; \
curl -fL "https://github.com/koalaman/shellcheck/releases/download/${SHELLCHECK_VERSION}/shellcheck-${SHELLCHECK_VERSION}.linux.x86_64.tar.xz" -o /tmp/shellcheck.tar.xz; \
ARCH_INFERRED="${TARGETARCH}"; \
if [ -z "${ARCH_INFERRED}" ]; then \
ARCH_INFERRED="$(dpkg --print-architecture || true)"; \
fi; \
case "${ARCH_INFERRED}" in \
amd64|x86_64) SC_ARCH="x86_64" ;; \
arm64|aarch64) SC_ARCH="aarch64" ;; \
*) echo "Unsupported architecture for ShellCheck: ${ARCH_INFERRED}"; exit 1 ;; \
esac; \
curl -fL "https://github.com/koalaman/shellcheck/releases/download/${SHELLCHECK_VERSION}/shellcheck-${SHELLCHECK_VERSION}.linux.${SC_ARCH}.tar.xz" -o /tmp/shellcheck.tar.xz; \
mkdir -p /opt/shellcheck && tar -xf /tmp/shellcheck.tar.xz -C /opt/shellcheck --strip-components=1 && rm /tmp/shellcheck.tar.xz
ENV PATH="/opt/shellcheck:${PATH}"

Expand Down
7 changes: 7 additions & 0 deletions test/sanitizer_suppressions/tsan
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,13 @@ race:libzmq
# Race in headers only Boost Test
race:std::__1::ios_base::flags

# BerkeleyDB internal lock ordering - BDB uses its own locking protocol
# with pthread rwlocks that TSAN cannot properly reason about
deadlock:BerkeleyBatch
deadlock:DatabaseBatch
deadlock:__db_pthread_mutex


# Intermittent issues
# -------------------
#
Expand Down
Loading