ci: build on-demand docker images for testing on devnets #4667

strophy · 2022-01-18T19:08:26Z

This PR:

Implements a new Dockerfile designed to quickly build and release a multi-arch docker image from any branch based on a semver tag provided by developers, as requested by @PastaPastaPasta here. Example built images are here. Note that these are not yet gitian builds, but I could modify this to do gitian builds instead (probably with a performance impact?)
Implements CI that integrates with this new AWS infrastructure repository, allowing builds to execute quickly on powerful ephemeral self-hosted runners. Build time is currently around 25 minutes.
Modifies existing CI to use an improved caching mechanism, and renames the existing release Dockerfile to live alongside the new Dockerfile for building alpha releases

…-demand

PastaPastaPasta

Also two linters fail lint-whitespace and lint-shell

.github/workflows/release_alpha.yml

contrib/gitian-build.py

…-demand

This reverts commit daece1c.

PastaPastaPasta

🤷 LGTM, still need to change strophy/dash

.github/workflows/release_alpha.yml

PastaPastaPasta

LGTM, I think it's good. utACK for squash merge

PastaPastaPasta

re-utACK

.github/workflows/release_alpha.yml

UdjinM6

utACK

PastaPastaPasta

re-utACK for squash merge ci is passing

strophy added 24 commits January 3, 2022 15:51

ci: build alpha action

4099572

ci: use self-hosted runner

85f71ee

ci: build faster

7775e58

ci: multicore dash build

507b704

chore: copy build, dockerize and push

fa57b40

chore: dockerize build

db4b2e0

chore: remove unnecessary cachedir and checkout steps

58a556b

chore: disable qemu

fb8b7f9

chore: remove buildx steps now included in AMI

ecda587

chore: move binaries in to place for second stage copy

6a2a5cd

fix: runner using incorrect build driver

c9a7001

chore: debug builder info

8dc2b17

chore: more detailed debug

58d4275

chore: install qemu and buildx in actions

ee25f80

fix: indentation

b1d3b02

fix: remove debug code

3851f0d

feat: multi-arch cross-compiling dockerfile

f2d50d8

chore: remove unnecessary docker layers

15ebdeb

chore: add debug output

8562532

chore: dump context

1fe0c35

fix: use event inputs tag for docker_meta

acdfb0b

chore: dump context again

2818864

fix: context reference syntax

100a596

Merge branch 'master' into gha-aws-build-on-demand

ff8b98c

strophy mentioned this pull request Jan 18, 2022

ci: build on-demand docker images for testing on devnets #4661

Closed

strophy added 4 commits January 19, 2022 13:00

feat: attempt to use gitian builder

a1505e8

Merge remote-tracking branch 'upstream/develop' into gha-aws-build-on…

b724667

…-demand

chore: ff changes from other branches

daece1c

chore: disable macOS build

b2afa22

strophy marked this pull request as ready for review March 8, 2022 05:01

PastaPastaPasta requested changes Mar 8, 2022

View reviewed changes

.github/workflows/release_alpha.yml Outdated Show resolved Hide resolved

contrib/gitian-build.py Outdated Show resolved Hide resolved

strophy added 3 commits March 9, 2022 14:08

Merge remote-tracking branch 'upstream/develop' into gha-aws-build-on…

89f59f3

…-demand

Revert "chore: ff changes from other branches"

d72c971

This reverts commit daece1c.

chore: restore GHA changes to fix incorrect merge

2b0b295

PastaPastaPasta requested a review from UdjinM6 March 11, 2022 19:34

PastaPastaPasta reviewed Mar 11, 2022

View reviewed changes

UdjinM6 requested changes Mar 13, 2022

View reviewed changes

strophy added 8 commits March 14, 2022 14:08

chore: remove duplicate code

4207f62

chore: bump buildx version to latest

4c6ec8f

chore: try to build without signing

0ef27b6

chore: setup tmate to find build

f8995f7

chore: run tmate on failure

6c4b255

fix: invalid path to binaries on copy

c40aa51

chore: remove unnecessary GPG steps

365b76f

chore: replace strophy with dashpay

f3bde3f

PastaPastaPasta previously approved these changes Mar 28, 2022

View reviewed changes

strophy mentioned this pull request Mar 29, 2022

ci: enable multiarch builds dashpay/platform#316

Merged

5 tasks

fix: gha cache not providing speedup

7337cb2

strophy dismissed PastaPastaPasta’s stale review via 7337cb2 March 31, 2022 08:41

strophy requested a review from PastaPastaPasta March 31, 2022 10:09

PastaPastaPasta previously approved these changes Apr 1, 2022

View reviewed changes

UdjinM6 requested changes Apr 3, 2022

View reviewed changes

.github/workflows/release_alpha.yml Outdated Show resolved Hide resolved

UdjinM6 dismissed PastaPastaPasta’s stale review via eaf74de April 3, 2022 10:06

Update .github/workflows/release_alpha.yml

eaf74de

UdjinM6 approved these changes Apr 3, 2022

View reviewed changes

UdjinM6 added this to the 18 milestone Apr 3, 2022

UdjinM6 requested a review from PastaPastaPasta April 3, 2022 11:25

PastaPastaPasta approved these changes Apr 3, 2022

View reviewed changes

PastaPastaPasta merged commit 1fddc53 into dashpay:develop Apr 3, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci: build on-demand docker images for testing on devnets #4667

ci: build on-demand docker images for testing on devnets #4667

strophy commented Jan 18, 2022

PastaPastaPasta left a comment

PastaPastaPasta left a comment

PastaPastaPasta left a comment

PastaPastaPasta left a comment

UdjinM6 left a comment

PastaPastaPasta left a comment

ci: build on-demand docker images for testing on devnets #4667

ci: build on-demand docker images for testing on devnets #4667

Conversation

strophy commented Jan 18, 2022

PastaPastaPasta left a comment

Choose a reason for hiding this comment

PastaPastaPasta left a comment

Choose a reason for hiding this comment

PastaPastaPasta left a comment

Choose a reason for hiding this comment

PastaPastaPasta left a comment

Choose a reason for hiding this comment

UdjinM6 left a comment

Choose a reason for hiding this comment

PastaPastaPasta left a comment

Choose a reason for hiding this comment