Build own docker images for integration tests #901

dklimpel · 2024-05-07T08:28:39Z

Checklist

make test-all (UNIX) passes. CI will also test this
unit and/or integration tests are included (if applicable)
documentation is changed or added (if applicable)

Description of change

Add own github workflow to build the docker images from the test and store them in a repo (ghcr.io). This can resolve the dependency on external repos in the future and ensure more up-to-date images or tests.

There are no significant changes at the Dockerfiles. I have therefore updated the md5 files so that the tests continue to run unchanged.

wheezy has expired repo signatures and needs a --force-yes
archlinux:base has moved
problems with pacman, but systemd-sysvcompat seems up to date

#6 [2/4] RUN pacman -Sy --noconfirm systemd-sysvcompat
#6 0.061 :: Synchronizing package databases...
#6 2.666  core downloading...
#6 2.666  extra downloading...
#6 2.671 resolving dependencies...
#6 2.838 :: There are 2 providers available for dbus-units:
#6 2.838 :: Repository core
#6 2.838    1) dbus-broker-units  2) dbus-daemon-units
#6 2.838 
#6 2.838 Enter a number (default=1): 
#6 2.838 looking for conflicting packages...
#6 2.839 
#6 2.839 Package (5)              Old Version  New Version  Net Change  Download Size
#6 2.839 
#6 2.839 core/dbus-broker                      36-2           0.35 MiB       0.15 MiB
#6 2.839 core/dbus-broker-units                36-2           0.00 MiB       0.00 MiB
#6 2.839 core/systemd             255.2-1      255.5-4       -0.47 MiB       8.00 MiB
#6 2.839 core/systemd-libs        255.2-1      255.5-4       -0.06 MiB       1.07 MiB
#6 2.839 core/systemd-sysvcompat  255.2-1      255.5-4        0.00 MiB       0.01 MiB
#6 2.839 
#6 2.839 Total Download Size:    9.22 MiB
#6 2.839 Total Installed Size:  33.84 MiB
#6 2.839 Net Upgrade Size:      -0.18 MiB
#6 2.839 
#6 2.839 :: Proceed with installation? [Y/n] 
#6 2.839 :: Retrieving packages...
#6 3.718  systemd-255.5-4-x86_64 downloading...
#6 3.718  systemd-libs-255.5-4-x86_64 downloading...
#6 3.718  dbus-broker-36-2-x86_64 downloading...
#6 3.718  systemd-sysvcompat-255.5-4-x86_64 downloading...
#6 3.718  dbus-broker-units-36-2-x86_64 downloading...
#6 3.718 checking keyring...
#6 4.912 checking package integrity...
#6 5.067 loading package files...
#6 5.075 checking for file conflicts...
#6 5.076 error: failed to commit transaction (conflicting files)
#6 5.122 dbus-broker-units: /usr/lib/systemd/system/dbus.service exists in filesystem (owned by dbus)
#6 5.122 dbus-broker-units: /usr/lib/systemd/user/dbus.service exists in filesystem (owned by dbus)
#6 5.122 Errors occurred, no packages were upgraded.
#6 ERROR: process "/bin/sh -c pacman -Sy --noconfirm systemd-sysvcompat" did not complete successfully: exit code: 1
------
 > [2/4] RUN pacman -Sy --noconfirm systemd-sysvcompat:
3.718  systemd-sysvcompat-255.5-4-x86_64 downloading...
3.718  dbus-broker-units-36-2-x86_64 downloading...
3.718 checking keyring...
4.912 checking package integrity...
5.067 loading package files...
5.075 checking for file conflicts...
5.076 error: failed to commit transaction (conflicting files)
5.122 dbus-broker-units: /usr/lib/systemd/system/dbus.service exists in filesystem (owned by dbus)
5.122 dbus-broker-units: /usr/lib/systemd/user/dbus.service exists in filesystem (owned by dbus)
5.122 Errors occurred, no packages were upgraded.
------
Dockerfile_arch:4
--------------------
   2 |     MAINTAINER @siddharthist
   3 |     
   4 | >>> RUN pacman -Sy --noconfirm systemd-sysvcompat
   5 |     RUN ln -s /does_not_exist /foo && \
   6 |         chmod 700 ~root
--------------------
ERROR: failed to solve: process "/bin/sh -c pacman -Sy --noconfirm systemd-sysvcompat" did not complete successfully: exit code: 1

Related to:

petemounce

lgtm.

How would a person go about running this locally if they need to debug / reproduce what CI is doing?

.github/workflows/docker-integration-tests.yaml

integration-tests/Dockerfile_arch

dklimpel · 2024-06-30T12:45:23Z

How would a person go about running this locally if they need to debug / reproduce what CI is doing?

You have this options for debugging.

Fork this repository and manually run this job in your GH action section.
Run docker build
Use script from repository https://github.com/goss-org/goss/blob/master/development/build_images.sh

petemounce

Beautiful. If GitHub agents have tree >=1.7, -J will return JSON.

Edit: But it doesn't look like they do.

dklimpel · 2024-07-13T17:47:17Z

I have converted back to draft because I am not sure if the condtion to build after every push to master is the best way.

on:
  push:
    branches:
      - master

aelsabbahy · 2024-08-15T19:42:15Z

Couple of small questions here:

How do I run this once it's merged. I assume it's ad-hoc and not triggered from a branch
Does this skip the build (or more importantly the push) if the git sha matches? Will it also update the .md5 file?

dklimpel · 2024-08-15T20:13:28Z

How do I run this once it's merged. I assume it's ad-hoc and not triggered from a branch

On the workflow page: https://github.com/goss-org/goss/actions/workflows/docker-integration-tests.yaml
is a button to trigger the job.

Does this skip the build (or more importantly the push) if the git sha matches?

Currently not

Will it also update the .md5 file?

Currently not

aelsabbahy · 2024-08-15T20:16:01Z

Sounds good, I'll merge this. The skip can be done as a follow up enhancement.

aelsabbahy

Thank you for the clear explanation and the continued awesome work!

dklimpel added 2 commits May 7, 2024 09:57

Build own docker images and upload to ghcr

a0406f4

update md5 sum for dockerfiles

65ce170

dklimpel marked this pull request as ready for review May 7, 2024 10:41

dklimpel requested a review from aelsabbahy as a code owner May 7, 2024 10:41

dklimpel marked this pull request as draft May 9, 2024 18:50

minor text updates

373e5af

dklimpel changed the title ~~Build own docker images and upload to ghcr~~ Build own docker images for integration tests May 10, 2024

dklimpel marked this pull request as ready for review May 10, 2024 22:15

This was referenced May 10, 2024

Add pipeline for build goss docker image #909

Merged

Improve CI pipeline and integration tests #911

Open

dklimpel mentioned this pull request May 28, 2024

bump alpine linux to 3.19 #907

Merged

3 tasks

dklimpel mentioned this pull request Jun 21, 2024

Pushed new integration test images and update image md5 #938

Merged

3 tasks

petemounce previously approved these changes Jun 28, 2024

View reviewed changes

.github/workflows/docker-integration-tests.yaml Outdated Show resolved Hide resolved

integration-tests/Dockerfile_arch Outdated Show resolved Hide resolved

Remove outdated lines from Dockerfile_arch

d38fce0

dklimpel dismissed petemounce’s stale review via d38fce0 June 30, 2024 12:24

Merge branch 'master' into build_docker

e31c124

petemounce previously approved these changes Jul 1, 2024

View reviewed changes

generate the list of dockerfiles dynamically

2abe0f5

dklimpel dismissed petemounce’s stale review via 2abe0f5 July 6, 2024 16:19

petemounce previously approved these changes Jul 7, 2024

View reviewed changes

dklimpel mentioned this pull request Jul 13, 2024

Add labels to container images when build with build_images.sh #948

Merged

3 tasks

dklimpel marked this pull request as draft July 13, 2024 17:45

dklimpel added 2 commits July 13, 2024 19:48

Merge branch 'master' into build_docker

d595098

update wheezy checksum

c5a0cc4

dklimpel dismissed petemounce’s stale review via c5a0cc4 July 13, 2024 17:49

dklimpel added 2 commits July 14, 2024 00:14

fix line endings

0cb9ddd

Merge remote-tracking branch 'origin/master' into build_docker

47ca341

dklimpel mentioned this pull request Aug 4, 2024

fix: convert line endings of Dockerfile to linux #952

Merged

3 tasks

disable automatic builds

6fc0ba1

dklimpel marked this pull request as ready for review August 14, 2024 14:38

dklimpel added 2 commits August 14, 2024 16:39

Merge branch 'master' into build_docker

1c701d3

bump httpd version for rockylinux

5ccb39c

aelsabbahy approved these changes Aug 15, 2024

View reviewed changes

aelsabbahy merged commit 191fe32 into goss-org:master Aug 15, 2024
7 checks passed

dklimpel deleted the build_docker branch August 15, 2024 20:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build own docker images for integration tests #901

Build own docker images for integration tests #901

dklimpel commented May 7, 2024 •

edited

Loading

petemounce left a comment

dklimpel commented Jun 30, 2024

petemounce left a comment •

edited

Loading

dklimpel commented Jul 13, 2024

aelsabbahy commented Aug 15, 2024

dklimpel commented Aug 15, 2024 •

edited

Loading

aelsabbahy commented Aug 15, 2024

aelsabbahy left a comment

Build own docker images for integration tests #901

Build own docker images for integration tests #901

Conversation

dklimpel commented May 7, 2024 • edited Loading

Checklist

Description of change

petemounce left a comment

Choose a reason for hiding this comment

dklimpel commented Jun 30, 2024

petemounce left a comment • edited Loading

Choose a reason for hiding this comment

dklimpel commented Jul 13, 2024

aelsabbahy commented Aug 15, 2024

dklimpel commented Aug 15, 2024 • edited Loading

aelsabbahy commented Aug 15, 2024

aelsabbahy left a comment

Choose a reason for hiding this comment

dklimpel commented May 7, 2024 •

edited

Loading

petemounce left a comment •

edited

Loading

dklimpel commented Aug 15, 2024 •

edited

Loading