Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Review repo structure, audit images #847

Closed
atc0005 opened this issue Feb 10, 2023 · 0 comments · Fixed by #874
Closed

Review repo structure, audit images #847

atc0005 opened this issue Feb 10, 2023 · 0 comments · Fixed by #874
Assignees
@atc0005
Labels
documentation Improvements or additions to documentation enhancement New feature or request Makefile oldstable stable stable-build-only stable-lint-only unstable
Milestone
v0.8.0

Comments

@atc0005
Copy link
Owner

atc0005 commented Feb 10, 2023 •
edited
Loading

Overview / Background

I found myself once again looking up how I was using the combined stable image in CI workflows. The README reminds me:

  • built from the latest version of the current stable golang image.
  • used for building Go applications, both directly and via Makefile builds.
  • intended for use in a build/test matrix of prior, current and upcoming Go
    releases
  • provides multiple linters

The LABEL values attached via the Dockerfile indicate that the image title is go-ci-stable. I then look at the shared CI workflows and see that the go-ci-stable, go-ci-oldstable and go-ci-unstable images are all used for:

  • matrix-based linting
  • matrix-based testing
  • matrix-based building
    • simple go build -v -mod=vendor ./... to validate that code will build on recent Go releases

The intent is to provide a common image that can be used for all of these purposes.

In another workflow I've been using an image named go-ci-lint-only for "quick" lint & test code CI tasks. This was intended to run and finish by the time I finished filling out a PR as a fail-fast indicator. Because the associated workflows were marked as required by each dependent project's branch protection, this halted the PR process and I believe further CI jobs from running.

Current images / sizes

From v0.7.10 build summary output:

REPOSITORY                           TAG                                                IMAGE ID       CREATED                  SIZE
ghcr.io/atc0005/go-ci                go-ci-unstable                                     caa28746357b   Less than a second ago   858MB
ghcr.io/atc0005/go-ci                go-ci-unstable-v0.7.10-0-gcc0a2418                 caa28746357b   Less than a second ago   858MB
registry-1.docker.io/atc0005/go-ci   go-ci-unstable                                     caa28746357b   Less than a second ago   858MB
registry-1.docker.io/atc0005/go-ci   go-ci-unstable-v0.7.10-0-gcc0a2418                 caa28746357b   Less than a second ago   858MB
ghcr.io/atc0005/go-ci                go-ci-oldstable                                    65d742cee580   About a minute ago       1.07GB
ghcr.io/atc0005/go-ci                go-ci-oldstable-v0.7.10-0-gcc0a2418                65d742cee580   About a minute ago       1.07GB
registry-1.docker.io/atc0005/go-ci   go-ci-oldstable                                    65d742cee580   About a minute ago       1.07GB
registry-1.docker.io/atc0005/go-ci   go-ci-oldstable-v0.7.10-0-gcc0a2418                65d742cee580   About a minute ago       1.07GB
ghcr.io/atc0005/go-ci                go-ci-lint-only                                    e01cebf78645   2 minutes ago            829MB
ghcr.io/atc0005/go-ci                go-ci-lint-only-v0.7.10-0-gcc0a2418                e01cebf78645   2 minutes ago            829MB
registry-1.docker.io/atc0005/go-ci   go-ci-lint-only                                    e01cebf78645   2 minutes ago            829MB
registry-1.docker.io/atc0005/go-ci   go-ci-lint-only-v0.7.10-0-gcc0a2418                e01cebf78645   2 minutes ago            829MB
ghcr.io/atc0005/go-ci                go-ci-stable-debian-build                          66bbc76c104a   3 minutes ago            1.84GB
ghcr.io/atc0005/go-ci                go-ci-stable-debian-build-v0.7.10-0-gcc0a2418      66bbc76c104a   3 minutes ago            1.84GB
registry-1.docker.io/atc0005/go-ci   go-ci-stable-debian-build                          66bbc76c104a   3 minutes ago            1.84GB
registry-1.docker.io/atc0005/go-ci   go-ci-stable-debian-build-v0.7.10-0-gcc0a2418      66bbc76c104a   3 minutes ago            1.84GB
ghcr.io/atc0005/go-ci                go-ci-stable-alpine-buildx86                       8fea166cc2da   5 minutes ago            733MB
ghcr.io/atc0005/go-ci                go-ci-stable-alpine-buildx86-v0.7.10-0-gcc0a2418   8fea166cc2da   5 minutes ago            733MB
registry-1.docker.io/atc0005/go-ci   go-ci-stable-alpine-buildx86                       8fea166cc2da   5 minutes ago            733MB
registry-1.docker.io/atc0005/go-ci   go-ci-stable-alpine-buildx86-v0.7.10-0-gcc0a2418   8fea166cc2da   5 minutes ago            733MB
ghcr.io/atc0005/go-ci                go-ci-stable-alpine-buildx64                       5ac8a7302002   5 minutes ago            746MB
ghcr.io/atc0005/go-ci                go-ci-stable-alpine-buildx64-v0.7.10-0-gcc0a2418   5ac8a7302002   5 minutes ago            746MB
registry-1.docker.io/atc0005/go-ci   go-ci-stable-alpine-buildx64                       5ac8a7302002   5 minutes ago            746MB
registry-1.docker.io/atc0005/go-ci   go-ci-stable-alpine-buildx64-v0.7.10-0-gcc0a2418   5ac8a7302002   5 minutes ago            746MB
ghcr.io/atc0005/go-ci                go-ci-stable                                       c2b722486f62   6 minutes ago            858MB
ghcr.io/atc0005/go-ci                go-ci-stable-v0.7.10-0-gcc0a2418                   c2b722486f62   6 minutes ago            858MB
ghcr.io/atc0005/go-ci                latest                                             c2b722486f62   6 minutes ago            858MB
registry-1.docker.io/atc0005/go-ci   go-ci-stable                                       c2b722486f62   6 minutes ago            858MB
registry-1.docker.io/atc0005/go-ci   go-ci-stable-v0.7.10-0-gcc0a2418                   c2b722486f62   6 minutes ago            858MB
registry-1.docker.io/atc0005/go-ci   latest                                             c2b722486f62   6 minutes ago            858MB
ghcr.io/atc0005/go-ci                go-ci-stable-mirror-build                          88284c55b668   5 days ago               777MB
ghcr.io/atc0005/go-ci                go-ci-stable-mirror-build-v0.7.10-0-gcc0a2418      88284c55b668   5 days ago               777MB
registry-1.docker.io/atc0005/go-ci   go-ci-stable-mirror-build                          88284c55b668   5 days ago               777MB
registry-1.docker.io/atc0005/go-ci   go-ci-stable-mirror-build-v0.7.10-0-gcc0a2418      88284c55b668   5 days ago               777MB

Drop images?

As I finish building images for the v0.7.10 release I see that the difference between the go-ci-stable and go-ci-lint-only images is about 30 MB. Honestly, this doesn't seem worth having two separate images.

Ultimately the difference comes down to the go-ci-stable having additional linters:

  • errwrap
  • tomll
  • httperroryzer

Both contain (base):

  • golangci-lint
  • staticcheck
  • govulncheck

We also have the go-ci-stable-debian-build image (see #657) that bundles all linters (additional & base) along with Mingw-w64 packages. This image is intended to support CGO-enabled builds, but I'm not currently using it in any of my projects and Dockerhub shows 0 downloads, so it's also under review to be dropped.

An alternative is to rework the workflows in my projects (currently just one) to use this image instead of installing the gcc-multilib and gcc-mingw-w64 packages as a CI workflow task. This will give the image a purpose, but possibly making up a need where there really isn't one?

Rename images

On the topic of not well defined names, the go-ci-stable-debian-build image provides a Mingw-w64 cross-compile environment to build CGO-enabled Windows binaries using the stable/current version Go (Go 1.20 as I write this).

This should be renamed to reflect its purpose.

New images

Up until this point all images were intended to be used in CI for validation (linting, testing, build) purposes. To make tracking build dependencies easier (e.g., nfpm for packages, tc-hib/go-winres for Windows binary metadata), I'm considering creating a new image intended solely for generating release assets.

I'm doing this to some degree now with the alpine-buildx64 and alpine-buildx86 images for CGO-enabled builds.

As noted previously, separate "linting only" and "combined" images were maintained with the intention that one would be super slim and for pre-PR CI tasks while the other would be more comprehensive and used for all validation tasks. The size difference between the two is negligible.

Current layout

$ tree . -I scratch
.
├── CHANGELOG.md
├── LICENSE
├── Makefile
├── README.md
├── oldstable
│   └── Dockerfile
├── stable
│   ├── build
│   │   ├── alpine-x64
│   │   │   └── Dockerfile
│   │   ├── alpine-x86
│   │   │   └── Dockerfile
│   │   ├── debian
│   │   │   └── Dockerfile
│   │   └── mirror
│   │       └── Dockerfile
│   ├── combined
│   │   └── Dockerfile
│   └── linting
│       └── Dockerfile
├── tools
│   ├── go.mod
│   └── tools.go
└── unstable
    └── Dockerfile

11 directories, 14 files

Potential layout

I was working up a layout that I planned to use as a replacement, but the more I worked with the mock-up the more I questioned the purpose of some existing images.

Going to need to give this further thought.

In the meantime I'll keep updating existing images and keep things as-is until next steps are a bit clearer.
@atc0005 atc0005 added this to the Future milestone Feb 10, 2023
@atc0005 atc0005 self-assigned this Feb 10, 2023
@atc0005 atc0005 pinned this issue Feb 10, 2023
@atc0005 atc0005 mentioned this issue Feb 10, 2023
Closed
atc0005 added a commit to atc0005/check-cert that referenced this issue Feb 15, 2023
@atc0005
Drop Push Validation workflow
086897f 
Remove `push-validation.yml` workflow file. This in turn removes the
dependency on the imported (upstream) `quick-validation.yml` workflow file.

Linting functionality previously relied upon is already supplied by separate
CI matrix managed workflow jobs.

refs atc0005/shared-project-resources#63
refs atc0005/go-ci#847
@atc0005 atc0005 mentioned this issue Feb 15, 2023
Merged
atc0005 added a commit to atc0005/bridge that referenced this issue Feb 16, 2023
@atc0005
Drop Push Validation workflow
1f7ad48 
Remove `push-validation.yml` workflow file. This in turn removes the
dependency on the imported (upstream) `quick-validation.yml` workflow file.

Linting functionality previously relied upon is already supplied by separate
CI matrix managed workflow jobs.

refs atc0005/shared-project-resources#63
refs atc0005/go-ci#847
@atc0005 atc0005 mentioned this issue Feb 16, 2023
Merged
atc0005 added a commit to atc0005/brick that referenced this issue Feb 16, 2023
@atc0005
Drop Push Validation workflow
6773c07 
Remove `push-validation.yml` workflow file. This in turn removes the
dependency on the imported (upstream) `quick-validation.yml` workflow file.

Linting functionality previously relied upon is already supplied by separate
CI matrix managed workflow jobs.

refs atc0005/shared-project-resources#63
refs atc0005/go-ci#847
@atc0005 atc0005 mentioned this issue Feb 16, 2023
Merged
atc0005 added a commit to atc0005/bounce that referenced this issue Feb 16, 2023
@atc0005
Drop Push Validation workflow
baa6673 
Remove `push-validation.yml` workflow file. This in turn removes the
dependency on the imported (upstream) `quick-validation.yml` workflow file.

Linting functionality previously relied upon is already supplied by separate
CI matrix managed workflow jobs.

refs atc0005/shared-project-resources#63
refs atc0005/go-ci#847
@atc0005 atc0005 mentioned this issue Feb 16, 2023
Merged
atc0005 added a commit to atc0005/check-illiad that referenced this issue Feb 16, 2023
@atc0005
Drop Push Validation workflow
3387e3c 
Remove `push-validation.yml` workflow file. This in turn removes the
dependency on the imported (upstream) `quick-validation.yml` workflow file.

Linting functionality previously relied upon is already supplied by separate
CI matrix managed workflow jobs.

refs atc0005/shared-project-resources#63
refs atc0005/go-ci#847
@atc0005 atc0005 mentioned this issue Feb 16, 2023
Merged
atc0005 added a commit to atc0005/check-mail that referenced this issue Feb 16, 2023
@atc0005
Drop Push Validation workflow
2907511 
Remove `push-validation.yml` workflow file. This in turn removes the
dependency on the imported (upstream) `quick-validation.yml` workflow file.

Linting functionality previously relied upon is already supplied by separate
CI matrix managed workflow jobs.

refs atc0005/shared-project-resources#63
refs atc0005/go-ci#847
@atc0005 atc0005 mentioned this issue Feb 16, 2023
Merged
atc0005 added a commit to atc0005/check-ntpt that referenced this issue Feb 16, 2023
@atc0005
Drop Push Validation workflow
66a51af 
Remove `push-validation.yml` workflow file. This in turn removes the
dependency on the imported (upstream) `quick-validation.yml` workflow file.

Linting functionality previously relied upon is already supplied by separate
CI matrix managed workflow jobs.

refs atc0005/shared-project-resources#63
refs atc0005/go-ci#847
@atc0005 atc0005 mentioned this issue Feb 16, 2023
Merged
atc0005 added a commit to atc0005/check-path that referenced this issue Feb 16, 2023
@atc0005
Drop Push Validation workflow
c1f68e5 
Remove `push-validation.yml` workflow file. This in turn removes the
dependency on the imported (upstream) `quick-validation.yml` workflow file.

Linting functionality previously relied upon is already supplied by separate
CI matrix managed workflow jobs.

refs atc0005/shared-project-resources#63
refs atc0005/go-ci#847
atc0005 added a commit to atc0005/check-process that referenced this issue Feb 16, 2023
@atc0005
Drop Push Validation workflow
786d26f 
Remove `push-validation.yml` workflow file. This in turn removes the
dependency on the imported (upstream) `quick-validation.yml` workflow file.

Linting functionality previously relied upon is already supplied by separate
CI matrix managed workflow jobs.

refs atc0005/shared-project-resources#63
refs atc0005/go-ci#847
atc0005 added a commit to atc0005/check-restart that referenced this issue Feb 16, 2023
@atc0005
Drop Push Validation workflow
f3d2e53 
Remove `push-validation.yml` workflow file. This in turn removes the
dependency on the imported (upstream) `quick-validation.yml` workflow file.

Linting functionality previously relied upon is already supplied by separate
CI matrix managed workflow jobs.

refs atc0005/shared-project-resources#63
refs atc0005/go-ci#847
atc0005 added a commit to atc0005/check-statuspage that referenced this issue Feb 16, 2023
@atc0005
Drop Push Validation workflow
383d0f5 
Remove `push-validation.yml` workflow file. This in turn removes the
dependency on the imported (upstream) `quick-validation.yml` workflow file.

Linting functionality previously relied upon is already supplied by separate
CI matrix managed workflow jobs.

refs atc0005/shared-project-resources#63
refs atc0005/go-ci#847
This was referenced Feb 24, 2023
Merged
Merged
Merged
atc0005 added a commit to atc0005/dnsc that referenced this issue Feb 24, 2023
@atc0005
Remove Push Validation workflow status badge
354c987 
This workflow was removed recently, but the "status badge" was
unintentionally left behind in the README file.

refs atc0005/shared-project-resources#63
refs atc0005/go-ci#847
atc0005 added a commit to atc0005/elbow that referenced this issue Feb 24, 2023
@atc0005
Remove Push Validation workflow status badge
fe0079d 
This workflow was removed recently, but the "status badge" was
unintentionally left behind in the README file.

refs atc0005/shared-project-resources#63
refs atc0005/go-ci#847
atc0005 added a commit to atc0005/go-lockss that referenced this issue Feb 24, 2023
@atc0005
Remove Push Validation workflow status badge
92a23d6 
This workflow was removed recently, but the "status badge" was
unintentionally left behind in the README file.

refs atc0005/shared-project-resources#63
refs atc0005/go-ci#847
This was referenced Feb 24, 2023
Merged
Merged
Merged
atc0005 added a commit to atc0005/go-ezproxy that referenced this issue Feb 24, 2023
@atc0005
Remove Push Validation workflow status badge
476670b 
This workflow was removed recently, but the "status badge" was
unintentionally left behind in the README file.

refs atc0005/shared-project-resources#63
refs atc0005/go-ci#847
@atc0005 atc0005 mentioned this issue Feb 24, 2023
Merged
atc0005 added a commit to atc0005/go-nagios that referenced this issue Feb 24, 2023
@atc0005
Remove Push Validation workflow status badge
730b4bf 
This workflow was removed recently, but the "status badge" was
unintentionally left behind in the README file.

refs atc0005/shared-project-resources#63
refs atc0005/go-ci#847
atc0005 added a commit to atc0005/go-teams-notify that referenced this issue Feb 24, 2023
@atc0005
Remove Push Validation workflow status badge
387a5aa 
This workflow was removed recently, but the "status badge" was
unintentionally left behind in the README file.

refs atc0005/shared-project-resources#63
refs atc0005/go-ci#847
@atc0005 atc0005 mentioned this issue Feb 24, 2023
Merged
atc0005 added a commit to atc0005/nagios-debug that referenced this issue Feb 24, 2023
@atc0005
Remove Push Validation workflow status badge
dcf44a4 
This workflow was removed recently, but the "status badge" was
unintentionally left behind in the README file.

refs atc0005/shared-project-resources#63
refs atc0005/go-ci#847
This was referenced Feb 24, 2023
Merged
Merged
atc0005 added a commit to atc0005/send2teams that referenced this issue Feb 24, 2023
@atc0005
Remove Push Validation workflow status badge
adc895a 
This workflow was removed recently, but the "status badge" was
unintentionally left behind in the README file.

refs atc0005/shared-project-resources#63
refs atc0005/go-ci#847
atc0005 added a commit to atc0005/tsm-pass that referenced this issue Feb 24, 2023
@atc0005
Remove Push Validation workflow status badge
4fe9bea 
This workflow was removed recently, but the "status badge" was
unintentionally left behind in the README file.

refs atc0005/shared-project-resources#63
refs atc0005/go-ci#847
atc0005 added a commit to atc0005/safelinks that referenced this issue Feb 24, 2023
@atc0005
Remove Push Validation workflow status badge
c655cd0 
This workflow was removed recently, but the "status badge" was
unintentionally left behind in the README file.

refs atc0005/shared-project-resources#63
refs atc0005/go-ci#847
atc0005 added a commit to atc0005/query-meta that referenced this issue Feb 24, 2023
@atc0005
Remove Push Validation workflow status badge
838a67d 
This workflow was removed recently, but the "status badge" was
unintentionally left behind in the README file.

refs atc0005/shared-project-resources#63
refs atc0005/go-ci#847
atc0005 added a commit to atc0005/mysql2sqlite that referenced this issue Feb 24, 2023
@atc0005
Remove Push Validation workflow status badge
c4031c4 
This workflow was removed recently, but the "status badge" was
unintentionally left behind in the README file.

refs atc0005/shared-project-resources#63
refs atc0005/go-ci#847
This was referenced Feb 24, 2023
Merged
Merged
Merged
Merged
Merged
atc0005 added a commit that referenced this issue Mar 8, 2023
@atc0005
Fix multiple Dependabot monitored Dockerfile paths
20d0347 
- /oldstable/build/release
- /oldstable/combined
- /stable/build/cgo-mingw-w64
- /unstable/build/release
- /unstable/combined

These were missed after all of the work in v0.9.0 to rearrange
paths & refresh images.

refs GH-932
refs GH-847
@atc0005 atc0005 mentioned this issue Mar 8, 2023
Merged
@atc0005 atc0005 mentioned this issue Mar 23, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@atc0005 atc0005

Labels
documentation Improvements or additions to documentation enhancement New feature or request Makefile oldstable stable stable-build-only stable-lint-only unstable
Projects
None yet
Milestone
v0.8.0
Development

Successfully merging a pull request may close this issue.

Rework images atc0005/go-ci
1 participant
@atc0005