Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revisit our docker build strategy for CI #404

Closed
ian-noaa opened this issue Aug 2, 2024 · 2 comments
Closed

Revisit our docker build strategy for CI #404

ian-noaa opened this issue Aug 2, 2024 · 2 comments
Assignees
Labels
task Tasks break a project down into discrete steps

Comments

@ian-noaa
Copy link
Contributor

ian-noaa commented Aug 2, 2024

I disabled the build cache in CI in support of #402. However, upon disabling, I discovered that the build cache wasn't doing anything as the container build time remained ~30 minutes.

The purpose of the build cache was to ensure the eccodes layer was only rebuilt when we changed the eccodes version as it's expensive to rebuild. As it wasn't doing that, we should look into building an eccodes in its own image so that we can copy the binaries out and control the update frequency or build and package eccodes for multiple architectures. (ARM & AMD64)

@ian-noaa ian-noaa added the task Tasks break a project down into discrete steps label Aug 2, 2024
@ian-noaa ian-noaa self-assigned this Aug 2, 2024
ian-noaa added a commit that referenced this issue Aug 5, 2024
We were updating the python tooling (`setuptools` & friends) in the
builder layer but forgot to update them in the prod layer. This resulted
in a `setuptools` vulnerability blocking CI.

This PR:
- updates `setuptools` & friends in the correct locations in the docker
image
- removes the docker build caching in CI as it wasn't doing anything. (I
made #404 to address this)
- Updates the security scanning step in CI to add more actionable
output.
@ian-noaa
Copy link
Contributor Author

ian-noaa commented Aug 7, 2024

@JeffHamiltonNOAA noted that we should make sure that we keep eccodes up-to-date for cfgrib. We may need to consider our update strategy and cfgrib's requirements.

@ian-noaa
Copy link
Contributor Author

The cfgrib Python wheel now bundles eccodes, so the eccodes build step was removed from the ingest Dockerfile in #423. That change obviates this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
task Tasks break a project down into discrete steps
Projects
None yet
Development

No branches or pull requests

1 participant