Detect package updates and rebuild images

[mthalman]

When updates are made available in a dependent package that is installed by one of the .NET Core Dockerfiles, this update should be detected and the image rebuilt and published to provide consumers with the latest version of the package. There is currently a system for detecting changes to base images and rebuilding dependent images so, in theory, this would be an extension of that to include the detection of package updates.

[MichaelSimons]

Recently there was an instance in which curl had a security vulnerability which is included in the .NET images. The ARM32 images didn't pick this up because the base image never changed. Vulnerability scanning caught this and a build was manually triggered to get the patched curl version that addressed the vulnerability.

Rebuilding any time that any of the packages that are included in the .NET images may cause a lot of churn. We may want to consider batching these to reduce churn if they are not addressing vulnerabilities.

[richlander]

This makes good sense, but is more complicated than the base image updates.

• IMO, we have no good choice but to accept base image updates and rebuild / republish.
• We have a lot more flexibility with packages. We'll get the new package the next time we rebuild .NET Core images, either due to a base image update or .NET Core servicing update. Typically, (together) those happen multiple times per month. So, it only makes to force a package update for CVEs.
• The big question is finding a way to know which of the package updates are functional vs a critical CVE. Anyone know of a magic API that offers that info?

[NasAmin]

Can something like Trivy be used?
https://github.com/aquasecurity/trivy

[mthalman]

Another example where this would have been useful: #2417. The krb5 package had been updated with a security fix but the parent Alpine image never changed. This meant we never automatically triggered a build to rebuild the image.

[mthalman]

As stated in dotnet/docker-tools#824 (comment), the plan is stored the package graph as metadata in the image info file. In addition to actually storing this metadata, we should also provide notifications whenever the package graph changes for a given Dockerfile (see #3271 (comment)). This allows us to be aware of any unexpected changes.

[mthalman]

Now that the infrastructure supports the generation of SBOMs, it seems more appropriate to consume the SBOM's description of the packages that are installed in an image rather than duplicating that information in the image info file. This would clean up code and provide a single source of truth.

[mthalman]

With CBL-Mariner 2.0, there are no longer the necessary commands to execute the scripts to get the set of installed packages in a Mariner 2.0 container because the sed and column commands aren't available. Installing them isn't a straightforward option because that would then impact the results of which packages are installed.

As a result, package gathering for Mariner 2.0 will not be done via these scripts. It's not worth the investment considering it makes more sense to use SBOMs as the source of packages instead.

By using SBOMs, we'll have a centralized, cross-platform method of retrieving package information that isn't reliant on execution within the container itself.

[mthalman]

This will need to account for the newly added Chiseled Ubuntu Dockerfiles. Those are unique in that they are based on scratch and use the chisel tool to install the packages. So whatever logic is used to determine the source of the packages to compare against needs to account for this scenario.