v4.8.0 Release

[Unreleased]

[v4.8.0] - 2024-10-09

NOTE

This release deprecates the updaters that rely on the Red Hat OVAL v2 security data in favor of the Red Hat VEX data. This change includes a database migration to delete all the vulnerabilities that originated from the OVAL v2 feeds, meaning there could be a time in production environments before the VEX updater completes for the first time when no Red Hat vulnerabilities exist. This release also contains a clairctl admin command to clean up the deprecated vulnerabilities outside of the migration workflow which allows an operator to pre-run the migration:

clairctl -D admin pre v4.8.0

Claircore

• rhel: move IgnoreUnpatched config key from updater to matcher

  Previously the IgnoreUnpatched config key was a part of the RHEL updater and would dictate whether or not the updater would ingest unpatched vulnerabilities. This change moves that key to the RHEL matcher and dictates whether the matcher should check for a fixed_in_version when querying potential vulnerabilities. This makes the config option more usable at the expense of DB size.

• rhel: add csaf/vex updater

  Replace the RHEL OVAL updater with a CSAF/VEX updater for Red Hat security data. Update the matching logic to deal with CPE patterns coming from the VEX files. Remove RHEL updater and add a migration to delete Red Hat OVAL data from the database.

• datastore: add vuln and enrich stream updates

  In an effort to reduce memory consumption during updating the vulnerability database, add support for iterators. Extend Updater interface with `UpdateVulnerabilitiesIter` method that performs the same operation as `UpdateVulnerabilities` but accepts an iterator function instead of a slice. Also, extend the `EnrichmentUpdater` interface with `UpdateEnrichmentsIter` in the same way.

• cpe: add match expression support

  This adds support for NIST IR 7696, aka CPE2.3 Name Matching. It's anticipated to be used in upcoming CSAF/VEX support. See https://doi.org/10.6028/NIST.IR.7696 for the specification.

'Chore

• ab3a754e: update claircore to v1.5.19
• f783b356: update claircore to v1.5.18
• 9286ab86: update claircore to v1.5.17

Admin

• d3467bad: add pre v4.8.0 admin command to delete OVAL vulns
• d53780b6: add a check for compatible migration version
• 87c24a9c: add command to update go packages with norm_version
• 02e6c925: add pre v4.7.3 admin command to create index

All

• 55294aaf: fix incorrect API paths
• daa78ec2: fix some typos

Amqp

• 8fcd294c: migrate to maintained package
• #1793### Auto
• 07b0ea7b: improve log messages
• #2092### Build(Deps)
• 5092198b: bump golang.org/x/time from 0.6.0 to 0.7.0
• e7b6deac: bump golang.org/x/net from 0.29.0 to 0.30.0
• 55fb7735: bump github.com/klauspost/compress from 1.17.9 to 1.17.10
• 7a2e7186: bump github.com/prometheus/client_golang
• 698d9170: bump github.com/rogpeppe/go-internal from 1.12.0 to 1.13.1
• 7ec7e04f: bump go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace
• 96ee336f: bump go.opentelemetry.io/otel/exporters/stdout/stdouttrace
• 5fb41ed8: bump golang.org/x/net from 0.28.0 to 0.29.0
• 2a13e7b7: bump peter-evans/create-pull-request from 6 to 7
• 061b1e09: bump github.com/prometheus/client_golang
• a2c920f4: bump go.opentelemetry.io/otel/exporters/stdout/stdouttrace
• bbaece4e: bump go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace
• 24aff4e4: bump go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp
• b203913a: bump github.com/prometheus/client_golang
• 96937294: bump github.com/grafana/pyroscope-go/godeltaprof
• 01b57db6: bump github.com/google/go-containerregistry
• 7ceeaaa2: bump github.com/go-stomp/stomp/v3 from 3.1.1 to 3.1.2
• c3ce1982: bump github.com/urfave/cli/v2 from 2.27.2 to 2.27.3
• 95f5a5f2: bump github.com/google/go-containerregistry
• 1a5f342c: bump github.com/go-stomp/stomp/v3 from 3.1.0 to 3.1.1
• 5821a5bf: bump golang.org/x/net from 0.26.0 to 0.27.0
• 08587861: bump github.com/google/go-containerregistry
• 74914938: bump go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace
• 67bdbbbe: bump go.opentelemetry.io/otel/exporters/stdout/stdouttrace
• dd9d6760: bump go.opentelemetry.io/otel from 1.27.0 to 1.28.0
• fcee4364: bump github.com/klauspost/compress from 1.17.8 to 1.17.9
• 3f229e99: bump github.com/google/go-containerregistry
• c5ae5021: bump docker/build-push-action from 5 to 6
• 7400db24: bump golang.org/x/net from 0.25.0 to 0.26.0
• 74b377b8: bump github.com/rs/zerolog from 1.32.0 to 1.33.0
• 1fff0726: bump go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace
• f2533fbf: bump go.opentelemetry.io/otel/exporters/stdout/stdouttrace
• 5376a756: bump github.com/rabbitmq/amqp091-go from 1.9.0 to 1.10.0
• d82ab343: bump golang.org/x/net from 0.24.0 to 0.25.0
• 453d2c60: bump github.com/urfave/cli/v2 from 2.27.1 to 2.27.2
• 5323fa31: bump go.opentelemetry.io/otel/exporters/stdout/stdouttrace
• 3e1f5c15: bump go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace
• 71078832: bump go.opentelemetry.io/otel from 1.25.0 to 1.26.0
• 1006287a: bump go.opentelemetry.io/otel/exporters/stdout/stdouttrace
• 43f3a3e4: bump go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace
• 343515af: bump ...

v4.7.4 Release

Unreleased

v4.7.4 - 2024-05-01

NOTE

The default layer download location has changed

Claircore

• tarfs: follow hardlinks in ReadFile

  This makes `fs.ReadFile` work as expected when opening hardlinks.

• debian: update how "source" packages are handled

  Previously, the Updater parsed metadata from the repository to try to record only "binary" packages. This was inaccurate and, with the new dpkg handling, now unneeded. The new approach should be more accurate.

• dpkg: improve Source handling

  The dpkg handling machinery now correctly records source packages and versions. Previously, version differences between a source package and the resulting binary package(s) were incorrect if the versions were not identical.

• libindex: add O_TMPFILE fallback logic

  After discovering that some common deployment methods are incompatible with using the `O_TMPFILE` `open(2)` flag, a fallback path has been added. The changes also move the default location of where temporary files are downloaded to, to better align with the layout recommended by systemd.

  Please see the documentation for specifics.

  26-0.20240325212310-fedb9d327aa7#NewRemoteFetchArena

• osv: parse database_specific severity when no CVSS severity is defined

  Occasionally there are OSV advisories that don't include any severity information in the `.severity` object but they do contain a severity in the `.database_specific` object. This change attempts to parse that severity if we don't get a severity from the native `.severity` object.

Build(Deps)

• 3ebd889c: bump peter-evans/create-pull-request from 6.0.0 to 6.0.1
• b7566a0f: bump peter-evans/create-pull-request from 5.0.2 to 6.0.0
• 4db2f09b: bump actions/cache from 3 to 4
• 6cef8311: bump actions/upload-artifact from 3 to 4
• 5ed80215: bump actions/download-artifact from 3 to 4
• c9e1f56b: bump actions/setup-go from 4 to 5
• 3ab3de55: bump actions/stale from 8 to 9
• 591188f0: bump docker/setup-buildx-action from 2 to 3
• 7ef6ef6b: bump docker/login-action from 2 to 3
• 5597e7cc: bump docker/build-push-action from 4 to 5
• 14d7f2b4: bump docker/setup-qemu-action from 2 to 3
• 1204db98: bump actions/checkout from 3 to 4

Chore

• 4170798b: 4.7.4 changelog bump
• 96dc6074: Add merge step when creating release binaries
• a1c7eb7c: update go version for release
• 6eeb9393: update claircore to v1.5.27
• 809dd5ab: update go version

Cicd

• e6378d03: add container version skew check
• 2ba3ecc0: update testing workflow
• ae135c49: don't upload workspace on failure
• 7222dc88: change version specifiers to be major-version only

Clairctl

• 2a2ba37f: warn when range requests are not honored

Dockerfile

• 5547b96a: remove sh loop

Docs

• 3753415b: add mention of disk space path and usage

Httptransport

• c6df986f: GET vuln report returns 404 when indexing in-progress

Initialize

• 9828576a: use defaults for NewRemoteFetcher

v4.7.3 Release

Highlights:

• The minimum TLS version is now 1.2.
  Previously, servers also allowed 1.1 connections.

• Claircore is updated to v1.5.25:

  • rhcc, rhel: support compression of sideband data

    If a Clair instance is using local files for the data needed for the `rhel` and `rhcc` indexers, this data may now be compressed. This should allow for the files to fit within a Kubernetes ConfigMap, making some deployments easier to wrangle.

  • datastore: add "delta" update interface

    This change should allow for updaters to use fewer resources and consume API-based data sources in the future. As of this change, no in-tree updaters have been converted to this interface.

  • java: size buffers correctly before use

    This should reduce memory consumption for indexing layers that have deeply nested Java archives.

  • postgres: remove internal timeouts

    Database queries now take as long as needed to execute. This shouldn't negatively affect any working uses, and should make some slower or less-optimized queries possible on larger instances.

  • integration: make PGVERSION a pattern

    The behavior of the setup of an embedded PostgreSQL in integration tests has changed. The relevant environment variable (`PGVERSION`) is now a pattern instead of a literal version string. Note that a version string would be a patten that matches itself, so that format continues to work.

    Additionally, the version used is now read from the distributed
    manifest, rather than hard-coded versions. Other than occasional network
    calls to fetch this manifest, users shouldn't notice any difference.

  • alpine: add edge support

    Alpine's `edge` version should now be supported for reporting.

  • rpm: support PGP V4 signatures

    Rpm has apparently started using "current"/V4 PGP signatures, which claircore was not handling. This adds support for these signatures.

  • jsonblob: add a disk buffering step

    This improves "offline" operation by eagerly buffering output to disk instead of creating a large in-memory data structure first.

    This makes the API trickier but given that there's a single (known and
    intended) user, this should be fine.

  • tarfs: check a potential interger overflow

    This change fixes a potential integer overflow in tar handling code.

    The possibility of exploiting this is effectively 0, as it would require
    more bytes to represent a sufficiently large integer than is available
    in the tar header.

    See also: https://github.com/quay/claircore/security/code-scanning/5

  • gobin: take into account package replacements

    Previously, there was a bug where package replacements were not considered for go binaries.

  • all: purge http.DefaultClient usage

    Some packages with less churn (`photon`, `oracle`, `aws`) were using older ways of getting an `*http.Client` or using `http.DefaultClient`.

    This change breaks some API in exchange for unifying the *http.Client
    handling. The practical upshot is that it's much easier to control the
    network contact surface.

  • all: share single FS implementation

    Claircore components that deal with `Layer` objects now share a single backing File and a single `fs.FS` implementation when using the `FS` method. There should be no noticeable changes for users, but out-of-tree implementations may want to move over to using the new FS method.

    This change should improve memory usage.

  • libindex: move to O_TMPFILE fetcher

    This release uses a new fetcher (the component responsible for pulling layers locally) that makes use of the O_TMPFILE flag to open(2). This ensures that layer files will be cleaned up even in the event of an unclean shutdown, including being sent a KILL signal.

v4.7.3 - 2024-02-26

Admin

• 9517c7be: add a check for compatible migration version
  See Also: #1915

Chore

• e5a896c9: v4.7.3 changelog bump
• d17ee97b: update claircore to v1.5.25
  See Also: #1990, #1957, #1942

Config

• 6ba32131: update minimum TLS version for server
  See Also: #1945

v4.7.2 Release

Unreleased

v4.7.2 - 2023-10-09

Claircore

• chore: update claircore to v1.5.19
• crda: remove crda support
  The CRDA API has been decommissioned and the functionality has been superseded by OSV support.
• chore: update toolkit to latest version v1.1.1
  v1.5.17 (toolkit/v1.1.0) introduced a bug where claircore could not handle empty strings when trying to Scan() a value into a cpe.WFN. toolkit/v1.1.1 mitigates this bug.

Clair

• admin: add pre v4.7.3 admin command to create index

  In order to facilitate faster deletes we need to add a migration to add an index in v4.7.3. This change adds an admin command to allow users to "manually" create the index CONCURRENTLY before the migration to avoid any down-time. This is something for users with larger indexer DBs to consider.

• contrib: add grafana dashboards for deletion metrics

  This has been a part of the API for some time and is starting to be used extensively.

• docs: add dropins to prose documentation

  This change explains how to use the dropins and updates the local-dev config to do so.

v4.7.1 Release

Unreleased

v4.7.1 - 2023-08-10

Build(Deps)

• bd4bdbf6: bump github.com/pyroscope-io/godeltaprof

Chore

• 25ab0f4e: bump claircore to v1.5.15
• 4bf37a11: bump claircore to v1.5.14

v4.7.0 Release

Unreleased

v4.7.0 - 2023-07-27

Auto

• 1e574c25: enable mutex, blocking profiles by default

Build(Deps)

• adee21df: bump golang.org/x/net from 0.11.0 to 0.12.0
• 32c9ae2e: bump github.com/klauspost/compress from 1.16.6 to 1.16.7

Chore

• 1bfbfa1b: bump claircore to v1.5.13
• 31cf5570: Bump claircore to v1.5.12
• 2d2d16a1: Bump claircore to v1.5.11
• 048ad2f1: Bump claircore to v1.5.10
• 5550b27a: bump Claircore to v1.5.9
• 7df2b863: add pyroscope to compose setup
• c28648e5: Update outdated docs and comment with default update period.
• a02a0f2f: remove refs to deprecated io/ioutil
• 44638edf: Remove dogstatsd variable and references

Clairctl

• bccabff1: Add post 4.7 admin command to delete pyupio vulns
• d2b3d826: Scan the pointer to the pointer of the bool
• 05bd8fa0: Add log line signifying admin is done
• c636e207: Remove DSN logging
• 89cae779: admin subcommand

Cmd

• 8231b438: version for old gits
• #882### Config
• 83ee24af: pick a real versioning scheme

Contrib

• 70d878eb: Add manifest for a Job to run DB jobs

Docs

• 394efe15: Fix up debug tools table
• a4ec17f6: Add description of debugging services available during local-dev

Httptransport

• 86f7a86a: add request ID to profiler labels

Introspection

• caba76e1: add delta pprof endpoints

v4.7.0-rc.1 Release

[Unreleased]

[v4.7.0-rc.1] - 2023-06-26

Airgap

• 94757c7d: Remove libindex Airgap option

All

• 5d30ed66: update to new config module

Build(Deps)

• 00a4279d: bump github.com/prometheus/client_golang
• f4f22e33: bump golang.org/x/net from 0.10.0 to 0.11.0
• 36a7c88c: bump github.com/klauspost/compress from 1.16.5 to 1.16.6
• 17cdc922: bump peter-evans/create-pull-request from 5.0.1 to 5.0.2
• b95be229: bump github.com/streadway/amqp from 1.0.0 to 1.1.0
• 45f808da: bump github.com/urfave/cli/v2 from 2.25.5 to 2.25.7
• b75a00c3: bump github.com/urfave/cli/v2 from 2.25.3 to 2.25.5
• 22a75603: bump github.com/google/go-containerregistry
• 300b1374: bump go.opentelemetry.io/otel/exporters/jaeger
• b2d7a091: bump github.com/urfave/cli/v2 from 2.3.0 to 2.25.3
• a21fb21d: bump go.opentelemetry.io/otel/exporters/stdout/stdouttrace
• b188cba7: bump github.com/quay/claircore from 1.5.2 to 1.5.3
• eb9d1225: bump golang.org/x/sync from 0.1.0 to 0.2.0
• f35c832f: bump golang.org/x/net from 0.9.0 to 0.10.0
• 3dbbaf7b: bump github.com/rs/zerolog from 1.29.0 to 1.29.1
• 1ee7cb8a: bump go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace
• dcb7a05a: bump go.opentelemetry.io/otel/exporters/jaeger
• fca257d7: bump go.opentelemetry.io/otel/exporters/stdout/stdouttrace
• 933cc5c7: bump github.com/ugorji/go/codec from 1.2.9 to 1.2.11
• 4f39b319: bump github.com/klauspost/compress from 1.16.4 to 1.16.5
• 3643f9d2: bump go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp
• c13eaecc: bump go.opentelemetry.io/otel/trace from 1.11.0 to 1.15.1
• 43e3daea: bump github.com/jackc/pgx/v4 from 4.18.0 to 4.18.1
• 2180bc40: bump gopkg.in/square/go-jose.v2 from 2.5.1 to 2.6.0
• f669244a: bump peter-evans/create-pull-request from 5.0.0 to 5.0.1
• 74bc404f: bump peter-evans/create-pull-request from 4.2.4 to 5.0.0
• 912c6e47: bump actions/stale from 7 to 8
• ddec3b43: bump peter-evans/create-pull-request from 4.2.3 to 4.2.4
• f35a3611: bump actions/setup-go from 3 to 4
• d3655eef: bump golang.org/x/net from 0.5.0 to 0.7.0
• 854a2fbf: bump docker/build-push-action from 3 to 4

Chore

• 9d58dba8: v4.7.0-rc.1 changelog bump
• 31823df2: bump Claircore to v1.5.8
• 836c0579: bump Claircore to v1.5.7
• e688e88b: bump Claircore to v1.5.6
• 3d61485d: bump Claircore to v1.5.5
• ddc4cc24: bump Claircore to v1.5.4
• 76686650: Add the osv updater to the local-dev config
• 56e63e8b: Update opentelemetry to v1.16.0
• 5df81b19: bump Claircore to v1.5.2
• cc0d9df4: bump Claircore to v1.5.1
• 35971dc9: produce nightly for ppc64le
• 471da4ee: Only ask dependabot to care about direct dependencies
• 62119209: updated nightly for s390x support
• 57774bd9: added s390x support
• 248a4733: move emulator tests to a nightly run
• bd0488ee: add gomod ecosystems to dependabot
• 8174e950: Remove 1.19
• efe27892: Bump Claircore to v1.4.22
• 1b857d13: Update go version in go.mod
• 5faf0fc9: Bump Claircore to v1.4.21
• a433c93c: Bump Claircore to v1.4.20
• d565775c: Add back GIT_HASH as needed for image name
• 12f38e45: Update go-image version in docker-compose manifest
• 02f311d5: Use our dedicated metric for the go version
• 896b2dfb: Update go version in Dockerfile
• d10c06e0: Bump claircore to v1.4.18

Cicd

• 58c26f4a: don't checkout source on clairctl builds
• 2eb10895: use common workflow in main module CI
• 83d9b2f5: use common workflow in config module CI
• e2f264f4: fix nightly connection strings
• 1ea95d83: rename yamllint config
• 7e2ae8fc: fix nightly-ci error
• 1267335e: use rabbitmq as STOMP broker in nightly CI
• 2edb4915: use rabbitmq as STOMP broker in tests
• 74c34c0c: update nightly job to work
• 30a98697: update go versions

Clair

• 5226d2a3: use new cmd.LoadConfig

Clairctl

• 06f5bc05: use new cmd.LoadConfig

Cmd

• 3ff924ad: implement modular configs
• d3e88775: better version information

Config

• cee776b3: add newtype for Durations
• [1ebbbf2](https://gi...

v4.6.1 Release

Unreleased

v4.6.1 - 2023-04-13

Airgap

• e02aba27: Remove libindex Airgap option

Chore

• 36990912: v4.6.1 changelog bump
• e676671c: Bump Claircore to v1.4.21

Go.Mod

• 36de97cc: update json (de)serializer

Httptransport

• 922f33d1: fix request_id logging

Httputil

• 9e8eacf5: fix ParseIP usage
• #1689### Notifier
• ffa4556d: Avoid double reference

v4.6.0 Release

Unreleased

v4.6.0 - 2023-01-20

All

• 577a55d4: use httputil to construct requests

Auto

• 1f1010fe: add automatic memory limit discovery

Build(Deps)

• ef896eb6: bump actions/stale from 6 to 7
• 5a212ffe: bump peter-evans/create-pull-request from 4.1.4 to 4.2.3
• b883bc2b: bump gsactions/commit-message-checker from 1 to 2

Chore

• 5fd26563: v4.6.0 changelog bump
• 33f4fcbd: Bump claircore to v1.4.17
• 54d44908: Bump Claircore to v1.4.16
• 430e6087: Bump Claircore to v1.4.15
• 652d8ce6: Bump Claircore to v1.4.14
• 9f6828cd: Update to Go 1.18 for local-dev
• 1c002bcd: added ppc64le support
• 4b37dcdf: Bump Claircore to v1.4.13
• 9b273420: Bump claircore to v1.4.12

Cicd

• 1dfb42a0: use extracted git archive
• aff17a4a: update usage of set-output
• a8a97f80: update cache action
• 7de63a9c: add tests for odd architectures
• e923360c: omit Dockerfile build args
• 14b8f690: enable go1.19
• 5a8128c1: inject version into built clairctl binaries
• #1649### Clairctl
• a367a7ae: use a better user-agent
• 3b9ff6de: update with new signer

Client

• ddea858f: Add the passed host to the signer
• adbaa567: use signer
• d8ad1ba4: update for httputil changes

Cmd

• 8b899803: use git-archive for version information

Documentation

• 9d1a7aab: fix typo in link

Httptransport

• 25ac033f: use new signer scheme in test
• a9228d40: add a request_id to logs
• #1547### Httputil
• e746ff05: rework request signing and request restriction

Service

• e08f3972: add signer option

Webhook

• d99f7005: add explicit signer argument

v4.5.1 Release

[Unreleased]

[v4.5.1] - 2022-11-09

Chore

• 0a0aa1cc: Bump claircore to v1.4.12
• #1646
  [Unreleased]: v4.5.1...HEAD
  [v4.5.1]: v4.5.0...v4.5.1