Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

admin: add command to update go packages with norm_version #1915

Merged
merged 2 commits into from
Nov 27, 2023

Conversation

crozzy
Copy link
Collaborator

@crozzy crozzy commented Nov 16, 2023

There are a number of Go packages in the package table that do not have the needed norm_version which is required for Go matching.

This looks up the potential Go packages from the matcher.vuln table, the difficultly doing this is entirely in the indexer DB is how to identify that these packages came from the gobin scanner. Trying to use the package_scanartifact (joining to the scanners table is prohibitively expensive as the query path is not accounted for in the index (i.e. we're always expecting the first clause to the the layer_id).

@crozzy crozzy requested a review from a team as a code owner November 16, 2023 18:07
@crozzy crozzy requested review from hdonnay and removed request for a team November 16, 2023 18:07
@crozzy
Copy link
Collaborator Author

crozzy commented Nov 16, 2023

Local output:

 ► go build -o . ./cmd/clairctl && ./clairctl -D -c local-dev/clair/config.yaml admin oneoff update-golang-packages
2023-11-16T10:00:32-08:00 INF using discovered connection params database=matcher host=localhost port=32775 user=clair
2023-11-16T10:00:32-08:00 DBG resizing pool pool size=2
2023-11-16T10:00:32-08:00 INF using discovered connection params database=indexer host=localhost port=32775 user=clair
2023-11-16T10:00:32-08:00 DBG resizing pool pool size=2
2023-11-16T10:00:32-08:00 DBG working on version package_name=k8s.io/apiserver version=v0.27.4
2023-11-16T10:00:32-08:00 INF successfully updated package row package_id=3818 package_name=k8s.io/apiserver rows affected=1 version=v0.27.4
2023-11-16T10:00:32-08:00 DBG working on version package_name=stdlib version="go1.20.10 X:strictfipsruntime"
2023-11-16T10:00:32-08:00 WRN error parsing semver error="Invalid Semantic Version" package_name=stdlib version="go1.20.10 X:strictfipsruntime"

@hdonnay
Copy link
Member

hdonnay commented Nov 16, 2023

Do we want this to be go specific, or couch it as a general "fix-up vulnerability table" command?
I don't feel strongly, other than my initial instinct was to do the latter.

@crozzy
Copy link
Collaborator Author

crozzy commented Nov 16, 2023

Do we want this to be go specific, or couch it as a general "fix-up vulnerability table" command? I don't feel strongly, other than my initial instinct was to do the latter.

I think the issue with it not being go-specific is; is the version parsing logic and the norm_kind value applicable to any other ecosystems? Most matchers aren't relying on that value, are you thinking that we could make it general purpose and convert some more matching logic over to do func (matcher *Matcher) VersionAuthoritative() bool { return true }?

@hdonnay
Copy link
Member

hdonnay commented Nov 16, 2023

I meant for the command itself -- are we ever going to need another update-someother-package command?

@crozzy crozzy force-pushed the update-go-packages-command branch 2 times, most recently from 15ad6c3 to 68780ec Compare November 17, 2023 18:44
hdonnay
hdonnay previously approved these changes Nov 17, 2023
Copy link
Member

@hdonnay hdonnay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@crozzy crozzy force-pushed the update-go-packages-command branch 3 times, most recently from 4b8bb00 to 301b7b7 Compare November 27, 2023 17:55
There are a number of Go packages in the `package` table that
do not have the needed norm_version which is required for Go
matching.

Signed-off-by: crozzy <joseph.crosland@gmail.com>
Add the ability to check migration version of the database before
running an admin command so you know you are operating on a compatible
database schema.

Signed-off-by: crozzy <joseph.crosland@gmail.com>
@crozzy crozzy force-pushed the update-go-packages-command branch from 301b7b7 to 1f2b34f Compare November 27, 2023 19:08
@crozzy crozzy merged commit d53780b into quay:main Nov 27, 2023
hdonnay pushed a commit to hdonnay/clair that referenced this pull request Feb 14, 2024
There are a number of Go packages in the `package` table that
do not have the needed norm_version which is required for Go
matching.

Backports: quay#1915
Signed-off-by: crozzy <joseph.crosland@gmail.com>
(cherry picked from commit 9cac271)
hdonnay pushed a commit to hdonnay/clair that referenced this pull request Feb 14, 2024
Add the ability to check migration version of the database before
running an admin command so you know you are operating on a compatible
database schema.

Backports: quay#1915
Signed-off-by: crozzy <joseph.crosland@gmail.com>
(cherry picked from commit 1f2b34f)
hdonnay pushed a commit to hdonnay/clair that referenced this pull request Feb 14, 2024
There are a number of Go packages in the `package` table that
do not have the needed norm_version which is required for Go
matching.

Backports: quay#1915
Signed-off-by: crozzy <joseph.crosland@gmail.com>
(cherry picked from commit 9cac271)
Signed-off-by: Hank Donnay <hdonnay@redhat.com>
hdonnay pushed a commit to hdonnay/clair that referenced this pull request Feb 14, 2024
Add the ability to check migration version of the database before
running an admin command so you know you are operating on a compatible
database schema.

Backports: quay#1915
Signed-off-by: crozzy <joseph.crosland@gmail.com>
(cherry picked from commit 1f2b34f)
hdonnay pushed a commit to hdonnay/clair that referenced this pull request Feb 26, 2024
There are a number of Go packages in the `package` table that
do not have the needed norm_version which is required for Go
matching.

Backports: quay#1915
Signed-off-by: crozzy <joseph.crosland@gmail.com>
(cherry picked from commit 9cac271)
Signed-off-by: Hank Donnay <hdonnay@redhat.com>
hdonnay pushed a commit to hdonnay/clair that referenced this pull request Feb 26, 2024
Add the ability to check migration version of the database before
running an admin command so you know you are operating on a compatible
database schema.

Backports: quay#1915
Signed-off-by: crozzy <joseph.crosland@gmail.com>
(cherry picked from commit 1f2b34f)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

2 participants