Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ci: Scheduled Trivy Scans of Eraser Images #858

Merged

Conversation

inFocus7
Copy link
Contributor

@inFocus7 inFocus7 commented Sep 5, 2023

What this PR does / why we need it:
We currently run Trivy scans during test workflow runs, but want to run them more frequently.

This adds a cron job to run Trivy scans every Monday at 7:00 AM UTC. Those scans are ran against eraser images built on main branch.

It also allows for manual dispatching to do above and allows running scans of specific release versions.

Each run's vulnerability can also be published to our security tab (I couldn't find a way to make them into issues) if either ran as part of the cron/schedule, or if in our manually dispatch we enable it.

I also added some workflow documentation, only information regarding this action is there, so any new developers can understand it better (hopefully).

Which issue(s) this PR fixes (optional, using fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when the PR gets merged):
Fixes #783

Special notes for your reviewer:

  • The matrix is a teeny bit weird. Mainly because each build command is different and the environment variable needed for them is as well so I added those necessary variables to the matrix.
  • Ran the scans using the current code in a separate fork
  • Results (that had vulnerabilities) are recorded in the security tab
    • If you see a lot added by Trivy in my results, it is likely scanned from v1.1.0 image, since I wanted our oldest release with the highest possible CVEs.

example of a vulnerability caught and recorded, scanning our released v1.1.0 images
as far as i've seen we can't really update the title or metadata, as these are generated straight from the SERIF file generated by the trivy results
Screen Shot 2023-09-07 at 7 44 09 PM

There are three runs:

  • One leaving the version input blank (default), which should be building aginst the branch it ran against, then testing those images.
  • One setting the version to the latest beta release, which skips building images, and scans those images.
  • A scheduled run

Signed-off-by: Fabian Gonzalez <fabiangonz98@gmail.com>
Signed-off-by: Fabian Gonzalez <fabiangonz98@gmail.com>
Signed-off-by: Fabian Gonzalez <fabiangonz98@gmail.com>
Signed-off-by: Fabian Gonzalez <fabiangonz98@gmail.com>
.github/workflows/scan-images.yaml Outdated Show resolved Hide resolved
.github/workflows/scan-images.yaml Outdated Show resolved Hide resolved
.github/workflows/scan-images.yaml Outdated Show resolved Hide resolved
Signed-off-by: Fabian Gonzalez <fabiangonz98@gmail.com>
Signed-off-by: Fabian Gonzalez <fabiangonz98@gmail.com>
Signed-off-by: Fabian Gonzalez <fabiangonz98@gmail.com>
Signed-off-by: Fabian Gonzalez <fabiangonz98@gmail.com>
@inFocus7
Copy link
Contributor Author

inFocus7 commented Sep 8, 2023

The trivy image we use (44.0) has a vulnerability. I ran scans against 44.1 & 45.0 (latest release) and both of those have it as well, so no version we can upgrade to (yet).

Not sure if we want to skip the trivy scan in the test for this pr (unsure if possible). I submitted a report just in case it hasn't been noticed yet on Trivy's end. If it gets fixed tomorrow that'll be very :nice:


Update
Was bumped with chore bot, so we'll just need to wait on a Trivy release, then bump our build of it. If we want to ignore it. If so, we may be able to create a .trivyignore file in root and add the vulnerability, but probably best to wait and not to have any.

Also not sure if backports are done in Trivy or how often, so worst-case we'd need to do a major bump from 44.X to 45.X.

@ashnamehrotra
Copy link
Contributor

The trivy image we use (44.0) has a vulnerability. I ran scans against 44.1 & 45.0 (latest release) and both of those have it as well, so no version we can upgrade to (yet).

Not sure if we want to skip the trivy scan in the test for this pr (unsure if possible). I submitted a report just in case it hasn't been noticed yet on Trivy's end. If it gets fixed tomorrow that'll be very :nice:

Update Was bumped with chore bot, so we'll just need to wait on a Trivy release, then bump our build of it. If we want to ignore it. If so, we may be able to create a .trivyignore file in root and add the vulnerability, but probably best to wait and not to have any.

Also not sure if backports are done in Trivy or how often, so worst-case we'd need to do a major bump from 44.X to 45.X.

We can wait for the trivy release to fix this, thanks for checking!

@inFocus7
Copy link
Contributor Author

inFocus7 commented Sep 17, 2023

@ashnamehrotra

Trivy just released a new version (0.45.1). Scanned it locally and no CVEs, so it should be good to pass now! 🎆

@ashnamehrotra
Copy link
Contributor

@inFocus7 you will also need to update the go.mod for the test to pass

Signed-off-by: Fabian Gonzalez <fabiangonz98@gmail.com>
@inFocus7
Copy link
Contributor Author

inFocus7 commented Sep 19, 2023

@ashnamehrotra Thanks! I'll update if it fails after this run where I saw I didn't update the tag in Makefile.
Doing a bump of trivy to 0.45.1 in go.mod led to a lot of k8s updates. I see we have replaces in hand, but didn't feel comfortable/want to risk break anything yet, if not necessary 😅

Update

Looks like the Trivy scan failed again. It is no longer due to the cyphar/filepath-securejoin CVE, which is good since it means it's using the latest Trivy release (0.45.1)... But, the updated Trivy image (which I also scanned locally trivy image ghcr.io/aquasecurity/trivy:0.45.1) has a CVE. 😢

Trying to think of workarounds:

  • Add a .trivyignore to ignore the CVE.
    • Good for now, but we'd want to update again it in the future once Trivy updates and CVE is good.
  • Skip or remove the Trivy scan portion of the test.
  • Don't skip, but don't fail/block CI on check failure.
    • I feel like it should be possible to make the check optional, but not 100% certain at this moment.

@ashnamehrotra
Copy link
Contributor

ashnamehrotra commented Sep 26, 2023

@inFocus7 I added the trivy ignore for now, thank you!

@ashnamehrotra ashnamehrotra merged commit 4e61997 into eraser-dev:main Sep 26, 2023
84 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

track trivy eraser image vulnerabilities as a recurring job
4 participants