x/vuln: delay reporting vulnerabilities for N days

[adamdecaf]

Once a new Go bugfix is released govulncheck will start to report issues from the standard library if security flaws were fixed in the latest bugfix. This causes a lot of failed builds across a fleet of CI jobs because the underlying machine may not have been upgraded yet. Failing builds for an issue that teams will address but haven't had time to solve yet causes frustration and pushes teams to silence warnings or rush through upgrades.

One option would be to delay reports for a time.Duration value. This would let teams upgrade dependencies and Go versions before seeing failures.

I'm proposing that the govulncheck command accepts a -stability-days N flag where N is a value parsable by time.ParseDuration. The check could read a vulnerabilities published field in the OSV format and use the machines local time for comparison.

Prior art: https://docs.renovatebot.com/configuration-options/#stabilitydays

[adamdecaf]

I opened https://go-review.googlesource.com/c/vuln/+/442497 to see how this would look in the code. I'm open to a different name if the flag is accepted.

[zpavlinovic]

Thanks for reporting this, we are currently looking into this. One question, is this problem mostly tied to std vulnerabilities or do you expect similar issues with third party modules as well?

[adamdecaf]

Initially we saw this with stdlib vulnerabilities, but it would be a similar request to delay when popular libraries are used. At my day job we have enough repositories and commonly used libraries that it can take a couple days to upgrade even if everyone has it as a high priority.

[zpavlinovic]

The solution for this issue falls under an umbrella of suppression mechanisms for govulncheck. We also thank for raising this issue and implementing a fix, but we believe that a more general suppression option is a better fit for this problem than adding a rather specific flag. We are actively exploring options in this space, but we don't have a timeline or decision yet.

One way to solve this problem is to analyze JSON output and then filter vulnerabilities. OSV entries have the necessary information for this. As a matter of fact, we are actively developing new JSON output that would simplify this.

[adamdecaf]

Is this proposal still something which could be added to govulncheck? On minor releases of Go many CI systems run into immediate errors (detecting valid CVE's) but are unable to resolve them until meta-infra is updated. Right now a PR to actions/setup-go is blocking successful builds for many projects, but offering a delay would let the meta-infra be updated.

[gaby]

We are currently being affected by this, all our CI pipelines are failing because govulncheck, golang releases, actions/go, actions/setup-go are not synched.

Rel: actions/setup-go#405

[marcosrmendezthd]

an option to consider is providing a way to limit the vulnerabilities to releases, not prereleases.

[ldemailly]

1.20.7 was released, it's just it takes a couple of days for image and action providers to update, meanwhile normal CI/PR/MR start failing with no recourse (than disabling govulncheck)

[zpavlinovic]

We are currently being affected by this, all our CI pipelines are failing because govulncheck, golang releases, actions/go, actions/setup-go are not synched.

Rel: actions/setup-go#405

I am not sure what is the concrete issue in your case, but did you take a look at govulncheck-action? You can specify which Go version to use (with another option on the way).

[zpavlinovic]

While a general suppression mechanism would be ideal, it seems that most of the issues described here would go away with suppression for Go vulnerabilities. Is that fair to say?

If so, could an acceptable solution be part of govulncheck-action?

[marcosrmendezthd]

we had the same issue, so we updated our pipelines to find the latest (acceptable) go release version and pass that to actions/setup-go.

[adamdecaf]

We addressed the problem of recently released stdlib updates (from new Go releases) causing builds to fail by using setup-go with go-version: stable and installing govulncheck in each CI build. This ensures the project and govulncheck are on the same Go release (which can be the new or old based on cache) and avoids this error. stable checks for the latest Go release anyway.

[zpavlinovic]

Thanks for the prompt answers.

If it means anything, govulncheck-action should also install govulncheck on each CI build and it can be passed the Go version of choice.