[Security] v2.10.2 contains 35 fixable vulnerabilities

[hpvd]

Search before asking

• I searched in the issues and found nothing similar.

Version

v2.10.2

Minimal reproduce step

look into trivy powered inspection for vulnerabilities
at artifacthub.io
https://artifacthub.io/packages/helm/apache/pulsar?modal=security-report

open details of in the latest helm chart v3.0.0 included pulsar v2.10.2 image

What did you expect to see?

very few fixable vulnerabilities, since v2.10.2 was released just 8 days ago https://github.com/apache/pulsar/releases

What did you see instead?

• 72 vulnerabilities have been detected in the image
• 35 of these should be fixable (most with a version bump of dependencies)

Anything else?

• this is related to
  [Security] v2.10.2 contains up to 9 year old vulnerabilities/CVEs -> get rid of the oldest #18338
• this is a follow up of
  Arguments for "why pulsar is secure?" #18041
• this is part of
  [security] further reduction of the 136 vulnerabilities (79 fixable) in helm chart v3.0.0 pulsar-helm-chart#334

Are you willing to submit a PR?

• I'm willing to submit a PR!

[michaeljmarshall]

Several of these have been fixed, and others are brought in by presto. @nicoloboschi does security scanning of the pulsar code base and frequently contributes updates to dependencies, when possible.

[hpvd]

yes and there is also an automated OWASP Dependency Check running since June 2021 #10855

doing all this,
our current procedure/routine seems to miss these 35 fixable vulnerabilities (CVE) when releasing latest version 2.10.2
okay a (very) few less... see #8815 (comment)
-> but that's the state of our latest, only 8 days old release.
not sooo bad, but not so good either.

[Jason918]

-> but that's the state of our latest, only 8 days old release.
not sooo bad, but not so good either.

The final release candidate of 2.10.2 is 23 days ago. And most of fixes comes in the early September.

I think the problem is that the current release process have too many work and not automated. It took too long for a version to be released. And I don't see any check process of CVEs before we cut the release. I am pretty sure "OWASP Dependency Check" passed before the first release candidate is cut.

[hpvd]

Looks like there are several points helping to produce this non-perfect state:

1. I absolutely agree with @Jason918 that's probably part of the problem, that the time from last fixes to final release may be too long, so very new findings and fixes were not included (but there are also very old ones in the list..)
2. Do we have already a named point in the release check list, that applying all (as many as possible) available fixes before any release is a real must?
   (=quality gate, similar to Automated security and update routine before every release #8815)
3. Another part may be the manual and not time constraint suppression of positive findings (for what ever reason)
   like named here [Security] Owasp dependency check: check if suppressing of vulnerabilities is still reasonable #17068
4. We not always use the latest version of the check tool.
   We use 7.1 from April 2022, latest is 7.3 see https://github.com/jeremylong/DependencyCheck/releases
5. Maybe the tool we use today and its reports are not as user-friendly and useful as it could be.
   If so, one option would be to move to trivy which is e.g. used for the great security overviews at artifacthub.io (see above), to
   • easily distinct between general and fixable vulnerabilities and
   • easily show to which version one should update the dependency...
     a githubaction for trivy is available https://github.com/aquasecurity/trivy-action
6. ...
7. what else do we have to consider?

[michaeljmarshall]

I think the problem is that the current release process have too many work and not automated. It took too long for a version to be released.

I agree. I think we're moving in the right direction by breaking out the C++ and Python clients, but it will take time to see those benefits. Maybe it is worth a mailing list discussion of how to share the load on the release manager?

[Jason918]

I think the problem is that the current release process have too many work and not automated. It took too long for a version to be released.

I agree. I think we're moving in the right direction by breaking out the C++ and Python clients, but it will take time to see those benefits. Maybe it is worth a mailing list discussion of how to share the load on the release manager?

Yes, this is one of the motivations of PIP-175: Extend time based release process

[hpvd]

when looking through the security report again,
I was wondering why there is netty version 3.10.6 listet with 3 named vulnerabilities.
This is a version from the year 2016 (https://mvnrepository.com/artifact/io.netty/netty/3.10.6.Final)
and as far I have seen, we have upgraded to >netty 4.1
e.g. in #15212

the question:
Can it also be another (small) part of the problem, that we only upgrade "some points of use" in the code when there is a notification about a security issue (and think everything solved)?

[hpvd]

to get the whole picture, just added some release dates of the versions with fixes:

[hpvd]

hmm just another number: the ubuntu 20.04 included, added 77 vulnerabilities with CVE...

[Jason918]

I was wondering why there is netty version 3.10.6 listet with 3 named vulnerabilities.

This is from presto-sql/pulsar-presto-distribution. presto-sql contains a lot of legacy dependencies and not easily updated. Previously, I saw @tisonkun working on some updates to Trino in PulsarSQL. Not sure if it's related.

@hpvd I think you can start a discuss thread on the mail-list to get more attention on this matter.

[tisonkun]

@Jason918 No. The upgrade of Trino (PrestoSQL) dependency happened only for the unreleased master (2.12.0).

[tisonkun]

And yes, if we don't update Trino dependency for 2.10 (I think so), these reports should be suppressed.

[github-actions]

The issue had no activity for 30 days, mark with Stale label.

[compuguy]

This issue is still relevant with release 3.1.0. Plenty of vulnerabilities in the latest release, especially with the connectors and offloaders...

Edit: this also applies to 3.1.1 and upcoming release 3.0.2. See #21457 for more information.

[hpvd]

when stale label is added automatically,
it should also be removed automatically if an issue is newly commented or maybe even if it's mentioned some else.

edit: added an issue for this:
#21458