Integrate vulnerablecode

[sameer1046]

No description provided.

[pombredanne]

@sameer1046 yes! @sbs2001 ping
Design-wise, what would be your idea?

[sbs2001]

Not super familiar with scancode.io, but here's a first draft : the use case here would be along the lines of "extract packages + their licenses + their vulnerabilities(if any) "

So we have the "extract packages + their licenses" part covered at the scan_codebase pipeline at https://github.com/nexB/scancode.io/blob/main/scanpipe/pipelines/scan_codebase.py and need to enrich the report with vulnerabilities.

I can think of having a vulnerablecode Pipe , which can be plugged in to the other pipelines like the scan_codebase. This Pipe will do the following :
(1) Take in a CodebaseResource (which should be scanned for packages obviously ) .
(2) Collect all DiscoveredPackage related to it. Collect their purls.
(3) Make a bulk_search api call to vulnerablecode server with purls obtained in (2) . More or less aboutcode-org/vulnerablecode#303 (comment)
(4) Iterate on the response given, for each requested DiscoveredPackage enrich it with vulnerablecode data. To do this we need to do some changes in models.

Model Changes:-

The vulnerability data should be on a package level. So changes must be done at DiscoveredPackage . I can think of having a many2many relation of a DiscoveredPackage with a new model Vulnerability. Essentially mirroring the whole https://github.com/nexB/vulnerablecode/blob/137d88a1a4eb319259330fdfe4c47dbc95808850/vulnerabilities/models.py#L84 . with minor changes. This is because we not only want to know whether a package is vulnerable, but also the severity of each.

@sschuberth ping, you guys are doing the same thing at ort right ? we can borrow some ideas here :)

[sschuberth]

@sschuberth ping, you guys are doing the same thing at ort right ? we can borrow some ideas here :)

Sure, feel free to have a look at our VulnerableCode integration as a ORT advisor, which enriches package metadata with security vulnerability information by leveraging our generic Kotlin / Java client for the VulnerableCode service.

[pombredanne]

Sure, feel free to have a look at our VulnerableCode integration as a ORT advisor, which enriches package metadata with security vulnerability information by leveraging our generic Kotlin / Java client for the VulnerableCode service.

@sschuberth that's awesome! Do you think this could be pushed as a separate reusable maven package of sorts for other could benefit from this?

[sschuberth]

Do you think this could be pushed as a separate reusable maven package of sorts for other could benefit from this?

Definitely. That has been the plan for all our libraries in the clients directory, and should be done as part of oss-review-toolkit/ort#2906. Someone just needs to get it done 🙄

[tdruez]

@sbs2001 is there a VulnerableCode instance available to start making "bulk_search api call"?

[sbs2001]

@tdruez atm no. But you can spin up one quickly using docker-compose and running some small importer

[tdruez]

@sbs2001 Any plan to deploy a VulnerableCode instance in the near future?

[sbs2001]

@tdruez that's on @pombredanne TODO list

[tdruez]

Completed in #551