fix: package-lock v3 missing sub-dependencies - UNIFY-258

[neil-snyk]

NPM lock-file V3 - missing transitive dependencies

Links

JIRA Ticket

UNIFY-258

Background

Problem

A customer discovered that when scanning a container that contains an NPM lock-file that is V3 no transitive dependencies are identified, only the direct
dependencies.

Impact

Due to that all transitive dependencies are missed there is a risk of not identifying serious vulnerabilities in customers containers.

Root cause

The cause of the issue was that when creating the depgraph from npm and yarn lock-files the snyk-docker-plugin was first creating a deptree using Snyk's
nodejs-lockfile-parser which was then converted to a depgraph. The nodejs-lockfile-parser function that creates deptree's has a bug and does not generate
the transitive dependencies for V3 npm lock-files. As deptrees are large structures, resulting in high memory usage and high CPU costs from serializing,
it is better to create depgraphs directly instead.

Change

Now deptrees are only created for npm V1 lock-files, for all other versions of npm and yarn lock-files, the deptree creation is skipped and a depgraph is directly created.
The logic in snyk-docker-plugin was also modified to mimic the CLI when scanning stand-alone node applications so
that container always produces the same result for containers containing node applications.

Testing

Unit tests were added for a couple of new functions and function tests were also added to check that the correct number of dependencies are identified in containers with
v3 npm lock files and that the number of identified dependencies is the same in V1, V2 and V3. Manual testing was also done to compare the result of the current version
of the CLI and a locally built CLI with a version of the snyk-docker-plugin with the changes mentioned above. There it could be seen that the local build of the CLI
identified the correct number of dependencies in a V3 npm lock-file and that that results from scanning a V1 and V2 npm lockfile was unchanged. Also containers with
yarn lock-files still returned the correct number of dependencies after the changes in snyk-docker-plugin.

Impact

This change will only impact customers that have container projects that contain node applications that use V3 npm loc-files. These customers will get a bump in identified
dependencies due to that not only direct dependencies will be detected but also transitive dependencies. This could also lead to new vulnerabilities being reported, which could
lead to pipelines failing. The amount of newly found dependencies and vulnerabilities that will be identified is hard to predict, but for example the container that the
customer added to the ticket was identified as having nearly 10 times the amount of dependencies compared to before these changes.

Benefits

The benefit is that customers will correctly identify all dependencies in containers that contain V3 npm lock-files and thereby correctly identify all vulnerabilities.

Future considerations

This change could lead to snapshot churn and that a number of customers pipelines could break due to that vulnerabilities are found in the transitive dependencies that are returned
in the depgraph. How much depends on how widespread the use of V3 npm lock-files is among our customers, something we currently don't have a metric, this is something that perhaps
should be added in the future.

[SteveShani]

Is there more information you can add to the error message?

I'm worried we might get "Failed to build dep graph from current project" in a zendesk ticket one day and we'd have no idea how to go from there.

[snyk-team-unify]

🎉 This PR is included in version 6.13.7 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀