Dependency's MIT license not picked up when scanning package-lock.json

[robertlagrant]

What happened
Outputting a CycloneDX SBOM does not pick up the sole dependency's licence in its output.

What you expected to happen:
Given I can see the dependency inside node_modules has a licence (MIT) in its package.json, and it also comes with a LICENSE file with the MIT licence text in it, I would expect the licence to appear in the cyclonedx output.

How to reproduce it (as minimally and precisely as possible):

% mkdir demo-syft-licence-bug
% cd demo-syft-licence-bug
demo-syft-licence-bug % npm init
<!snip!>
demo-syft-licence-bug % cat package.json
{
  "name": "demo-syft-licence-bug",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "Rob Grant <>",
  "license": "MIT"
}
demo-syft-licence-bug % npm install es-leftpad
added 1 package, and audited 2 packages in 2s
found 0 vulnerabilities
demo-syft-licence-bug %
demo-syft-licence-bug % demo-syft-licence-bug % syft file:package-lock.json -o cyclonedx
 ✔ Indexed package-lock.json
 ✔ Cataloged packages      [1 packages]
<?xml version="1.0" encoding="UTF-8"?>
<bom xmlns="http://cyclonedx.org/schema/bom/1.4" serialNumber="urn:uuid:fbd2213e-3623-413a-89e6-df0eb77043cf" version="1">
  <metadata>
    <timestamp>2022-07-21T09:47:20+01:00</timestamp>
    <tools>
      <tool>
        <vendor>anchore</vendor>
        <name>syft</name>
        <version>0.51.0</version>
      </tool>
    </tools>
    <component bom-ref="f11fc099fef741ab" type="file">
      <name>package-lock.json</name>
    </component>
  </metadata>
  <components>
    <component bom-ref="pkg:npm/es-leftpad@1.0.0?package-id=38a238232ed53007" type="library">
      <name>es-leftpad</name>
      <version>1.0.0</version>
      <cpe>cpe:2.3:a:es-leftpad:es-leftpad:1.0.0:*:*:*:*:*:*:*</cpe>
      <purl>pkg:npm/es-leftpad@1.0.0</purl>
      <properties>
        <property name="syft:package:foundBy">javascript-lock-cataloger</property>
        <property name="syft:package:language">javascript</property>
        <property name="syft:package:type">npm</property>
        <property name="syft:cpe23">cpe:2.3:a:es-leftpad:es_leftpad:1.0.0:*:*:*:*:*:*:*</property>
        <property name="syft:cpe23">cpe:2.3:a:es_leftpad:es-leftpad:1.0.0:*:*:*:*:*:*:*</property>
        <property name="syft:cpe23">cpe:2.3:a:es_leftpad:es_leftpad:1.0.0:*:*:*:*:*:*:*</property>
        <property name="syft:cpe23">cpe:2.3:a:es:es-leftpad:1.0.0:*:*:*:*:*:*:*</property>
        <property name="syft:cpe23">cpe:2.3:a:es:es_leftpad:1.0.0:*:*:*:*:*:*:*</property>
        <property name="syft:cpe23">cpe:2.3:a:*:es-leftpad:1.0.0:*:*:*:*:*:*:*</property>
        <property name="syft:cpe23">cpe:2.3:a:*:es_leftpad:1.0.0:*:*:*:*:*:*:*</property>
        <property name="syft:location:0:path">/Users/rob.grant/code/demo-syft-licence-bug/package-lock.json</property>
      </properties>
    </component>
  </components>
</bom>
demo-syft-licence-bug %

^ no licence listed in the cyclonedx output.

Anything else we need to know?:
It also doesn't appear if I output with SPDX format. If I try syft-json then it says "licences": [],.

Environment:

• Output of syft version:

demo-syft-licence-bug % syft version
Application:        syft
Version:            0.51.0
JsonSchemaVersion:  3.3.1
BuildDate:          2022-07-11T17:12:56Z
GitCommit:          470b13045bbbf150f3c79d1487a01ae6acc5592d
GitDescription:     v0.51.0
Platform:           darwin/arm64
GoVersion:          go1.18.3
Compiler:           gc

• OS (e.g: cat /etc/os-release or similar): OSX 12.4 Monterey.

[spiffcs]

Similar issue linked here! Thanks for filing the bug. We'll take a look and see what needs to be done to get licenses populated for dir scans.

#845

[robertlagrant]

@spiffcs thank you! Just to be doubly sure, the above was meant to be me running against a lockfile, and not doing a dir scan! I ran this: syft file:package-lock.json -o cyclonedx.

[spiffcs]

Thanks for the clarification!

It's an interesting case because it poses the question "Given some manifest file, should syft traverse extra noninput paths to find additional metadata information?"

package-lock.json does not have license information in and of itself, so syft in its current state is doing the best it can with the information provided.

I'm going to add it to our community meeting today for discussion since we've knocked around crossing this boundary before but have stayed on the side of reporting only on the truth of the input and not something the tool inferred.

At any rate we definitely should solve for dir:. in this case and conclude licenses for our raw JSON output when node_modules are included in the scan.

[robertlagrant]

Yes, when you lay it out like that I see more the depth of the complexity. It's even trickier when perhaps the package.json license field and the text in the LICEN[CS]E file don't match!

I'm currently using a tool called cdxgen which optionally calls out to a server to get the results, so for big projects it takes a while to respond. Might be worth a look.

[kzantow]

I believe this may have been fixed when #845 was implemented -- note: this requires running an npm install or similar, so package.json files are present in node_modules. Does this work for you now @robertlagrant ? (Also note: npm lockfiles v2 are now supported, which may include license information for dependencies.)

[spiffcs]

I just went back to validate this and it looks like after running the reproduce steps above we do not get the license from just package-lock.json:

The license IS picked up when running syft dir:. -o cyclonedx:

It looks like resolver.FilesByPath in the function addLicenses is not returning a location.

syft/syft/pkg/cataloger/javascript/cataloger.go

Lines 38 to 45 in 1b0cfe7

	func addLicenses(resolver source.FileResolver, location source.Location, p *pkg.Package) error {
	dir := path.Dir(location.RealPath)
	pkgPath := []string{dir, "node_modules"}
	pkgPath = append(pkgPath, strings.Split(p.Name, "/")...)
	pkgPath = append(pkgPath, "package.json")
	pkgFile := path.Join(pkgPath...)
	locations, err := resolver.FilesByPath(pkgFile)
	if err != nil {

The correct path for the package.json is requested:

But the resolver returns an empty slice of locations:

I can't remember if we opted for allowing syft to traverse over to node modules even if the user specified only the file directive as a prefix.

[spiffcs]

If we do want this to work where node_modules is accounted for as a post catalog task I think we would need to rebuild the tree for this post process function and then use RelativeFileByPath forming something like ../node_modules/{package}/package.json

[kzantow]

This is expected behavior when using the file: prefix... if you want directory traversals, you should use the dir: prefix.