Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dependency's MIT license not picked up when scanning package-lock.json #1113

Closed
robertlagrant opened this issue Jul 21, 2022 · 8 comments
Closed
Labels
bug Something isn't working enhancement New feature or request

Comments

@robertlagrant
Copy link

robertlagrant commented Jul 21, 2022

What happened
Outputting a CycloneDX SBOM does not pick up the sole dependency's licence in its output.

What you expected to happen:
Given I can see the dependency inside node_modules has a licence (MIT) in its package.json, and it also comes with a LICENSE file with the MIT licence text in it, I would expect the licence to appear in the cyclonedx output.

How to reproduce it (as minimally and precisely as possible):

% mkdir demo-syft-licence-bug
% cd demo-syft-licence-bug
demo-syft-licence-bug % npm init
<!snip!>
demo-syft-licence-bug % cat package.json
{
  "name": "demo-syft-licence-bug",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "Rob Grant <>",
  "license": "MIT"
}
demo-syft-licence-bug % npm install es-leftpad
added 1 package, and audited 2 packages in 2s
found 0 vulnerabilities
demo-syft-licence-bug %
demo-syft-licence-bug % demo-syft-licence-bug % syft file:package-lock.json -o cyclonedx
 ✔ Indexed package-lock.json
 ✔ Cataloged packages      [1 packages]
<?xml version="1.0" encoding="UTF-8"?>
<bom xmlns="http://cyclonedx.org/schema/bom/1.4" serialNumber="urn:uuid:fbd2213e-3623-413a-89e6-df0eb77043cf" version="1">
  <metadata>
    <timestamp>2022-07-21T09:47:20+01:00</timestamp>
    <tools>
      <tool>
        <vendor>anchore</vendor>
        <name>syft</name>
        <version>0.51.0</version>
      </tool>
    </tools>
    <component bom-ref="f11fc099fef741ab" type="file">
      <name>package-lock.json</name>
    </component>
  </metadata>
  <components>
    <component bom-ref="pkg:npm/es-leftpad@1.0.0?package-id=38a238232ed53007" type="library">
      <name>es-leftpad</name>
      <version>1.0.0</version>
      <cpe>cpe:2.3:a:es-leftpad:es-leftpad:1.0.0:*:*:*:*:*:*:*</cpe>
      <purl>pkg:npm/es-leftpad@1.0.0</purl>
      <properties>
        <property name="syft:package:foundBy">javascript-lock-cataloger</property>
        <property name="syft:package:language">javascript</property>
        <property name="syft:package:type">npm</property>
        <property name="syft:cpe23">cpe:2.3:a:es-leftpad:es_leftpad:1.0.0:*:*:*:*:*:*:*</property>
        <property name="syft:cpe23">cpe:2.3:a:es_leftpad:es-leftpad:1.0.0:*:*:*:*:*:*:*</property>
        <property name="syft:cpe23">cpe:2.3:a:es_leftpad:es_leftpad:1.0.0:*:*:*:*:*:*:*</property>
        <property name="syft:cpe23">cpe:2.3:a:es:es-leftpad:1.0.0:*:*:*:*:*:*:*</property>
        <property name="syft:cpe23">cpe:2.3:a:es:es_leftpad:1.0.0:*:*:*:*:*:*:*</property>
        <property name="syft:cpe23">cpe:2.3:a:*:es-leftpad:1.0.0:*:*:*:*:*:*:*</property>
        <property name="syft:cpe23">cpe:2.3:a:*:es_leftpad:1.0.0:*:*:*:*:*:*:*</property>
        <property name="syft:location:0:path">/Users/rob.grant/code/demo-syft-licence-bug/package-lock.json</property>
      </properties>
    </component>
  </components>
</bom>
demo-syft-licence-bug %

^ no licence listed in the cyclonedx output.

Anything else we need to know?:
It also doesn't appear if I output with SPDX format. If I try syft-json then it says "licences": [],.

Environment:

  • Output of syft version:
demo-syft-licence-bug % syft version
Application:        syft
Version:            0.51.0
JsonSchemaVersion:  3.3.1
BuildDate:          2022-07-11T17:12:56Z
GitCommit:          470b13045bbbf150f3c79d1487a01ae6acc5592d
GitDescription:     v0.51.0
Platform:           darwin/arm64
GoVersion:          go1.18.3
Compiler:           gc
  • OS (e.g: cat /etc/os-release or similar): OSX 12.4 Monterey.
@robertlagrant robertlagrant added the bug Something isn't working label Jul 21, 2022
@spiffcs
Copy link
Contributor

spiffcs commented Jul 21, 2022

Similar issue linked here! Thanks for filing the bug. We'll take a look and see what needs to be done to get licenses populated for dir scans.

#845

@robertlagrant
Copy link
Author

robertlagrant commented Jul 21, 2022

@spiffcs thank you! Just to be doubly sure, the above was meant to be me running against a lockfile, and not doing a dir scan! I ran this: syft file:package-lock.json -o cyclonedx.

@spiffcs
Copy link
Contributor

spiffcs commented Jul 21, 2022

Thanks for the clarification!

It's an interesting case because it poses the question "Given some manifest file, should syft traverse extra noninput paths to find additional metadata information?"

package-lock.json does not have license information in and of itself, so syft in its current state is doing the best it can with the information provided.

I'm going to add it to our community meeting today for discussion since we've knocked around crossing this boundary before but have stayed on the side of reporting only on the truth of the input and not something the tool inferred.

At any rate we definitely should solve for dir:. in this case and conclude licenses for our raw JSON output when node_modules are included in the scan.

@spiffcs spiffcs changed the title Dependency's MIT licence not picked up Dependency's MIT license not picked up when scanning package-lock.json Jul 21, 2022
@spiffcs spiffcs added the enhancement New feature or request label Jul 21, 2022
@robertlagrant
Copy link
Author

robertlagrant commented Jul 22, 2022

Yes, when you lay it out like that I see more the depth of the complexity. It's even trickier when perhaps the package.json license field and the text in the LICEN[CS]E file don't match!

I'm currently using a tool called cdxgen which optionally calls out to a server to get the results, so for big projects it takes a while to respond. Might be worth a look.

@kzantow
Copy link
Contributor

kzantow commented Aug 23, 2022

I believe this may have been fixed when #845 was implemented -- note: this requires running an npm install or similar, so package.json files are present in node_modules. Does this work for you now @robertlagrant ? (Also note: npm lockfiles v2 are now supported, which may include license information for dependencies.)

@spiffcs
Copy link
Contributor

spiffcs commented Aug 31, 2022

I just went back to validate this and it looks like after running the reproduce steps above we do not get the license from just package-lock.json:

Screen Shot 2022-08-31 at 10 27 53 AM

The license IS picked up when running syft dir:. -o cyclonedx:
Screen Shot 2022-08-31 at 10 28 31 AM

It looks like resolver.FilesByPath in the function addLicenses is not returning a location.

func addLicenses(resolver source.FileResolver, location source.Location, p *pkg.Package) error {
dir := path.Dir(location.RealPath)
pkgPath := []string{dir, "node_modules"}
pkgPath = append(pkgPath, strings.Split(p.Name, "/")...)
pkgPath = append(pkgPath, "package.json")
pkgFile := path.Join(pkgPath...)
locations, err := resolver.FilesByPath(pkgFile)
if err != nil {

The correct path for the package.json is requested:
Screen Shot 2022-08-31 at 11 00 17 AM

But the resolver returns an empty slice of locations:
Screen Shot 2022-08-31 at 11 02 14 AM

I can't remember if we opted for allowing syft to traverse over to node modules even if the user specified only the file directive as a prefix.

@spiffcs
Copy link
Contributor

spiffcs commented Aug 31, 2022

If we do want this to work where node_modules is accounted for as a post catalog task I think we would need to rebuild the tree for this post process function and then use RelativeFileByPath forming something like ../node_modules/{package}/package.json

@kzantow
Copy link
Contributor

kzantow commented Nov 17, 2022

This is expected behavior when using the file: prefix... if you want directory traversals, you should use the dir: prefix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working enhancement New feature or request
Projects
Archived in project
Development

No branches or pull requests

3 participants