Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(detector/library): update JAR-like files' Name/Version in library list #1874

Merged
merged 3 commits into from
Mar 19, 2024

Conversation

shino
Copy link
Collaborator

@shino shino commented Mar 18, 2024

What did you implement:

Before this pull request, JAR-like files' names and versions are partially correct and partially wrong, beause correction by Trivy's Java DB is performed in vuls report phase.
For example, one entry in "libraries" list of scan result looks like below

     {
        "Name": ":spring-web",  ## <== NOT correct, does not have organization (before colon)
        "Version": "3.2.18.RELEASE",
        "PURL": "",
        "FilePath": "home/shino/g/vuls/integration/data/lockfile/juddiv3-war-3.3.5.war/WEB-INF/lib/spring-web-3.2.18.RELEASE.jar",
        "Digest": "sha1:bc0bdade0a7a52b8fae88e1febc8479383a2acad"
      },

and libraryFixedIns of vuls report result

      "libraryFixedIns": [
        {
          "key": "java",
          "name": "org.springframework:spring-web",  ## <== Correct name, thanks to Java DB!
          "fixedIn": "6.0.0",
          "path": "home/shino/g/vuls/integration/data/lockfile/juddiv3-war-3.3.5.war"
        }
      ]

It's not easy to find what is the current version of a library in libraryFixedIns.
Furthermore, there can be more broken names like:

  • "Name": "Apache Software Foundation:org.apache.xml.resolver.Catalog",
  • "Name": "France Telecom R&D:ASM",

This overwrite of Name's and Version's does not conserve the input of vuls report as is.
But these entries are not trusted from the first. Only trusted entry is Digest's.
(If there is no Digest field, Name and version are trusted and not overwritten)

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

  • CVE list of vuls report does not change
  • After vuls report library list is updated by the help of Java DB
% git checkout master
Already on 'master'
Your branch is up to date with 'origin/master'.

% CGO_ENABLED=0 go run cmd/vuls/main.go scan -config ./integration/int-config.toml war
[Mar 18 20:24:51]  INFO [localhost] vuls-`make build` or `make install` will show the version-
[Mar 18 20:24:51]  INFO [localhost] Start scanning
[Mar 18 20:24:51]  INFO [localhost] config: ./integration/int-config.toml
[Mar 18 20:24:51]  INFO [localhost] Validating config...
[Mar 18 20:24:51]  INFO [localhost] Detecting Server/Container OS...
[Mar 18 20:24:51]  INFO [localhost] Detecting OS of servers...
[Mar 18 20:24:51]  INFO [localhost] (1/1) Detected: war: pseudo
[Mar 18 20:24:51]  INFO [localhost] Detecting OS of containers...
[Mar 18 20:24:51]  INFO [localhost] Checking Scan Modes...
[Mar 18 20:24:51]  INFO [localhost] Detecting Platforms...
[Mar 18 20:24:51]  INFO [localhost] (1/1) war is running on other
[Mar 18 20:24:51]  INFO [war] Scanning listen port...
[Mar 18 20:24:51]  INFO [war] Using Port Scanner: Vuls built-in Scanner
[Mar 18 20:24:51]  INFO [war] Scanning Language-specific Packages...


Scan Summary
================
war     pseudo  0 installed, 0 updatable        79 libs





To view the detail, vuls tui is useful.
To send a report, run vuls report -h.

% CGO_ENABLED=0 go run cmd/vuls/main.go report -config ./integration/int-config.toml -refresh-cve -quiet G CVE- > old.txt

## Move to this branch
% git checkout shino/correct-jar-entries
Switched to branch 'shino/correct-jar-entries'
Your branch is up to date with 'origin/shino/correct-jar-entries'.
% CGO_ENABLED=0 go run cmd/vuls/main.go report -config ./integration/int-config.toml -refresh-cve -quiet G CVE- > new.txt


## Compare two CVE lists
% sha1sum old.txt new.txt
0acf2ce20991cd3a9d4d4bcaeb9c5acd616d2a78  old.txt
0acf2ce20991cd3a9d4d4bcaeb9c5acd616d2a78  new.txt

Name's before report

% find results -type f -printf '%T@ %p\n' | grep '.json' | sort -n | tail -1 | head -1 | cut -d' ' -f2 | xargs -I{} bash -c "cat "{}"" | jq -C '.libraries' | grep Name
        "Name": ":antlr",
        "Name": ":aopalliance",
        "Name": ":cglib-nodep",
        "Name": ":json",
        "Name": ":spring-aop",
        "Name": ":spring-beans",
        "Name": ":spring-context",
        "Name": ":spring-core",
        "Name": ":spring-expression",
        "Name": ":spring-web",
        "Name": ":stax-api-1.0",
        "Name": "Apache Software Foundation:org.apache.xml.resolver.Catalog",
        "Name": "France Telecom R&D:ASM",
        "Name": "IBM:WSDL4J",
        "Name": "JCP:StAX 1.0 API",
        "Name": "MetaStuff Ltd.:org.dom4j",
        "Name": "Sun Microsystems Inc.:javax.xml.transform",
        "Name": "com.fasterxml:classmate",
        "Name": "com.sun:JAXB Reference Implementation",
        "Name": "commons-codec:commons-codec",
        "Name": "commons-collections:commons-collections",
        "Name": "commons-configuration:commons-configuration",
        "Name": "commons-lang:commons-lang",
        "Name": "commons-logging:commons-logging",
        "Name": "commons-pool:commons-pool",
        "Name": "javax.ws.rs:javax.ws.rs-api",
        "Name": "log4j:log4j",
        "Name": "net.sourceforge.serp:serp",
        "Name": "org.apache.commons:commons-lang3",
        "Name": "org.apache.cxf:cxf-api",
        "Name": "org.apache.cxf:cxf-rt-bindings-soap",
        "Name": "org.apache.cxf:cxf-rt-bindings-xml",
        "Name": "org.apache.cxf:cxf-rt-core",
        "Name": "org.apache.cxf:cxf-rt-databinding-jaxb",
        "Name": "org.apache.cxf:cxf-rt-frontend-jaxrs",
        "Name": "org.apache.cxf:cxf-rt-frontend-jaxws",
        "Name": "org.apache.cxf:cxf-rt-frontend-simple",
        "Name": "org.apache.cxf:cxf-rt-rs-extension-providers",
        "Name": "org.apache.cxf:cxf-rt-transports-http",
        "Name": "org.apache.cxf:cxf-rt-ws-addr",
        "Name": "org.apache.cxf:cxf-rt-ws-policy",
        "Name": "org.apache.geronimo.javamail:geronimo-javamail_1.4_mail",
        "Name": "org.apache.geronimo.javamail:geronimo-javamail_1.4_provider",
        "Name": "org.apache.geronimo.specs:geronimo-javamail_1.4_spec",
        "Name": "org.apache.geronimo.specs:geronimo-javamail_1.4_spec",
        "Name": "org.apache.geronimo.specs:geronimo-jms_1.1_spec",
        "Name": "org.apache.geronimo.specs:geronimo-jpa_2.0_spec",
        "Name": "org.apache.geronimo.specs:geronimo-jta_1.1_spec",
        "Name": "org.apache.httpcomponents:httpclient",
        "Name": "org.apache.httpcomponents:httpcore",
        "Name": "org.apache.juddi:juddi-client",
        "Name": "org.apache.juddi:juddi-core-openjpa",
        "Name": "org.apache.juddi:juddi-rest-cxf",
        "Name": "org.apache.juddi:juddiv3-war",
        "Name": "org.apache.juddi:uddi-ws",
        "Name": "org.apache.neethi:neethi",
        "Name": "org.apache.openjpa:openjpa",
        "Name": "org.apache.openjpa:openjpa-jdbc",
        "Name": "org.apache.openjpa:openjpa-jest",
        "Name": "org.apache.openjpa:openjpa-kernel",
        "Name": "org.apache.openjpa:openjpa-lib",
        "Name": "org.apache.openjpa:openjpa-persistence",
        "Name": "org.apache.openjpa:openjpa-persistence-jdbc",
        "Name": "org.apache.openjpa:openjpa-slice",
        "Name": "org.apache.openjpa:openjpa-xmlstore",
        "Name": "org.apache.ws.xmlschema:xmlschema-core",
        "Name": "org.apache.xbean:xbean-asm-util",
        "Name": "org.apache.xbean:xbean-asm4-shaded",
        "Name": "org.apache:Jakarta Commons Logging",
        "Name": "org.codehaus.jettison:jettison",
        "Name": "org.codehaus.woodstox:stax2-api",
        "Name": "org.hibernate.javax.persistence.hibernate-jpa-2:Java Persistence API",
        "Name": "org.hibernate:hibernate-commons-annotations",
        "Name": "org.hibernate:hibernate-core",
        "Name": "org.hibernate:hibernate-entitymanager",
        "Name": "org.javassist:javassist",
        "Name": "org.jboss.logging:jboss-logging",
        "Name": "org.jboss:jandex",
        "Name": "woodstox-core-asl:Woodstox XML-processor",

Name's after report

        "Name": ":antlr",
        "Name": ":stax-api-1.0",
        "Name": "aopalliance:aopalliance",
        "Name": "asm:asm",
        "Name": "cglib:cglib-nodep",
        "Name": "com.fasterxml:classmate",
        "Name": "com.sun.xml.bind:jaxb-impl",
        "Name": "commons-codec:commons-codec",
        "Name": "commons-collections:commons-collections",
        "Name": "commons-configuration:commons-configuration",
        "Name": "commons-lang:commons-lang",
        "Name": "commons-logging:commons-logging",
        "Name": "commons-logging:commons-logging-api",
        "Name": "commons-pool:commons-pool",
        "Name": "dom4j:dom4j",
        "Name": "javax.ws.rs:javax.ws.rs-api",
        "Name": "log4j:log4j",
        "Name": "net.sourceforge.serp:serp",
        "Name": "org.apache.commons:commons-lang3",
        "Name": "org.apache.cxf:cxf-api",
        "Name": "org.apache.cxf:cxf-rt-bindings-soap",
        "Name": "org.apache.cxf:cxf-rt-bindings-xml",
        "Name": "org.apache.cxf:cxf-rt-core",
        "Name": "org.apache.cxf:cxf-rt-databinding-jaxb",
        "Name": "org.apache.cxf:cxf-rt-frontend-jaxrs",
        "Name": "org.apache.cxf:cxf-rt-frontend-jaxws",
        "Name": "org.apache.cxf:cxf-rt-frontend-simple",
        "Name": "org.apache.cxf:cxf-rt-rs-extension-providers",
        "Name": "org.apache.cxf:cxf-rt-transports-http",
        "Name": "org.apache.cxf:cxf-rt-ws-addr",
        "Name": "org.apache.cxf:cxf-rt-ws-policy",
        "Name": "org.apache.geronimo.javamail:geronimo-javamail_1.4_mail",
        "Name": "org.apache.geronimo.javamail:geronimo-javamail_1.4_provider",
        "Name": "org.apache.geronimo.specs:geronimo-javamail_1.4_spec",
        "Name": "org.apache.geronimo.specs:geronimo-javamail_1.4_spec",
        "Name": "org.apache.geronimo.specs:geronimo-jms_1.1_spec",
        "Name": "org.apache.geronimo.specs:geronimo-jpa_2.0_spec",
        "Name": "org.apache.geronimo.specs:geronimo-jta_1.1_spec",
        "Name": "org.apache.httpcomponents:httpclient",
        "Name": "org.apache.httpcomponents:httpcore",
        "Name": "org.apache.juddi:juddi-client",
        "Name": "org.apache.juddi:juddi-core-openjpa",
        "Name": "org.apache.juddi:juddi-rest-cxf",
        "Name": "org.apache.juddi:juddiv3-war",
        "Name": "org.apache.juddi:uddi-ws",
        "Name": "org.apache.neethi:neethi",
        "Name": "org.apache.openjpa:openjpa",
        "Name": "org.apache.openjpa:openjpa-jdbc",
        "Name": "org.apache.openjpa:openjpa-jest",
        "Name": "org.apache.openjpa:openjpa-kernel",
        "Name": "org.apache.openjpa:openjpa-lib",
        "Name": "org.apache.openjpa:openjpa-persistence",
        "Name": "org.apache.openjpa:openjpa-persistence-jdbc",
        "Name": "org.apache.openjpa:openjpa-slice",
        "Name": "org.apache.openjpa:openjpa-xmlstore",
        "Name": "org.apache.ws.xmlschema:xmlschema-core",
        "Name": "org.apache.xbean:xbean-asm-util",
        "Name": "org.apache.xbean:xbean-asm4-shaded",
        "Name": "org.codehaus.jettison:jettison",
        "Name": "org.codehaus.woodstox:stax2-api",
        "Name": "org.codehaus.woodstox:woodstox-core-asl",
        "Name": "org.hibernate.javax.persistence:hibernate-jpa-2.1-api",
        "Name": "org.hibernate:hibernate-commons-annotations",
        "Name": "org.hibernate:hibernate-core",
        "Name": "org.hibernate:hibernate-entitymanager",
        "Name": "org.javassist:javassist",
        "Name": "org.jboss.logging:jboss-logging",
        "Name": "org.jboss:jandex",
        "Name": "org.json:json",
        "Name": "org.springframework:spring-aop",
        "Name": "org.springframework:spring-beans",
        "Name": "org.springframework:spring-context",
        "Name": "org.springframework:spring-core",
        "Name": "org.springframework:spring-expression",
        "Name": "org.springframework:spring-web",
        "Name": "stax:stax-api",
        "Name": "wsdl4j:wsdl4j",
        "Name": "xml-apis:xml-apis",
        "Name": "xml-resolver:xml-resolver",

Checklist:

You don't have to satisfy all of the following.

  • Write tests
  • Write documentation
  • Check that there aren't other open pull requests for the same issue/feature
  • Format your source code by make fmt
  • Pass the test by make test
  • Provide verification config / commands
  • Enable "Allow edits from maintainers" for this PR
  • Update the messages below

Is this ready for review?: Yes

@shino shino force-pushed the shino/correct-jar-entries branch from 3028518 to 9d72e8d Compare March 18, 2024 11:59
@shino shino changed the title Update JAR-like files in library list Update JAR-like files' Name/Version in library list Mar 18, 2024
@shino shino marked this pull request as ready for review March 18, 2024 12:09
@shino shino requested a review from MaineK00n March 18, 2024 12:10
detector/library.go Outdated Show resolved Hide resolved
Co-authored-by: MaineK00n <mainek00n.1229@gmail.com>
detector/library.go Outdated Show resolved Hide resolved
@MaineK00n MaineK00n changed the title Update JAR-like files' Name/Version in library list feat(detector/library): update JAR-like files' Name/Version in library list Mar 19, 2024
@shino shino merged commit 99cf9db into master Mar 19, 2024
7 checks passed
@shino shino deleted the shino/correct-jar-entries branch March 19, 2024 06:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants