Incorrect AGPL finding in Ruby file

[MartinPetkov]

Description

Please leave a brief discription of the bug or feature request:
Steps to reproduce:

$ git clone https://github.com/chef/chef
$ (cd chef && git checkout f47bd8c1e6d521805e133949693fc84df3544271)
$ scancode --verbose --license -n 12 --json-pp ./results.json --only-findings --info --strip-root --license-text ./chef/lib/chef/util/diff.rb
$ cat ./results.json

You see this finding, which should clearly not be there:

{
  "key": "agpl-1.0",
  "score": 92.94,
  "name": "Affero General Public License 1.0",
  "short_name": "AGPL 1.0",
  "category": "Copyleft",
  "is_exception": false,
  "owner": "Affero",
  "homepage_url": "http://www.affero.org/oagpl.html",
  "text_url": "http://www.affero.org/oagpl.html",
  "reference_url": "https://enterprise.dejacode.com/urn/urn:dje:license:agpl-1.0",
  "spdx_license_key": "AGPL-1.0-only",
  "spdx_url": "https://spdx.org/licenses/AGPL-1.0-only",
  "start_line": 21,
  "end_line": 41,
  "matched_rule": {
    "identifier": "mit_or_agpl-1.0_1.RULE",
    "license_expression": "mit OR agpl-1.0",
    "licenses": [
      "mit",
      "agpl-1.0"
    ],
    "is_license_text": false,
    "is_license_notice": true,
    "is_license_reference": false,
    "is_license_tag": false,
    "matcher": "3-seq",
    "rule_length": 170,
    "matched_length": 158,
    "match_coverage": 92.94,
    "rule_relevance": 100
  },
  "matched_text": "MIT\n#\n# Permission is hereby granted, free of charge, to any person\n# obtaining a copy of this software and associated documentation files\n# (the \"Software\"), to deal in the Software without restriction,\n# including without limitation the rights to use, copy, modify, merge,\n# publish, distribute, sublicense, and/or sell copies of the Software,\n# and to permit persons to whom the Software is furnished to do so,\n# subject the following conditions:\n#\n# The above copyright notice and this permission notice shall be\n# included in all copies or substantial portions of [this] Software.\n#\n# THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND,\n# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\n# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\n# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\n# BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\n# ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OF OR IN\n# CONNECTION WITH THE SOFTWARE OR THE USE [OF] OTHER DEALINGS IN THE\n# SOFTWARE."
}

What's strange is how high the score is, 92.94. I don't think almost any of this text overlaps with the AGPL 1.0 license text.

System configuration

For bug reports, it really helps us to know:

• What OS are you running on? (Windows/MacOS/Linux)
  Linux
• What version of scancode-toolkit was used to generate the scan file?
  ScanCode version 3.0.2.post1114.8b6916601
• What installation method was used to install/run scancode? (pip/source download/other)
  pip

[pombredanne]

Thank you for this. I went ahead and ran a scan on all of chef for sanity and found a few more issues and files this too chef/chef#9016

[pombredanne]

A fix is available in the #1779 PR. Thanks again for the report!

[pombredanne]

@MartinPetkov you wrote:

What's strange is how high the score is, 92.94. I don't think almost any of this text overlaps with the AGPL 1.0 license text.

Actually the matched rule file is mit_or_agpl-1.0_1.RULE that has an AGPL reference, but that part is not matched. The fix has been in this case to ensure that mit_or_agpl-1.0_1.RULE must be matched almost entirely to be considered by requesting a minimum_coverage of 95% of its words (aka. tokens). Each of the MIT and AGPL parts are also matched otherwise as solo rules, so this is not an issue anymore with this minimum_coverage boost.

Note that with "matched_length": 158 words/tokens out of a total "rule_length": 170 words/tokens we have a ratio of 158/170 that yields a "match_coverage": 92.94%, which is also the score as therelevance of that rule is 100%, e.g. there is no penalty for the match being to a shorter or less relevant rule.

[pombredanne]

merged in develop now. And this is in the next candidate RC in a few.
Thank you ++ for the report!