Incorrect BSD-3 finding for Ruby license

[jkschulz]

Description

These lines are being identified as BSD-3 license, though it says "ruby license"

To reproduce:

$ git clone https://github.com/rubyzip/rubyzip.git
$ (cd rubyzip && git checkout 9d891f7353e66052283562d3e252fe380bb4b199)
$ ./scancode-toolkit-3.1.1/scancode --verbose --license -n 12 --json-pp ./results.json --only-findings --info --strip-root --license-text rubyzip/samples/example.rb

License finding:

"licenses": [
        {
          "key": "bsd-new",
          "score": 89.47,
          "name": "BSD-3-Clause",
          "short_name": "BSD-3-Clause",
          "category": "Permissive",
          "is_exception": false,
          "owner": "Regents of the University of California",
          "homepage_url": "http://www.opensource.org/licenses/BSD-3-Clause",
          "text_url": "http://www.opensource.org/licenses/BSD-3-Clause",
          "reference_url": "https://enterprise.dejacode.com/urn/urn:dje:license:bsd-new",
          "spdx_license_key": "BSD-3-Clause",
          "spdx_url": "https://spdx.org/licenses/BSD-3-Clause",
          "start_line": 80,
          "end_line": 81,
          "matched_rule": {
            "identifier": "bsd-new_zlib2.RULE",
            "license_expression": "bsd-new",
            "licenses": [
              "bsd-new"
            ],
            "is_license_text": false,
            "is_license_notice": true,
            "is_license_reference": false,
            "is_license_tag": false,
            "matcher": "3-seq",
            "rule_length": 19,
            "matched_length": 17,
            "match_coverage": 89.47,
            "rule_relevance": 100
          },
          "matched_text": "# rubyzip is free software; you can redistribute it and/or\n# modify it under the terms of the ruby license."
        }
      ],

System configuration

For bug reports, it really helps us to know:

• What OS are you running on? MacOS
• What version of scancode-toolkit was used to generate the scan file? 3.1.1
• What installation method was used to install/run scancode? source download

[pombredanne]

@jkschulz thanks for the report!
This is fixed in the https://github.com/nexB/scancode-toolkit/compare/10-27-license-updates branch now

[jkschulz]

Thanks so much for the quick fix!

[pombredanne]

@jkschulz re:

Thanks so much for the quick fix!

Sure thing! we like our bugs quickly squashed when possible! 🐛
And this is the kind of reports that helps make things much better for everyone so this is really appreciated. Please send more of these our way!

Note that because of the way scancode works, we eventually perform a pairwise diff.
Since we did not had anything exactly matching the rubyzip notice wording:

... is free software; you can redistribute it and/or modify it under the terms of the ruby license.

This was matched instead to this rule https://github.com/nexB/scancode-toolkit/blob/3a007baa7b04c668d2cf2fb70945ad86b156728b/src/licensedcode/data/rules/bsd-new_zlib2.RULE:

... is free software; you can redistribute it and/or modify it under the terms of the [BSD] license.

where one word was not matched in the diff ([BSD]) and the match score was not perfect either.

FWIW, this is a wording seen originally in GNU license notices and rarely in Ruby or BSD notices.

Adding a new rule and data was the fix:

• https://github.com/nexB/scancode-toolkit/blob/3a007baa7b04c668d2cf2fb70945ad86b156728b/src/licensedcode/data/rules/bsd-simplified_or_ruby_2.RULE

• https://github.com/nexB/scancode-toolkit/blob/3a007baa7b04c668d2cf2fb70945ad86b156728b/src/licensedcode/data/rules/bsd-simplified_or_ruby_2.yml

Note the quirk that today's Ruby license is really a choice of BSD-2-Clause (bsd-simplified) OR Ruby, when why we report this when the rule is matched and not a bare ruby.

Finally I also pushed a few extra improved rules based on a scan review of rubyzip in the same commit.