Fail to detect gpl-3.0 and gfdl-1.3

[chinyeungli]

Description

license detection fails to detect the gpl-3.0 and gfdl-1.3 from the following license notice:

This is the Debian prepackaged version of the GNU diffutils package.
GNU `diff' was written by Mike Haertel, David Hayes, Richard Stallman,
Len Tower, and Paul Eggert.  Wayne Davison designed and implemented
the unified output format. GNU `diff3' was written by Randy Smith.
GNU `sdiff' was written by Thomas Lord.  GNU `cmp' was written by
Torbjorn Granlund and David MacKenzie.

The source for this release was obtained from

http://ftp.gnu.org/gnu/diffutils/diffutils-3.3.tar.xz

Program copyright and license:
=============================

Copyright (C) 1988-1996, 1998, 2001-2002, 2004, 2006-2007, 2009-2013
Free Software Foundation, Inc.

  This program is free software: you can redistribute it and/or modify
  it under the terms of the GNU General Public License as published by
  the Free Software Foundation, either version 3 of the License, or
  (at your option) any later version.

  This program is distributed in the hope that it will be useful,
  but WITHOUT ANY WARRANTY; without even the implied warranty of
  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  GNU General Public License for more details.

On Debian systems, the complete text of the GNU General Public License
may be found in `/usr/share/common-licenses/GPL'.


Manual copyright and license:
============================

Copyright (C) 1992-1994, 1998, 2001-2002, 2004, 2006, 2009-2013 Free
Software Foundation, Inc.

  Permission is granted to copy, distribute and/or modify this document
  under the terms of the GNU Free Documentation License, Version 1.3 or
  any later version published by the Free Software Foundation; with no
  Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.

On Debian systems, the complete text of the GNU Free Documentation
License may be found in `/usr/share/common-licenses/GFDL'.

Ref: https://changelogs.ubuntu.com/changelogs/pool/main/d/diffutils/diffutils_3.3-3/copyright

This is the current returned license detection:

      "licenses": [
        {
          "key": "gpl-2.0-plus",
          "score": 89.36,
          "name": "GNU General Public License 2.0 or later",
          "short_name": "GPL 2.0 or later",
          "category": "Copyleft",
          "is_exception": false,
          "owner": "Free Software Foundation (FSF)",
          "homepage_url": "http://www.gnu.org/licenses/old-licenses/gpl-2.0-standalone.html",
          "text_url": "http://www.gnu.org/licenses/old-licenses/gpl-2.0-standalone.html",
          "reference_url": "https://enterprise.dejacode.com/urn/urn:dje:license:gpl-2.0-plus",
          "spdx_license_key": "GPL-2.0-or-later",
          "spdx_url": "https://spdx.org/licenses/GPL-2.0-or-later",
          "start_line": 18,
          "end_line": 28,
          "matched_rule": {
            "identifier": "gpl-2.0-plus_505.RULE",
            "license_expression": "gpl-2.0-plus",
            "licenses": [
              "gpl-2.0-plus"
            ],
            "is_license_text": false,
            "is_license_notice": true,
            "is_license_reference": false,
            "is_license_tag": false,
            "matcher": "3-seq",
            "rule_length": 94,
            "matched_length": 84,
            "match_coverage": 89.36,
            "rule_relevance": 100.0
          }
        },
        {
          "key": "gpl-1.0-plus",
          "score": 99.0,
          "name": "GNU General Public License 1.0 or later",
          "short_name": "GPL 1.0 or later",
          "category": "Copyleft",
          "is_exception": false,
          "owner": "Free Software Foundation (FSF)",
          "homepage_url": "http://www.gnu.org/licenses/old-licenses/gpl-1.0-standalone.html",
          "text_url": "http://www.gnu.org/licenses/old-licenses/gpl-1.0-standalone.html",
          "reference_url": "https://enterprise.dejacode.com/urn/urn:dje:license:gpl-1.0-plus",
          "spdx_license_key": "GPL-1.0-or-later",
          "spdx_url": "https://spdx.org/licenses/GPL-1.0-or-later",
          "start_line": 29,
          "end_line": 29,
          "matched_rule": {
            "identifier": "gpl-1.0-plus_154.RULE",
            "license_expression": "gpl-1.0-plus",
            "licenses": [
              "gpl-1.0-plus"
            ],
            "is_license_text": false,
            "is_license_notice": false,
            "is_license_reference": false,
            "is_license_tag": true,
            "matcher": "2-aho",
            "rule_length": 2,
            "matched_length": 2,
            "match_coverage": 100.0,
            "rule_relevance": 99.0
          }
        },
        {
          "key": "unknown",
          "score": 22.0,
          "name": "Unknown license detected but not recognized",
          "short_name": "unknown",
          "category": "Unstated License",
          "is_exception": false,
          "owner": "Unspecified",
          "homepage_url": null,
          "text_url": "",
          "reference_url": "https://enterprise.dejacode.com/urn/urn:dje:license:unknown",
          "spdx_license_key": null,
          "spdx_url": "",
          "start_line": 38,
          "end_line": 38,
          "matched_rule": {
            "identifier": "lead-in_unknown_16.RULE",
            "license_expression": "unknown",
            "licenses": [
              "unknown"
            ],
            "is_license_text": false,
            "is_license_notice": false,
            "is_license_reference": true,
            "is_license_tag": false,
            "matcher": "2-aho",
            "rule_length": 4,
            "matched_length": 4,
            "match_coverage": 100.0,
            "rule_relevance": 22
          }
        },
        {
          "key": "gpl-1.0",
          "score": 54.55,
          "name": "GNU General Public License 1.0",
          "short_name": "GPL 1.0",
          "category": "Copyleft",
          "is_exception": false,
          "owner": "Free Software Foundation (FSF)",
          "homepage_url": "http://www.gnu.org/licenses/gpl-1.0.html",
          "text_url": "http://www.gnu.org/licenses/gpl-1.0.txt",
          "reference_url": "https://enterprise.dejacode.com/urn/urn:dje:license:gpl-1.0",
          "spdx_license_key": "GPL-1.0-only",
          "spdx_url": "https://spdx.org/licenses/GPL-1.0-only",
          "start_line": 38,
          "end_line": 40,
          "matched_rule": {
            "identifier": "gpl-1.0_or_gpl-2.0.RULE",
            "license_expression": "gpl-1.0 OR gpl-2.0",
            "licenses": [
              "gpl-1.0",
              "gpl-2.0"
            ],
            "is_license_text": false,
            "is_license_notice": true,
            "is_license_reference": false,
            "is_license_tag": false,
            "matcher": "3-seq",
            "rule_length": 33,
            "matched_length": 18,
            "match_coverage": 54.55,
            "rule_relevance": 100
          }
        },
        {
          "key": "gpl-2.0",
          "score": 54.55,
          "name": "GNU General Public License 2.0",
          "short_name": "GPL 2.0",
          "category": "Copyleft",
          "is_exception": false,
          "owner": "Free Software Foundation (FSF)",
          "homepage_url": "http://www.gnu.org/licenses/gpl-2.0.html",
          "text_url": "http://www.gnu.org/licenses/gpl-2.0.txt",
          "reference_url": "https://enterprise.dejacode.com/urn/urn:dje:license:gpl-2.0",
          "spdx_license_key": "GPL-2.0-only",
          "spdx_url": "https://spdx.org/licenses/GPL-2.0-only",
          "start_line": 38,
          "end_line": 40,
          "matched_rule": {
            "identifier": "gpl-1.0_or_gpl-2.0.RULE",
            "license_expression": "gpl-1.0 OR gpl-2.0",
            "licenses": [
              "gpl-1.0",
              "gpl-2.0"
            ],
            "is_license_text": false,
            "is_license_notice": true,
            "is_license_reference": false,
            "is_license_tag": false,
            "matcher": "3-seq",
            "rule_length": 33,
            "matched_length": 18,
            "match_coverage": 54.55,
            "rule_relevance": 100
          }
        },
        {
          "key": "gfdl-1.1-plus",
          "score": 100.0,
          "name": "GNU Free Documentation License v1.1 or later",
          "short_name": "GFDL 1.1 or later",
          "category": "Copyleft Limited",
          "is_exception": false,
          "owner": "Free Software Foundation (FSF)",
          "homepage_url": "http://www.gnu.org/licenses/old-licenses/fdl-1.1.txt",
          "text_url": "http://www.gnu.org/licenses/fdl-1.1.txt",
          "reference_url": "https://enterprise.dejacode.com/urn/urn:dje:license:gfdl-1.1-plus",
          "spdx_license_key": "GFDL-1.1-or-later",
          "spdx_url": "https://spdx.org/licenses/GFDL-1.1-or-later",
          "start_line": 39,
          "end_line": 39,
          "matched_rule": {
            "identifier": "gfdl-1.1-plus_10.RULE",
            "license_expression": "gfdl-1.1-plus",
            "licenses": [
              "gfdl-1.1-plus"
            ],
            "is_license_text": false,
            "is_license_notice": false,
            "is_license_reference": true,
            "is_license_tag": false,
            "matcher": "2-aho",
            "rule_length": 4,
            "matched_length": 4,
            "match_coverage": 100.0,
            "rule_relevance": 100.0
          }
        },
        {
          "key": "gfdl-1.2-plus",
          "score": 50.0,
          "name": "GNU Free Documentation License v1.2 or later",
          "short_name": "GFDL 1.2 or later",
          "category": "Copyleft Limited",
          "is_exception": false,
          "owner": "Free Software Foundation (FSF)",
          "homepage_url": "http://www.gnu.org/licenses/old-licenses/fdl-1.2.txt",
          "text_url": "http://www.gnu.org/licenses/fdl-1.2.txt",
          "reference_url": "https://enterprise.dejacode.com/urn/urn:dje:license:gfdl-1.2-plus",
          "spdx_license_key": "GFDL-1.2-or-later",
          "spdx_url": "https://spdx.org/licenses/GFDL-1.2-or-later",
          "start_line": 39,
          "end_line": 41,
          "matched_rule": {
            "identifier": "gfdl-1.2-plus_4.RULE",
            "license_expression": "gfdl-1.2-plus",
            "licenses": [
              "gfdl-1.2-plus"
            ],
            "is_license_text": false,
            "is_license_notice": true,
            "is_license_reference": false,
            "is_license_tag": false,
            "matcher": "3-seq",
            "rule_length": 46,
            "matched_length": 23,
            "match_coverage": 50.0,
            "rule_relevance": 100
          }
        },
        {
          "key": "gfdl-1.1-plus",
          "score": 100.0,
          "name": "GNU Free Documentation License v1.1 or later",
          "short_name": "GFDL 1.1 or later",
          "category": "Copyleft Limited",
          "is_exception": false,
          "owner": "Free Software Foundation (FSF)",
          "homepage_url": "http://www.gnu.org/licenses/old-licenses/fdl-1.1.txt",
          "text_url": "http://www.gnu.org/licenses/fdl-1.1.txt",
          "reference_url": "https://enterprise.dejacode.com/urn/urn:dje:license:gfdl-1.1-plus",
          "spdx_license_key": "GFDL-1.1-or-later",
          "spdx_url": "https://spdx.org/licenses/GFDL-1.1-or-later",
          "start_line": 43,
          "end_line": 44,
          "matched_rule": {
            "identifier": "gfdl-1.1-plus_10.RULE",
            "license_expression": "gfdl-1.1-plus",
            "licenses": [
              "gfdl-1.1-plus"
            ],
            "is_license_text": false,
            "is_license_notice": false,
            "is_license_reference": true,
            "is_license_tag": false,
            "matcher": "2-aho",
            "rule_length": 4,
            "matched_length": 4,
            "match_coverage": 100.0,
            "rule_relevance": 100.0
          }
        }
      ],
      "license_expressions": [
        "gpl-2.0-plus",
        "gpl-1.0-plus",
        "unknown",
        "gpl-1.0 OR gpl-2.0",
        "gfdl-1.1-plus",
        "gfdl-1.2-plus",
        "gfdl-1.1-plus"
      ],

[pombredanne]

This has been fixed and merged in develop. Thank you!