Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow license rules to require the presence of certain defining keywords #2773

Merged
merged 28 commits into from
Dec 25, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
5450469
Add feature to set key phrases in rules
mrombout Nov 4, 2021
983fd13
Fix false positive detection of cc-by-4.0
mrombout Nov 19, 2021
5e4e256
Fix false positive detection of gpl-2.0
mrombout Nov 19, 2021
0617520
Fix false positive of bsd-simplified AND gpl-2.0
mrombout Nov 19, 2021
763cbe2
Fix false positive of apache-2.0 AND cc-by-sa-4.0
mrombout Nov 19, 2021
23936f4
Fix false positive detection of ruby
mrombout Nov 19, 2021
00d210f
Fix false positive detection of apache-2.0
mrombout Nov 19, 2021
1b0a266
Fix false positive of apache-2.0 OR epl-2.0
mrombout Nov 19, 2021
99ab33d
Fix false positive of apache-2.0 OR gpl-2.0-plus
mrombout Nov 19, 2021
ee16cac
Fix false positive detection of mpl-1.1
mrombout Nov 8, 2021
0e678f0
Fix false positive detection of cddl-1.0
mrombout Nov 8, 2021
05d479e
Fix false positive detection of mit
mrombout Nov 8, 2021
3af9de4
Fix false positive detection of imagemagick
mrombout Nov 8, 2021
65384ab
Add unit tests to key_phrase_tokenizer
mrombout Nov 22, 2021
627ace3
Replace key_phrase_spans default to use factory
mrombout Nov 22, 2021
00b8483
Explain key phrase syntax in new license rule guide
mrombout Nov 25, 2021
a6a913e
Do not use defaultdict for Query unknowns
pombredanne Nov 25, 2021
4510855
Throw InvalidRule when key phrase is not closed
mrombout Nov 29, 2021
7c5ff8d
Fix false positive where bsd-3-clause is detected as gpl-2.0-plus_and…
dd-jy Nov 18, 2021
0859a44
Add key phrases to CHANGELOG
mrombout Nov 30, 2021
8765b25
Add name to AUTHORS.rst
mrombout Nov 30, 2021
459c1ca
Update filter_key_phrase_spans to also check qspan
mrombout Dec 1, 2021
06da853
Add docstring to BasicRule::key_phrase_spans
mrombout Dec 2, 2021
e603be2
Add missing rule for cc-by-nc-sa-4.0
mrombout Dec 2, 2021
7648550
Remove incompatible key phrase syntax from rules
mrombout Dec 2, 2021
b9902e3
Add @dd-jy to AUTHORS.rst
mrombout Dec 2, 2021
a9a85a0
Move up key phrase filter to improve results
mrombout Dec 6, 2021
83840df
Fix false positive of apache-2.0
mrombout Dec 15, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions AUTHORS.rst
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ The following organizations or individuals have contributed to ScanCode:
- Jelmer Vernooij @jelmer
- Jillian Daguil @jdaguil
- Jiri Popelka @jpopelka
- Jiyeong Seok @dd-jy
- Johannes Najjar @joshovi
- Jose Nazario @paralax
- Lemo Shi @lemoshi
Expand All @@ -43,6 +44,7 @@ The following organizations or individuals have contributed to ScanCode:
- Maximilian Huber @maxhbr
- Michael Herzog @mjherzog
- Michael Rupprecht @michaelrup
- Mike Rombout @mrombout
pombredanne marked this conversation as resolved.
Show resolved Hide resolved
- Mrinal Paliwal @mnpw
- nexB Inc. @nexB
- Nirmal Sarswat @vivonk
Expand Down
3 changes: 3 additions & 0 deletions CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,9 @@ License detection:
- XXXX new license detection rules have been added, and
- XXXX existing license rules have been updated.

- Key phrases can now be defined in RULEs by surrounding one or more words with
pombredanne marked this conversation as resolved.
Show resolved Hide resolved
`{{` and `}}`. When defined a RULE will only match when the key phrases match
exactly.

Package detection:
~~~~~~~~~~~~~~~~~~
Expand Down
4 changes: 4 additions & 0 deletions docs/source/how-to-guides/add_new_license_detection_rule.rst
Original file line number Diff line number Diff line change
Expand Up @@ -62,3 +62,7 @@ More (advanced) rules options:
- Each rules needs have one flag such as is_license_notice. See the
``src/licensedcode/models.py`` directory for a list of all possible values and
other options.

- you can specify key phrases by surrounding one or more words between the `{{`
and `}}` tags. Key phrases are words that **must** be matched/present in order
for a RULE to be considered a match.
1 change: 1 addition & 0 deletions src/licensedcode/data/rules/apache-2.0_1026.RULE
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
This SDK is distributed under the Apache License, Version 2.0, see LICENSE for more information.
3 changes: 3 additions & 0 deletions src/licensedcode/data/rules/apache-2.0_1026.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
license_expression: apache-2.0
is_license_notice: yes
relevance: 100
2 changes: 1 addition & 1 deletion src/licensedcode/data/rules/apache-2.0_571.RULE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<licenses>
<license>
<name>Apache License 2.0</name>
<name>{{Apache License 2.0}}</name>
<url>http://repository.jboss.org/licenses/apache-2.0.txt</url>
<distribution>repo</distribution>
</license>
Expand Down
Original file line number Diff line number Diff line change
@@ -1 +1 @@
Code is released under the [Apache 2.0 license](LICENSE). This `README.md` file and the [`CONTRIBUTING.md`](CONTRIBUTING.md) file are licensed under the Creative Commons Attribution 4.0 International License under the terms and conditions set forth in the file [`LICENSE.docs`](LICENSE.docs). You may obtain a duplicate copy of the same license, titled CC BY-SA 4.0, at http://creativecommons.org/licenses/by-sa/4.0/.
Code is released under the [{{Apache 2.0}} license](LICENSE). This `README.md` file and the [`CONTRIBUTING.md`](CONTRIBUTING.md) file are licensed under the {{Creative Commons Attribution 4.0 International License}} under the terms and conditions set forth in the file [`LICENSE.docs`](LICENSE.docs). You may obtain a duplicate copy of the same license, titled CC BY-SA 4.0, at http://creativecommons.org/licenses/by-sa/4.0/.
4 changes: 2 additions & 2 deletions src/licensedcode/data/rules/apache-2.0_or_epl-2.0_1.RULE
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
<licenses>
<license>
<name>The Apache Software License, Version 2.0</name>
<name>{{The Apache Software License}}, Version 2.0</name>
<url>http://www.apache.org/licenses/LICENSE-2.0.txt</url>
<distribution>repo</distribution>
</license>
<license>
<name>Eclipse Public License - v 2.0</name>
<name>{{Eclipse Public License}} - v 2.0</name>
<url>http://www.eclipse.org/legal/epl-v20.html</url>
<distribution>repo</distribution>
</license>
Expand Down
4 changes: 2 additions & 2 deletions src/licensedcode/data/rules/apache-2.0_or_epl-2.0_2.RULE
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
<licenses>
<license>
<name>The Apache Software License, Version 2.0</name>
<name>The {{Apache Software License, Version 2.0}}</name>
<url>https://www.apache.org/licenses/LICENSE-2.0.txt</url>
<distribution>repo</distribution>
</license>
<license>
<name>Eclipse Public License - v 2.0</name>
<name>{{Eclipse Public License - v 2.0}}</name>
<url>http://www.eclipse.org/legal/epl-v20.html</url>
<distribution>repo</distribution>
</license>
Expand Down
Original file line number Diff line number Diff line change
@@ -1 +1 @@
<licenses><license><name>The Apache Software License, Version 2.0</name><url>http://www.apache.org/licenses/LICENSE-2.0.txt</url><distribution>repo</distribution></license><license><name>GNU Lesser General Public License (LGPL), Version 2.1</name><url>http://www.fsf.org/licensing/licenses/lgpl.txt</url><distribution>repo</distribution></license></licenses>
<licenses><license><name>The {{Apache Software License}}, Version 2.0</name><url>http://www.apache.org/licenses/LICENSE-2.0.txt</url><distribution>repo</distribution></license><license><name>{{GNU Lesser General Public License}} (LGPL), Version 2.1</name><url>http://www.fsf.org/licensing/licenses/lgpl.txt</url><distribution>repo</distribution></license></licenses>
Original file line number Diff line number Diff line change
@@ -1 +1 @@
<licenses><license><name>The Apache Software License, Version 2.0</name><url>https://www.apache.org/licenses/LICENSE-2.0.txt</url><distribution>repo</distribution></license><license><name>GNU Lesser General Public License (LGPL), Version 2.1</name><url>http://www.fsf.org/licensing/licenses/lgpl.txt</url><distribution>repo</distribution></license></licenses>
<licenses><license><name>The {{Apache Software License}}, Version 2.0</name><url>https://www.apache.org/licenses/LICENSE-2.0.txt</url><distribution>repo</distribution></license><license><name>{{GNU Lesser General Public License}} (LGPL), Version 2.1</name><url>http://www.fsf.org/licensing/licenses/lgpl.txt</url><distribution>repo</distribution></license></licenses>
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
<!-- Licensing: ASL or GPL -->
<licenses><license><name>The Apache Software License, Version 2.0</name><url>https://www.apache.org/licenses/LICENSE-2.0.txt</url><distribution>repo</distribution></license><license><name>GNU Lesser General Public License (LGPL), Version 2.1</name><url>http://www.fsf.org/licensing/licenses/lgpl.txt</url><distribution>repo</distribution></license></licenses>
<licenses><license><name>The {{Apache Software License}}, Version 2.0</name><url>https://www.apache.org/licenses/LICENSE-2.0.txt</url><distribution>repo</distribution></license><license><name>{{GNU Lesser General Public License}} (LGPL), Version 2.1</name><url>http://www.fsf.org/licensing/licenses/lgpl.txt</url><distribution>repo</distribution></license></licenses>
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
<licenses>
<license>
<name>Apache License, Version 2.0</name>
<name>{{Apache License, Version 2.0}}</name>
<url>http://www.apache.org/licenses/LICENSE-2.0</url>
<distribution>repo</distribution>
</license>
<license>
<name>GNU General Public License (GPL) version 2, or any later version</name>
<name>{{GNU General Public License (GPL) version 2}}, or any later version</name>
<url>https://www.gnu.org/licenses/</url>
<distribution>repo</distribution>
</license>
<license>
<name>GPLv2 with Classpath exception</name>
<name>{{GPLv2 with Classpath exception}}</name>
<url>https://www.gnu.org/software/classpath/license.html</url>
<distribution>repo</distribution>
</license>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
<licenses>
<license>
<name>Apache License, Version 2.0</name>
<name>{{Apache License}}, Version 2.0</name>
<url>https://www.apache.org/licenses/LICENSE-2.0</url>
<distribution>repo</distribution>
</license>
<license>
<name>GNU General Public License (GPL) version 2, or any later version</name>
<name>{{GNU General Public License}} (GPL) version 2, or any later version</name>
<url>https://www.gnu.org/licenses/</url>
<distribution>repo</distribution>
</license>
<license>
<name>GPLv2 with Classpath exception</name>
<name>{{GPLv2}} with {{Classpath exception}}</name>
<url>https://www.gnu.org/software/classpath/license.html</url>
<distribution>repo</distribution>
</license>
Expand Down
2 changes: 1 addition & 1 deletion src/licensedcode/data/rules/apache-2.0_url_12.RULE
Original file line number Diff line number Diff line change
@@ -1 +1 @@
<licenses> <license> <name>Apache License, Version 2.0</name> <url>https://www.apache.org/licenses/LICENSE-2.0</url> <distribution>repo</distribution> </license> </licenses>
<licenses> <license> <name>{{Apache License}}, Version 2.0</name> <url>https://www.apache.org/licenses/LICENSE-2.0</url> <distribution>repo</distribution> </license> </licenses>
2 changes: 1 addition & 1 deletion src/licensedcode/data/rules/apache-2.0_url_13.RULE
Original file line number Diff line number Diff line change
@@ -1 +1 @@
<licenses> <license> <name>Apache License, Version 2.0</name> <url>https://apache.org/licenses/LICENSE-2.0</url> <distribution>repo</distribution> </license> </licenses>
<licenses> <license> <name>{{Apache License}}, Version 2.0</name> <url>https://apache.org/licenses/LICENSE-2.0</url> <distribution>repo</distribution> </license> </licenses>
2 changes: 1 addition & 1 deletion src/licensedcode/data/rules/apache-2.0_url_14.RULE
Original file line number Diff line number Diff line change
@@ -1 +1 @@
<licenses> <license> <name>Apache License, Version 2.0</name> <url>http://www.apache.org/licenses/LICENSE-2.0</url> <distribution>repo</distribution> </license> </licenses>
<licenses> <license> <name>{{Apache License, Version 2.0}}</name> <url>http://www.apache.org/licenses/LICENSE-2.0</url> <distribution>repo</distribution> </license> </licenses>
2 changes: 1 addition & 1 deletion src/licensedcode/data/rules/apache-2.0_url_15.RULE
Original file line number Diff line number Diff line change
@@ -1 +1 @@
<licenses> <license> <name>Apache License, Version 2.0</name> <url>http://apache.org/licenses/LICENSE-2.0</url> <distribution>repo</distribution> </license> </licenses>
<licenses> <license> <name>{{Apache License}}, Version 2.0</name> <url>http://apache.org/licenses/LICENSE-2.0</url> <distribution>repo</distribution> </license> </licenses>
2 changes: 1 addition & 1 deletion src/licensedcode/data/rules/bsd-new_531.RULE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<licenses>
<license>
<name>Berkeley Software Distribution (BSD) License</name>
<name>{{Berkeley Software Distribution}} (BSD) License</name>
<url>http://www.opensource.org/licenses/bsd-license.html</url>
<distribution>repo</distribution>
</license>
Expand Down
2 changes: 1 addition & 1 deletion src/licensedcode/data/rules/bsd-new_58.RULE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<licenses>
<license>
<name>Berkeley Software Distribution (BSD) License</name>
<name>{{Berkeley Software Distribution (BSD) License}}</name>
<url>http://www.opensource.org/licenses/bsd-license.html</url>
</license>
</licenses>
4 changes: 2 additions & 2 deletions src/licensedcode/data/rules/bsd-simplified_and_gpl-2.0_1.RULE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
This repository uses 2 different licenses :
- all files in the `lib` directory use a BSD 2-Clause license
- all other files use a GPLv2 license, unless explicitly stated otherwise
- all files in the `lib` directory use a {{BSD 2-Clause}} license
- all other files use a {{GPLv2}} license, unless explicitly stated otherwise

Relevant license is reminded at the top of each source file,
and with presence of COPYING or LICENSE file in associated directories.
2 changes: 1 addition & 1 deletion src/licensedcode/data/rules/cc-by-4.0_16.RULE
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
under the Creative Commons Attribution 4.0 International License (the "License");
under the {{Creative Commons Attribution 4.0 International License}} (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

Expand Down
7 changes: 7 additions & 0 deletions src/licensedcode/data/rules/cc-by-nc-sa-4.0_24.RULE
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
This work is licensed under a {{Creative Commons Attribution-NonCommercial-ShareAlike 4.0}}
International License (the "License"). You may not use this file except in compliance with the
License. A copy of the License is located at http://creativecommons.org/licenses/by-nc-sa/4.0/.

This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
either express or implied. See the License for the specific language governing permissions and
limitations under the License.
5 changes: 5 additions & 0 deletions src/licensedcode/data/rules/cc-by-nc-sa-4.0_24.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
license_expression: cc-by-nc-sa-4.0
is_license_notice: yes
relevance: 100
ignorable_urls:
- http://creativecommons.org/licenses/by-nc-sa/4.0
1 change: 1 addition & 0 deletions src/licensedcode/data/rules/cc-by-nc-sa-4.0_25.RULE
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{{Creative Commons Attribution-NonCommercial-ShareAlike 4.0}} International License
3 changes: 3 additions & 0 deletions src/licensedcode/data/rules/cc-by-nc-sa-4.0_25.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
license_expression: cc-by-nc-sa-4.0
is_license_reference: yes
relevance: 100
2 changes: 1 addition & 1 deletion src/licensedcode/data/rules/cddl-1.0_27.RULE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<licenses>
<license>
<name>CDDL 1</name>
<name>{{CDDL 1}}</name>
<url>http://opensource.org/licenses/CDDL-1.0</url>
<distribution>repo</distribution>
</license>
Expand Down
4 changes: 2 additions & 2 deletions src/licensedcode/data/rules/epl-1.0_or_lgpl-2.1-plus_4.RULE
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
<licenses>
<license>
<name>Eclipse Public License 1.0</name>
<name>{{Eclipse Public License 1.0}}</name>
<url>http://www.eclipse.org/org/documents/epl-v10.php</url>
<distribution>Manual</distribution>
</license>
<license>
<name>GNU Lesser GPL 2.1</name>
<name>{{GNU Lesser GPL 2.1}}</name>
<url>http://www.gnu.org/licenses/old-licenses/lgpl-2.1.html</url>
<distribution>manual</distribution>
</license>
Expand Down
4 changes: 2 additions & 2 deletions src/licensedcode/data/rules/epl-1.0_or_lgpl-2.1-plus_5.RULE
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
<licenses>
<license>
<name>Eclipse Public License 1.0</name>
<name>{{Eclipse Public License 1.0}}</name>
<url>http://www.eclipse.org/org/documents/epl-v10.php</url>
<distribution>Manual</distribution>
</license>
<license>
<name>GNU Lesser GPL 2.1</name>
<name>{{GNU Lesser GPL 2.1}}</name>
<url>https://www.gnu.org/licenses/old-licenses/lgpl-2.1.html</url>
<distribution>manual</distribution>
</license>
Expand Down
2 changes: 1 addition & 1 deletion src/licensedcode/data/rules/false-positive_ansible_2.RULE
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,4 @@
# - GPLv3
# - Apache
# - CC-BY
license: {{ license }}
license: license
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
Distributed under the terms of the GNU General Public License (version 2
# or later) with exception for distributing the bootloader.
Distributed under the terms of the {{GNU General Public License}} (version 2
# or later) with exception for distributing the {{bootloader}}.
#
# The full license is in the file COPYING.txt, distributed with this software.
# The full license is in the file COPYING.txt, distributed with this software.
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,4 @@ is_license_notice: yes
relevance: 100
referenced_filenames:
- COPYING.txt
minimum_coverage: 70
2 changes: 1 addition & 1 deletion src/licensedcode/data/rules/gpl-2.0_1044.RULE
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
Licensed under the GNU General Public License v2 (the "License");
Licensed under the {{GNU General Public License v2}} (the "License");
you may not use this file except in compliance with the License. The terms
of the License are located in the COPYING file of this distribution.
4 changes: 2 additions & 2 deletions src/licensedcode/data/rules/gpl-2.0_644.RULE
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#{{IS_RIGHT
#IS_RIGHT
# This program is distributed under GPL Version 2.0 in the hope that
# it will be useful, but WITHOUT ANY WARRANTY.
#}}IS_RIGHT
#IS_RIGHT
4 changes: 2 additions & 2 deletions src/licensedcode/data/rules/gpl-3.0_111.RULE
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@

{{IS_RIGHT
IS_RIGHT
This program is distributed under GPL Version 3.0 in the hope that
it will be useful, but WITHOUT ANY WARRANTY.
}}IS_RIGHT
IS_RIGHT
2 changes: 1 addition & 1 deletion src/licensedcode/data/rules/imagemagick_19.RULE
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ You may add Your own copyright statement to Your modifications and may provide a

How to Apply the License to your Work

To apply the ImageMagick License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information (don't include the brackets). The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives.
To apply the {{ImageMagick}} License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information (don't include the brackets). The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives.

Copyright [yyyy] [name of copyright owner]

Expand Down
2 changes: 1 addition & 1 deletion src/licensedcode/data/rules/lgpl-3.0-plus_216.RULE
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
GNU Lesser General Public License, Version 3 {{{ */
GNU Lesser General Public License, Version 3 */

is free software: you can redistribute it and/or modify it under
the terms of the GNU Lesser General Public License as published by the
Expand Down
2 changes: 1 addition & 1 deletion src/licensedcode/data/rules/lgpl-3.0-plus_27.RULE
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
GNU Lesser General Public License, Version 3 {{{ */
GNU Lesser General Public License, Version 3 */

is free software: you can redistribute it and/or modify it under
the terms of the GNU Lesser General Public License as published by the
Expand Down
2 changes: 1 addition & 1 deletion src/licensedcode/data/rules/mit_923.RULE
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
License
Distributed under the MIT License. See LICENSE for more information.
Distributed under the {{MIT}} License. See LICENSE for more information.
2 changes: 1 addition & 1 deletion src/licensedcode/data/rules/mpl-1.0_14.RULE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
MPL:

"The contents of this file are subject to the Mozilla Public License Version 1.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.mozilla.org/MPL/
"The contents of this file are subject to the {{Mozilla Public License}} Version 1.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.mozilla.org/MPL/

Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License.

Expand Down
2 changes: 1 addition & 1 deletion src/licensedcode/data/rules/mpl-1.0_8.RULE
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
* The contents of this file are subject to the Mozilla Public License
* The contents of this file are subject to the {{Mozilla Public License}}
* Version 1.0 (the "License"); you may not use this file except in
* compliance with the License. You may obtain a copy of the License at
* http://www.mozilla.org/MPL/
Expand Down
2 changes: 1 addition & 1 deletion src/licensedcode/data/rules/mpl-1.1_36.RULE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
The MPL v1.1:

The contents of this file are subject to the Mozilla Public License
The contents of this file are subject to the {{Mozilla Public License}}
Version 1.1 (the "License"); you may not use this file except in
compliance with the License. You may obtain a copy of the License at
http://www..com/mpl.html
Expand Down
2 changes: 1 addition & 1 deletion src/licensedcode/data/rules/mpl-1.1_40.RULE
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
The contents of this file are subject to the Mozilla Public License Version 1.1
The contents of this file are subject to the {{Mozilla Public License}} Version 1.1
(the License); you may not use this file except in compliance with the License.
You may obtain a copy of the License at http://www.mozilla.org/MPL/
Software distributed under the License is distributed on an AS IS basis,
Expand Down
2 changes: 1 addition & 1 deletion src/licensedcode/data/rules/mpl-1.1_6.RULE
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
%% The contents of this file are subject to the Mozilla Public License
%% The contents of this file are subject to the {{Mozilla Public License}}
%% Version 1.1 (the "License"); you may not use this file except in
%% compliance with the License. You may obtain a copy of the License at
%% http://www.mozilla.org/MPL/
Expand Down
4 changes: 2 additions & 2 deletions src/licensedcode/data/rules/npl-1.1_1.RULE
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
* The contents of this file are subject to the Netscape Public
* License Version 1.1 (the "License"); you may not use this file
* The contents of this file are subject to the {{Netscape Public
* License}} Version 1.1 (the "License"); you may not use this file
* except in compliance with the License. You may obtain a copy of
* the License at http://www.mozilla.org/NPL/
*
Expand Down
2 changes: 1 addition & 1 deletion src/licensedcode/data/rules/proprietary_133.RULE
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
* The contents of this file are subject to the KnowledgeTree Public
* The contents of this file are subject to the {{KnowledgeTree}} Public
* License Version 1.1 ("License"); You may not use this file except in
* compliance with the License. You may obtain a copy of the License at
* http://www.ktdms.com/KPL
Expand Down
2 changes: 1 addition & 1 deletion src/licensedcode/data/rules/ruby_5.RULE
Original file line number Diff line number Diff line change
@@ -1 +1 @@
This library is distributed under the terms of the Ruby license.
This library is distributed under the terms of the {{Ruby}} license.
3 changes: 3 additions & 0 deletions src/licensedcode/index.py
Original file line number Diff line number Diff line change
Expand Up @@ -330,6 +330,9 @@ def _add_rules(self, rules, _legalese=common_license_words, _spdx_tokens=frozens
# "weak" rules can only be matched with an automaton.
is_weak = True

# note down any key phrase spans that must be present for the rule to pass through refinement
rule.key_phrase_spans = list(rule.key_phrases())

for rts in rule.tokens():
rtid = dictionary_get(rts)
if rtid is None:
Expand Down
Loading