Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Schema update - remove list from the possible list element #1681

Merged
merged 6 commits into from
Dec 24, 2022
Merged

Schema update - remove list from the possible list element #1681

merged 6 commits into from
Dec 24, 2022

Conversation

goneall
Copy link
Member

@goneall goneall commented Oct 15, 2022

Related to issue #1680

Signed-off-by: Gary O'Neall gary@sourceauditor.com

@goneall
Schema update - remove list from the possible list element
d015372 
Related to issue #1680

Signed-off-by: Gary O'Neall <gary@sourceauditor.com>
@goneall
Copy link
Member Author

goneall commented Oct 15, 2022

Unfortunately, this doesn't flag the following as invalid:

<list>
       <item>
            <bullet>1.</bullet>
          Licenses.
            <br/>Licensor hereby grants you the following rights, provided that you comply with all of the
                 restrictions set forth in this License and provided, further, that you distribute an
                 unmodified copy of this License with the Program:

        </item>
        <list>
            <item>
               <bullet>(a)</bullet>
            You may copy and distribute literal (i.e., verbatim) copies of the Program's source code as
               you receive it throughout the world, in any medium.
          </item>
            <item>
               <bullet>(b)</bullet>
            You may modify the Program, create works based on the Program and distribute copies of such
               throughout the world, in any medium.
          </item>
        </list>
</list>

We don't want to allow a list to be directly under another list.

@zvr - Can you review this and let me know what I did wrong?

@goneall goneall mentioned this pull request Oct 16, 2022
Open
@zvr
Copy link
Member

zvr commented Oct 19, 2022

Wow, it's been a long while since I've looked at this XSD -- not sure whether I remember everything.

First quick observation, @goneall : you've changed listType but not listStandardLicenseHeaderType right below; was this on purpose?

I'll have to trace a parse to see what type is used where; not sure when I'll manage it.

@goneall
Remove list from possible listStandardLicenseHeaderType
6cdaf7b 
Signed-off-by: Gary O'Neall <gary@sourceauditor.com>
@goneall
Copy link
Member Author

goneall commented Oct 21, 2022

First quick observation, @goneall : you've changed listType but not listStandardLicenseHeaderType right below; was this on purpose?

@zvr - I missed that - I just fixed it and it solved the problem. Thanks!

@goneall
Add proper cardinality to listStandardLicenseType item in schema
764f845 
Signed-off-by: Gary O'Neall <gary@sourceauditor.com>
@goneall
Copy link
Member Author

goneall commented Oct 21, 2022

There are 45 licenses XML's that will need to be fixed before this schema can be merged:

	Parsing error in XML file AAL.xml at line 47, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file AAL.xml
	Parsing error in XML file Aladdin.xml at line 60, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file Aladdin.xml
	Parsing error in XML file APSL-1.0.xml at line 31, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file APSL-1.0.xml
	Parsing error in XML file APSL-1.1.xml at line 33, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file APSL-1.1.xml
	Parsing error in XML file APSL-1.2.xml at line 23, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file APSL-1.2.xml
	Parsing error in XML file Artistic-2.0.xml at line 129, column 22:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file Artistic-2.0.xml
	Parsing error in XML file BSD-3-Clause-Clear.xml at line 23, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file BSD-3-Clause-Clear.xml
	Parsing error in XML file BSD-Protection.xml at line 52, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file BSD-Protection.xml
	Parsing error in XML file CC-BY-2.5-AU.xml at line 33, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file CC-BY-2.5-AU.xml
	Parsing error in XML file CC-BY-2.5.xml at line 32, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file CC-BY-2.5.xml
	Parsing error in XML file CC-BY-NC-SA-1.0.xml at line 183, column 22:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file CC-BY-NC-SA-1.0.xml
	Parsing error in XML file CC-BY-SA-2.0.xml at line 257, column 19:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file CC-BY-SA-2.0.xml
	Parsing error in XML file LGPL-2.0+.xml at line 276, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file LGPL-2.0+.xml
	Parsing error in XML file LGPL-2.0-only.xml at line 430, column 13:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file LGPL-2.0-only.xml
	Parsing error in XML file LGPL-2.0-or-later.xml at line 416, column 13:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file LGPL-2.0-or-later.xml
	Parsing error in XML file LGPL-2.0.xml at line 424, column 13:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file LGPL-2.0.xml
	Parsing error in XML file Motosoto.xml at line 95, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file Motosoto.xml
	Parsing error in XML file NBPL-1.0.xml at line 59, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file NBPL-1.0.xml
	Parsing error in XML file OLDAP-1.1.xml at line 60, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file OLDAP-1.1.xml
	Parsing error in XML file OLDAP-1.2.xml at line 61, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file OLDAP-1.2.xml
	Parsing error in XML file OLDAP-1.3.xml at line 61, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file OLDAP-1.3.xml
	Parsing error in XML file OLDAP-1.4.xml at line 60, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file OLDAP-1.4.xml
	Parsing error in XML file AAL.xml at line 47, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file AAL.xml
	Parsing error in XML file Aladdin.xml at line 60, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file Aladdin.xml
	Parsing error in XML file APSL-1.0.xml at line 31, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file APSL-1.0.xml
	Parsing error in XML file APSL-1.1.xml at line 33, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file APSL-1.1.xml
	Parsing error in XML file APSL-1.2.xml at line 23, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file APSL-1.2.xml
	Parsing error in XML file Artistic-2.0.xml at line 129, column 22:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file Artistic-2.0.xml
	Parsing error in XML file BSD-3-Clause-Clear.xml at line 23, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file BSD-3-Clause-Clear.xml
	Parsing error in XML file BSD-Protection.xml at line 52, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file BSD-Protection.xml
	Parsing error in XML file CC-BY-2.5-AU.xml at line 33, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file CC-BY-2.5-AU.xml
	Parsing error in XML file CC-BY-2.5.xml at line 32, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file CC-BY-2.5.xml
	Parsing error in XML file CC-BY-NC-SA-1.0.xml at line 183, column 22:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file CC-BY-NC-SA-1.0.xml
	Parsing error in XML file CC-BY-SA-2.0.xml at line 257, column 19:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file CC-BY-SA-2.0.xml
	Parsing error in XML file LGPL-2.0+.xml at line 276, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file LGPL-2.0+.xml
	Parsing error in XML file LGPL-2.0-only.xml at line 430, column 13:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file LGPL-2.0-only.xml
	Parsing error in XML file LGPL-2.0-or-later.xml at line 416, column 13:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file LGPL-2.0-or-later.xml
	Parsing error in XML file LGPL-2.0.xml at line 424, column 13:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file LGPL-2.0.xml
	Parsing error in XML file Motosoto.xml at line 95, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file Motosoto.xml
	Parsing error in XML file NBPL-1.0.xml at line 59, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file NBPL-1.0.xml
	Parsing error in XML file OLDAP-1.1.xml at line 60, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file OLDAP-1.1.xml
	Parsing error in XML file OLDAP-1.2.xml at line 61, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file OLDAP-1.2.xml
	Parsing error in XML file OLDAP-1.3.xml at line 61, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file OLDAP-1.3.xml
	Parsing error in XML file OLDAP-1.4.xml at line 60, column 15:cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.spdx.org/license":list}'. One of '{"http://www.spdx.org/license":item}' is expected., Skipping file OLDAP-1.4.xml

@goneall goneall mentioned this pull request Oct 21, 2022
Merged
@swinslow
Copy link
Member

@goneall Just to make sure I understand the situation here, is the following correct?

  • The current schema allows a <list> to be directly nested inside of another <list>
  • But, when translating this to HTML, this results in invalid HTML
  • To fix this, a nested <list> should only be indirectly included, by being wrapped inside an <item>
  • This PR would change the schema to impose this restriction
  • But, if it is merged, the licenses listed above would no longer validate -- so their XML needs to be fixed

Is that all correct? If so, I may be able to help with cleaning up those files.

@swinslow
Fix validation with switch to Python xml validator
b4acba5 
Signed-off-by: Steve Winslow <steve@swinslow.net>

This replaces the now-breaking node-based XML validation script
with a Python script that uses the xmlschema PyPI dependency.
@swinslow
Add more detailed reporting on validation errors
ea109eb 
Signed-off-by: Steve Winslow <steve@swinslow.net>
@zvr
Copy link
Member

zvr commented Dec 23, 2022

All what you wrote is completely correct, @swinslow.

@goneall
Copy link
Member Author

goneall commented Dec 23, 2022

@goneall Just to make sure I understand the situation here, is the following correct?

  • The current schema allows a <list> to be directly nested inside of another <list>
  • But, when translating this to HTML, this results in invalid HTML
  • To fix this, a nested <list> should only be indirectly included, by being wrapped inside an <item>
  • This PR would change the schema to impose this restriction
  • But, if it is merged, the licenses listed above would no longer validate -- so their XML needs to be fixed

Is that all correct? If so, I may be able to help with cleaning up those files.

yes - all the above is correct. And thanks for any help!

@swinslow
Add item tags to fix list tags
0bae651 
Signed-off-by: Steve Winslow <steve@swinslow.net>
@swinslow
Copy link
Member

Thanks @zvr and @goneall! 0bae651 is perhaps the most boring commit I've ever put together ;-) but should take care of this.

The tests are running now but they passed on my local system, so if this looks good to you, feel free to take the "draft" setting off of the PR whenever you're ready!

@goneall goneall marked this pull request as ready for review December 24, 2022 04:16
@goneall
Copy link
Member Author

goneall commented Dec 24, 2022

Thanks @swinslow - Looks like it's passing the tests - I'll go ahead and merge

@goneall goneall merged commit dc3371a into main Dec 24, 2022
@goneall goneall deleted the listschema branch December 24, 2022 04:21
@swinslow swinslow added XML schema change XML markup change potential change or addition to XML markup in license labels Dec 24, 2022
@swinslow swinslow added this to the 3.20 milestone Dec 24, 2022
@mlinksva mlinksva mentioned this pull request Dec 26, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@goneall goneall

@swinslow swinslow

Labels
XML markup change potential change or addition to XML markup in license XML schema change
Projects
None yet
Milestone
3.20
Development

Successfully merging this pull request may close these issues.
3 participants
@goneall @zvr @swinslow