CC0-1.0 accommodation for text used in ~30k repos on github.com #489

mlinksva · 2017-11-25T04:10:54Z

Evidence: https://github.com/search?utf8=%E2%9C%93&q=filename%3ALICENSE+%22For+more+information%2C+please+see+%3Chttp%3A%2F%2Fcreativecommons.org%2Fpublicdomain%2Fzero%2F1.0%2F%3E%22&type=Code

The changes stem from the CC0-1.0 text at https://choosealicense.com/licenses/cc0-1.0/

The text at choosealicense.com should be replaced with the text from CC or SPDX, but it might be worth matching texts with the trivial changes so widely deployed.

(This edit is independent of/should not conflict with #487, which addresses a different line present in the text from CC, but not in the text in this repo or at choosealicense.com.)

Evidence: https://github.com/search?utf8=%E2%9C%93&q=filename%3ALICENSE+%22For+more+information%2C+please+see+%3Chttp%3A%2F%2Fcreativecommons.org%2Fpublicdomain%2Fzero%2F1.0%2F%3E%22&type=Code The changes stem from the CC0-1.0 text at https://choosealicense.com/licenses/cc0-1.0/ This should be replaced with the text from CC or SPDX, but it _might_ be worth matching texts with the trivial changes included.

wking · 2017-11-26T04:56:46Z

src/CC0-1.0.xml

@@ -6,7 +6,7 @@
         <crossRef>http://creativecommons.org/publicdomain/zero/1.0/legalcode</crossRef>
      </crossRefs>
      <titleText>
-         <p>Creative Commons CC0 1.0 Universal</p>
+         <p><optional>Creative Commons </optional>CC0 1.0 Universal</p>


+1 for making “Creative Commons” optional. Following their legalcode.txt, we may want:

<titleText> <optional>Creative Commons<optional> Legal Code</optional><optional> CC0 1.0 Universal </titleText>

Since at least 2009-03-14, their HTML legal code has used <h1> and <h2> for those titles, but we don't have sized headers in our XML format (yet?).

https://spdx.org/licenses/CC0-1.0.html has "Creative Commons CC0 1.0 Universal" all on one line though.

wking · 2017-11-26T05:01:05Z

src/CC0-1.0.xml

@@ -143,6 +143,6 @@
            </list>
        </item>
      </list>
-
+      <optional><p>For more information, please see &lt;http://creativecommons.org/publicdomain/zero/1.0/&gt;</p></optional>


Since we don't know where this line comes from (I can't find it anywhere on creativecommons.org, anyway), I'd rather use an <alt> to keep it out of the canonical text. Something like:

<alt match="For more information, please see <http://creativecommons.org/publicdomain/zero/1.0/>" name="upstreamLink"></alt>

It was introduced at github/choosealicense.com#126 with no explanation of that line but I guess it was an attempt to provide something equivalent to the "Back to Commons Deed" link present at the time. Anyway, you're right, I'll use your suggestion!

Hmm, it seems  still warranted . One way to get it would be to wrap in <optional>, but this probably isn't ideal:

<optional><alt match="For more information, please see <http://creativecommons.org/publicdomain/zero/1.0/>" name="upstreamLink"></alt></optional>

It looks to me as if <alt match> hasn't been used with mixed content so far. Can it?

Yes - text between <alt ... and </alt> will be considered the "original text" and rendered in the web page and any generated text. You can include formatting for that text.

Hmm, it seems  still warranted .

Why? The canonical form has no content there; certainly not an empty paragraph. And the match content need only match the whitespace-normalized license content (per the matching guidelines). So I don't think we need  at all.

You're right, it didn't occur to me that valid HTML output is irrelevant for text not included in the output. 😄

… it didn't occur to me that valid HTML output is irrelevant for text not included in the output.

I think there's a larger issue here about “how do we inform folks reading HTML renderings of the set of allowed alternatives?”. The current previews just use:

<copyright holder>

and similar. I think we should at least add a title attribute with the regexp, although I'm not very excited about exposing the regexps directly to end-users, many of whom may not be familiar with regexps.

But that's all orthogonal to this PR ;).

wking · 2017-11-27T18:39:47Z

src/CC0-1.0.xml

@@ -143,6 +143,6 @@
            </list>
        </item>
      </list>
-
+      <alt><p>For more information, please see &lt;http://creativecommons.org/publicdomain/zero/1.0/&gt;</p></alt>


Does <alt> work without a name and match? And while I think we want to match files which include this line, I don't think we want to include it in our canonical text. So I still prefer the phrasing I'd suggested here.

OK, I didn't fully understand -- it's including text in a match that keeps it out of canonical text, not wrapping it in <alt> -- I'll go with your suggestion, thanks for explaining.

jlovejoy · 2017-12-19T18:56:34Z

I think this should use the optional tag as the text in question could be there or missing and still be a match. This is how we've defined/used the optional tag otherwise, so we should be consistent

jlovejoy · 2017-12-19T18:57:19Z

this also will allow the text to be displayed on the website as optional, like this: https://spdx.org/licenses/Apache-1.1.html (in blue text)

mlinksva · 2017-12-19T23:02:29Z

@jlovejoy I started with optional and switched to alt at @wking's suggestion in order to keep the text out of the text displayed on the website, see #489 (comment) ... the text should be matched as there are lots of legitimate copies of the license with it, but there's no reason for a licensor to include the nonstandard text. Alt hides it while enabling matching, IIUC. Happy to be wrong!

jlovejoy · 2017-12-21T19:03:32Z

as this ties into a bigger conversation as to how/when we use these tags, we are going to boot this for further discussion as it could have a knock-on effect with other licenses/markup

wking · 2017-12-26T19:50:22Z

On Thu, Dec 21, 2017 at 07:03:33PM +0000, Jilayne Lovejoy wrote: as this ties into a bigger conversation as to how/when we use these tags…

I think we've *been* using alt vs. optional based on which form we consider canonical (e.g. see #429, which very narrowly bears on this issue). There is work on display for empty-canonical-alts in spdx/tools#120, but I think that's a tooling issue and not a “what should we put in the template XML?” issue.

… we are going to boot this for further discussion as it could have a knock-on effect with other licenses/markup

What sort of knock-on effect do you foresee? I don't see a reason to punt on this at the moment.

wking · 2018-01-27T21:45:21Z

src/CC0-1.0.xml

@@ -147,6 +147,7 @@
            </list>
        </item>
      </list>
+      <alt match="For more information, please see <http://creativecommons.org/publicdomain/zero/1.0/>" name="upstreamLink"></alt>


You may need <, etc. escapes in the match value to get valid XML.

mlinksva · 2018-01-27T22:51:09Z

The remaining failure https://travis-ci.org/spdx/license-list-XML/jobs/334218553#L471-L472 looks to be related to #487 rather than this change:

Test for license CC0-1.0 failed: Test for license ID CC0-1.0 failed due to difference found Variable text rule upstreamLink did not match the compare text at end of text while processing rule var: upstreamLink. Last optional text was not found due to the optional difference:
Normal text of license does not match starting at line 1 column 21 "1" when comparing to template text " Legal Code"

wking · 2018-01-27T23:39:01Z

src/CC0-1.0.xml

@@ -147,6 +147,7 @@
            </list>
        </item>
      </list>
+      <alt match="For more information, please see &lt;http://creativecommons.org/publicdomain/zero/1.0/>" name="upstreamLink"></alt>


I think you need > -> > in the match value too.

It's not necessary, and the validate-schema test passes. Happy to escape it though if that's stylistically preferred. The remaining failing test isn't contingent on the > being escaped or not (tried locally).

mlinksva · 2018-01-28T00:49:35Z

That's actually two distinct errors:

Test for license CC0-1.0 failed: Test for license ID CC0-1.0 failed due to difference found Variable text rule upstreamLink did not match the compare text at end of text while processing rule var: upstreamLink

This can be "fixed" by replacing the <alt match... with <optional.... But that then fails to distinguish between canonical and non-canonical text.

Last optional text was not found due to the optional difference:
Normal text of license does not match starting at line 1 column 21 "1" when comparing to template text " Creative Commons Legal Code"

This can be "fixed" with something like

      <titleText>
         <optional><p>Creative Commons Legal Code</p></optional>
         <p><optional>Creative Commons </optional>CC0 1.0 Universal</p>
      </titleText>

but that allows for more variations than seen in the wild.

I suspect both of these have to do with the matcher implementation being slightly less flexible than the schema would imply, but I haven't tried to confirm this by looking at the code. I'll play with it some more, just leaving this as a note to self, but hints welcome. 😄

wking · 2018-01-28T01:46:44Z

That's actually two distinct errors...

I think the first is a knock-on from the second, because master passes.

mlinksva · 2018-01-28T02:30:28Z

I don't understand the knock-on causal relation, but you're right. Fixed the second by wrapping the alt with empty default in optional. Not certain that's the right thing, but the tests pass now...

goneall · 2018-01-28T05:37:32Z

Just catching up on the conversation. It looks like we ended with a reasonable solution IMHO.

BTW - the error reporting for parsing errors is not very satisfactory for the tool that compare the code to the canonical text. I've been working on trying to improve it, but it can get quite complicated when there are nested alt's and optionals and it is not clear where the error occurs. Suggestions and tool improvements are welcome.

wking · 2018-01-28T22:14:05Z

On Sat, Jan 27, 2018 at 09:37:32PM -0800, goneall wrote: It looks like we ended with a reasonable solution IMHO.

If the matching implementation has trouble with empty alt elements, perhaps it could wrap them in optional tags as a pre-processing step so we didn't have to remember to do that here? On the other hand, test failures will remind us, so not a big deal if we leave things as they stand.

bradleeedmondson · 2018-04-05T17:26:43Z

Putting a ping on @goneall @mlinksva @wking -- did we reach a decision on this? Current result of this PR seems to substantively add only one line, and not address the title question.

wking

I think the need for the wrapping <optional> is unfortunate, but I'm fine with this PR landing as it stands.

wking · 2018-04-05T17:43:09Z

Current result of this PR seems … not address the title question.

Doesn't it? I thought the issue was that choosealicense had been including the line that this PR is adding. It's not part of the upstream text, and with the empty <alt> here the new line is not included in our canonical wording. But the new empty <alt> does allow for matching choosealicense's previous wording, which is what the title is claiming.

bradleeedmondson · 2018-04-05T17:48:42Z

Doesn't it? I thought the issue was that choosealicense had been including the line that this PR is adding. It's not part of the upstream text, and with the empty here the new line is not included in our canonical wording. But the new empty does allow for matching choosealicense's previous wording, which is what the title is claiming.

I didn't grok this well enough to resolve that this was the outstanding issue. Basically if the previous commenters you, @mlinksva, and @goneall are happy (that that's the question, and it's adequately resolved), then I'm happy.

mlinksva · 2018-04-05T19:50:42Z

The title question was addressed in #487, which has been merged. This PR is also good to go.

bradleeedmondson · 2018-04-05T20:05:13Z

Based on today's discussion here and on @goneall review on Jan 28, I understand this is ready to merge. Glad we'll be able to match on a commonly used variant; thanks all.

wking reviewed Nov 26, 2017

View reviewed changes

wking mentioned this pull request Nov 26, 2017

Add optional "Creative Commons Legal Code" to title #487

Merged

mlinksva added 3 commits November 26, 2017 17:05

optional->alt for non-canonical "For more info..."

72644ef

keep title changes separate, in spdx#487

cf4506c

really backout title changes

4c25251

wking reviewed Nov 27, 2017

View reviewed changes

Finally go with @wking's initial suggestion, now that I understand it :)

e82e34d

wking mentioned this pull request Nov 27, 2017

Show regexp in HTML output for replacable text spdx/tools#120

Closed

jlovejoy added legal issue technical issue labels Dec 21, 2017

jlovejoy added this to the Later Release milestone Dec 21, 2017

This was referenced Dec 29, 2017

GPL-*-only: Drop standard license headers? #579

Closed

format-xml: Add a script for pretty-printing our XML source #432

Closed

goneall mentioned this pull request Jan 13, 2018

schema/ListedLicense: Punt license text into <text> #452

Merged

Merge branch 'master' into patch-4

32396ac

wking reviewed Jan 27, 2018

View reviewed changes

escape < in attribute

fa51ba4

wking reviewed Jan 27, 2018

View reviewed changes

wrap alt with empty default in optional?

f18f377

mlinksva mentioned this pull request Jan 31, 2018

Detect CC0-1.0 both as published by CC/SPDX and choosealicense.com 2013-2019 licensee/licensee#253

Merged

jlovejoy modified the milestones: Later Release, 3.2 release Mar 22, 2018

bradleeedmondson added discuss on legal call and removed legal issue labels Apr 5, 2018

wking approved these changes Apr 5, 2018

View reviewed changes

bradleeedmondson merged commit 5f44596 into spdx:master Apr 5, 2018

mlinksva deleted the patch-4 branch April 5, 2018 20:10

bradleeedmondson mentioned this pull request Apr 5, 2018

Use CC0-1.0 license text from source github/choosealicense.com#488

Merged

goneall mentioned this pull request Apr 9, 2018

Show regexp in HTML output for replacable text spdx/LicenseListPublisher#17

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CC0-1.0 accommodation for text used in ~30k repos on github.com #489

CC0-1.0 accommodation for text used in ~30k repos on github.com #489

mlinksva commented Nov 25, 2017

wking Nov 26, 2017

mlinksva Nov 26, 2017

wking Nov 26, 2017

mlinksva Nov 26, 2017

mlinksva Nov 26, 2017

goneall Nov 26, 2017

wking Nov 27, 2017

mlinksva Nov 27, 2017

wking Nov 27, 2017 •

edited

Loading

wking Nov 27, 2017

mlinksva Nov 27, 2017

jlovejoy commented Dec 19, 2017

jlovejoy commented Dec 19, 2017

mlinksva commented Dec 19, 2017

jlovejoy commented Dec 21, 2017

wking commented Dec 26, 2017 via email

wking Jan 27, 2018

mlinksva commented Jan 27, 2018

wking Jan 27, 2018

mlinksva Jan 28, 2018

mlinksva commented Jan 28, 2018 •

edited

Loading

wking commented Jan 28, 2018

mlinksva commented Jan 28, 2018

goneall commented Jan 28, 2018

wking commented Jan 28, 2018 via email

bradleeedmondson commented Apr 5, 2018

wking left a comment

wking commented Apr 5, 2018

bradleeedmondson commented Apr 5, 2018

mlinksva commented Apr 5, 2018

bradleeedmondson commented Apr 5, 2018

CC0-1.0 accommodation for text used in ~30k repos on github.com #489

CC0-1.0 accommodation for text used in ~30k repos on github.com #489

Conversation

mlinksva commented Nov 25, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wking Nov 27, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jlovejoy commented Dec 19, 2017

jlovejoy commented Dec 19, 2017

mlinksva commented Dec 19, 2017

jlovejoy commented Dec 21, 2017

wking commented Dec 26, 2017 via email

Choose a reason for hiding this comment

mlinksva commented Jan 27, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mlinksva commented Jan 28, 2018 • edited Loading

wking commented Jan 28, 2018

mlinksva commented Jan 28, 2018

goneall commented Jan 28, 2018

wking commented Jan 28, 2018 via email

bradleeedmondson commented Apr 5, 2018

wking left a comment

Choose a reason for hiding this comment

wking commented Apr 5, 2018

bradleeedmondson commented Apr 5, 2018

mlinksva commented Apr 5, 2018

bradleeedmondson commented Apr 5, 2018

wking Nov 27, 2017 •

edited

Loading

mlinksva commented Jan 28, 2018 •

edited

Loading