standardize license configuration #9262

philippconzett · 2023-01-05T09:02:55Z

What this PR does / why we need it:
This PR is part of the work to make license information from Dataverse installations harvestable in a way that complies with DataCite recommendations as described in issue #8512 Standardize standard license configuration.

Which issue(s) this PR closes:
Together with other PRs, this PR closes #8512.

Special notes for your reviewer:
See my comment from January 5, 2023 to issue #8512.

Suggestions on how to test this:
In a test environmen:

Install the JSON license files.
Create a dataset.
Select a standard license.
Export DataCite metadata.
Verify that the license information is as recommended by DataCite.

Does this PR introduce a user interface change? If mockups are available, please link/include them here:
No.

Is there a release notes update needed for this change?:
Yes. See my comment from January 5, 2023 to issue #8512.

Additional documentation:
The longer we wait implementing this change, the more installations will have to do clean-ups. Therefore, I hope this PR can be prioritized.

pdurbin

Took a quick look. I like this data-driven approach. Create the JSON files first, fix the code later. However, the code need to be fixed. And the existing licenses need to be moved into new database columns that need to be added as part of this PR. I'm not sure what size to give this. Due to uncertainty, maybe an 80 (56 hours).

pdurbin · 2023-01-05T14:01:21Z

scripts/api/data/licenses/licenseCC-BY-4.0.json

+  "schemeURI": "https://spdx.org/licenses/",
+  "rightsShortDescription": "Creative Commons Attribution 4.0 International.",
+  "rightsIconUrl": "https://licensebuttons.net/l/by/4.0/88x31.png",
+  "rightsActive": "true",


Not surprisingly edu.harvard.iq.dataverse.api.Licenses.addLicense is failing because the field names have changed. In this stack trace, for example, the code is looking for uri rather than rightsURI: addlicense.txt

This pull request is great for showing a future direction, but the code needs to be adjust to add new fields to the database and we'd need to write SQL upgrade scripts to migrate the old licenses into the new fields.

pdurbin · 2023-01-05T14:02:12Z

scripts/api/data/licenses/licenseAGPL-3.0-or-later.json

@@ -0,0 +1,10 @@
+{


@philippconzett can you please add a release note for this PR? (I'm just adding a comment here because it's the first line under "files changed".

pdurbin

We should update the docs as well.

pdurbin · 2023-01-05T14:26:17Z

scripts/api/data/licenses/licenseApache-2.0.json

@@ -0,0 +1,10 @@
+{
+  "rightsName": "Apache 2.0",


In the docs, we should add all these new licenses to the list at https://github.com/philippconzett/dataverse/blob/8512-standardize-license-configuration/doc/sphinx-guides/source/installation/config.rst#adding-licenses so they are easily downloadable to installations that would like to use them.

pdurbin · 2023-01-05T22:26:47Z

@philippconzett hi! Just checking if you're open to adding additional JSON files (additional licenses) to this PR. Please see some comments from earlier today starting here: IQSS/dataverse.harvard.edu#193 (comment)

Alternatively, we could add them later, of course, in a new PR.

Changed licenseURI to the be the SPDX URI, as requested in issue IQSS#9302.

pdurbin · 2024-04-11T16:55:34Z

JP and I just wrote some guidance on adding licenses the future: #10426 (comment)

Please take a look and let us know what you think!

jmurugan-fzj · 2024-05-23T11:29:07Z

@pdurbin Where can I find the up-to-date list of licenses which could be used for the dataset creation?
More specifically, what all values can be provided in the metadata "name" field below while creating the data set in dataverse

I tried "CC0 1.0" & "CC BY 4.0", both of them succeed, whereas the other versions for e.g. "CC BY 3.0" simply fails with the following server error: Reason: Bad Request, Info: {'status': 'ERROR', 'message': 'Error parsing Json: Invalid or unsupported license: CC BY 3.0'}

At-least it would be nice to see the list of presently supported values for "name" field, is this documented somewhere? I looked in the help pages, but could not find it anywhere quickly: complete list of presently supported license names!

Also I have noticed that the value set for "uri" field is not considered at all during the dataset creation, even if I give some wrong URI or invalid value too, the dataset is created with the license specified in the name field and appropriate URI is automatically populated from the name, is this a bug?

Metadata reference: dataset-create-new-all-default-fields.json

pdurbin · 2024-05-23T17:34:09Z

@jmurugan-fzj hi! Now that the following #10426 has been merged, there is new guidance coming in the next release (probably 6.3) on what to put for "name", "uri" etc. Here's a screenshot from a preview at https://dataverse-guide--10426.org.readthedocs.build/en/10426/installation/config.html#contributing-to-the-collection-of-standard-licenses-above

Does that help?

qqmyers · 2024-05-23T17:49:06Z

FWIW: /api/licenses tells you which licenses are installed and active (ones you can select for a new/draft dataset) for a give Dataverse installation.

jp-tosca · 2024-05-23T22:27:58Z

Hi @jmurugan-fzj, regarding your 3 points:

This message is a bit unclear, sorry about that. This is not a parsing error but indeed this happens because the license is not installed.
As @qqmyers mentioned, there is an API to check the licenses on the installation so that should eliminate these errors.
The URI parameter as documented on the code:

dataverse/src/main/java/edu/harvard/iq/dataverse/util/json/JsonParser.java

Lines 470 to 475 in 2bf05c1

    
           // If uri is provided, we'll try that first. This is an easier lookup 
        
           // method; the uri is always the same. The name may have been customized 
        
           // (translated) on this instance, so we may be dealing with such translated 
        
           // name, if this is exported json that we are processing. Meaning, unlike  
        
           // the uri, we cannot simply check it against the name in the License 
        
           // database table.

So it should be prioritized over the name to search for the license but it seems optional. Please feel free to open an issue if you think this has a different behavior. Let me know if you have any questions or if we can provide any additional help.

jmurugan-fzj · 2024-05-24T09:08:50Z

/api/licenses

@qqmyers Thanks for the information, I see that more details available here in native API page

I have one more question related to the license support: We have an older version: 4.20 installed in our institute and does not seem to support this particular endpoint:

https://data.fz-juelich.de/api/licenses
{"status":"ERROR","code":404,"message":"'/api/v1/licenses' endpoint does not exist on this server. Please check your code for typos, or consult our API guide at http://guides.dataverse.org."}
https://data.fz-juelich.de/api/info/version

Presently I use "CC0 1.0" for creating the datasets, but the license information is populated as "None" in the created metadata: e.g. sample created dataset metadata

So I assume the license management is not supported at all in version 4.2?

Do you know from which version on wards is the license terms management properly supported? We would like to have the license terms also properly documented along with our datasets. If you could tell me a particular version, this will be really helpful and I can ask the administration here to upgrade to that particular one or above.

jmurugan-fzj · 2024-05-24T09:10:16Z

@jmurugan-fzj hi! Now that the following #10426 has been merged, there is new guidance coming in the next release (probably 6.3) on what to put for "name", "uri" etc. Here's a screenshot from a preview at https://dataverse-guide--10426.org.readthedocs.build/en/10426/installation/config.html#contributing-to-the-collection-of-standard-licenses-above

Does that help?

@pdurbin Thank you very much, this is exactly what I was looking for and very helpful (y)

jmurugan-fzj · 2024-05-24T09:22:44Z

Hi @jmurugan-fzj, regarding your 3 points:

This message is a bit unclear, sorry about that. This is not a parsing error but indeed this happens because the license is not installed.

As @qqmyers mentioned, there is an API to check the licenses on the installation so that should eliminate these errors.

The URI parameter as I documented on the code

dataverse/src/main/java/edu/harvard/iq/dataverse/util/json/JsonParser.java

Lines 470 to 475 in 2bf05c1

// If uri is provided, we'll try that first. This is an easier lookup

// method; the uri is always the same. The name may have been customized

// (translated) on this instance, so we may be dealing with such translated

// name, if this is exported json that we are processing. Meaning, unlike

// the uri, we cannot simply check it against the name in the License

// database table.

So it should be prioritized over the name to search for the license but it seems optional. Please feel free to open an issue if you think this has a different behavior. Let me know if you have any questions or if we can provide any additional help.

@jp-tosca Thanks again for the detailed response, sorry for being unclear in my previous question, I was not aware of the fact that the licenses need to be added before creating the datasets, with the help page links provided above, now the license terms usage is more clear to me :)

Regarding the URI field, I don't see this really as a critical issue at the moment, now that I am aware of the endpoint:/api/licenses, I can even restrict the possible values for the "name" & "url" field, so all good! Thanks again for the quick response and support (y)

qqmyers · 2024-05-24T11:10:01Z

@jmurugan-fzj - support for multiple licenses was added in v5.10 and there's a v5.10.1 that is probably the minimum you'd want. That said, we strongly recommend keeping current.

jmurugan-fzj · 2024-05-24T11:28:38Z

@jmurugan-fzj - support for multiple licenses was added in v5.10 and there's a v5.10.1 that is probably the minimum you'd want. That said, we strongly recommend keeping current.

@qqmyers That's very helpful, let me discuss with the team here and I would also like to go for the latest version, Thanks again :)

philippconzett added 8 commits January 5, 2023 09:44

Updated licenseCC-BY-4.0.json

16599aa

Updated licenseCC-BY-NC-4.0.json

9a30cb9

Updated licenseCC-BY-NC-ND-4.0.json

1f510f5

Updated licenseCC-BY-NC-SA-4.0.json

beada4f

Updated licenseCC-BY-ND-4.0.json

b93327a

Updated licenseCC-BY-SA-4.0.json

83ea0bf

Updated licenseCC0-1.0.json

662719a

Added additional license files

4c8774e

philippconzett mentioned this pull request Jan 5, 2023

Feature Request/Idea: Standardize standard license configuration #8512

Closed

pdurbin reviewed Jan 5, 2023

View reviewed changes

pdurbin marked this pull request as draft January 5, 2023 16:16

pdurbin mentioned this pull request Jan 5, 2023

Spike: Add more licenses using multiple license feature IQSS/dataverse.harvard.edu#193

Closed

pdurbin mentioned this pull request Jan 20, 2023

add custom license for France to the Dataverse Doc #9302

Draft

mreekie added the Size: Queued PM has called this issue out specifically for sizing label Jan 23, 2023

Update licenseetalab-2.0.json

40225d3

Changed licenseURI to the be the SPDX URI, as requested in issue IQSS#9302.

dozed mentioned this pull request Sep 22, 2023

Feature Request/Idea: Provide an "Open Access" badge on a dataset page #9943

Open

pdurbin added Size: 80 A percentage of a sprint. 56 hours. and removed Size: Queued PM has called this issue out specifically for sizing labels Feb 28, 2024

pdurbin mentioned this pull request Mar 14, 2024

Add MIT Software license to HDV IQSS/dataverse.harvard.edu#248

Closed

pdurbin changed the title ~~8512 standardize license configuration~~ standardize license configuration Mar 27, 2024

pdurbin mentioned this pull request Apr 11, 2024

Add MIT license file and guidance for more license files #10426

Merged

pdurbin added the Type: Feature a feature request label Oct 9, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

standardize license configuration #9262

standardize license configuration #9262

philippconzett commented Jan 5, 2023

pdurbin left a comment

pdurbin Jan 5, 2023

pdurbin Jan 5, 2023

pdurbin left a comment

pdurbin Jan 5, 2023

pdurbin commented Jan 5, 2023

pdurbin commented Apr 11, 2024

jmurugan-fzj commented May 23, 2024 •

edited

Loading

pdurbin commented May 23, 2024

qqmyers commented May 23, 2024

jp-tosca commented May 23, 2024 •

edited

Loading

jmurugan-fzj commented May 24, 2024 •

edited

Loading

jmurugan-fzj commented May 24, 2024

jmurugan-fzj commented May 24, 2024

qqmyers commented May 24, 2024

jmurugan-fzj commented May 24, 2024

standardize license configuration #9262

Are you sure you want to change the base?

standardize license configuration #9262

Conversation

philippconzett commented Jan 5, 2023

pdurbin left a comment

Choose a reason for hiding this comment

pdurbin Jan 5, 2023

Choose a reason for hiding this comment

pdurbin Jan 5, 2023

Choose a reason for hiding this comment

pdurbin left a comment

Choose a reason for hiding this comment

pdurbin Jan 5, 2023

Choose a reason for hiding this comment

pdurbin commented Jan 5, 2023

pdurbin commented Apr 11, 2024

jmurugan-fzj commented May 23, 2024 • edited Loading

pdurbin commented May 23, 2024

qqmyers commented May 23, 2024

jp-tosca commented May 23, 2024 • edited Loading

jmurugan-fzj commented May 24, 2024 • edited Loading

jmurugan-fzj commented May 24, 2024

jmurugan-fzj commented May 24, 2024

qqmyers commented May 24, 2024

jmurugan-fzj commented May 24, 2024

jmurugan-fzj commented May 23, 2024 •

edited

Loading

jp-tosca commented May 23, 2024 •

edited

Loading

jmurugan-fzj commented May 24, 2024 •

edited

Loading