make Alternative Title repeatable #9440

lubitchv · 2023-03-13T17:37:03Z

What this PR does / why we need it:
Make "Alternative Title" multiple (repeatable) field such that DDI export would not break.

Which issue(s) this PR closes:
Closes #9428

Special notes for your reviewer:
If one does not want "Alternative Title" to be repeatable it is enough to remove TRUE from citation.tsv and solr, but leave the changes in the code that will allow for those who wants repeatable.
Suggestions on how to test this:
Create a dataset with multiple alternative titles, publish it. Test DDI Exporter and OpenAire exporter. It should not break and should have multiple alternative titles.

Does this PR introduce a user interface change? If mockups are available, please link/include them here:
Yes

Is there a release notes update needed for this change?:
Yes. 9428-alternative-title.md

Update

Update from IQSS

Update from develop

lubitchv · 2023-03-13T18:49:04Z

I do not have access to continuous integration testing server. Please let me know which integration tests are failing.

pdurbin · 2023-03-14T21:29:24Z

@lubitchv yeah, it would be very nice to give this insight on the outside somehow. Here are some screenshots at least! From https://jenkins.dataverse.org/job/IQSS-Dataverse-Develop-PR/job/PR-9440/2/testReport/

lubitchv · 2023-03-16T18:59:01Z

The testImportDDI had error for productionPlace field (there was problem there with multiple). It was not related to this PR, but I fixed it. testArchivalStatusAPI and testLoadMetadataBlock_NoErrorPath I also fixed. But it seems that some integration tests are still failing and I do not know which.

pdurbin · 2023-03-16T20:04:50Z

@lubitchv this time ( https://jenkins.dataverse.org/job/IQSS-Dataverse-Develop-PR/job/PR-9440/5/ ) it wasn't API tests that failed. No API tests were run at all. I just clicked "re-run".

lubitchv · 2023-03-27T16:44:28Z

API tests are now running fine.

pdurbin · 2023-04-26T02:06:57Z

@lubitchv hi! Can you please resolve the merge conflicts in the PR?

There were probably caused by this PR being merged yesterday:

Make Dataverse produce valid DDI 3648 #9484

lubitchv · 2023-05-11T21:31:33Z

@pdurbin I updated the branch from develop

lubitchv · 2023-05-12T15:39:43Z

@pdurbin It says that there is 1 errored check, but I cannot see it, or I do not have permissions to see it.

pdurbin · 2023-05-12T19:56:31Z

@lubitchv hi! Sorry for the noise. The failure in Jenkins is unrelated to your changes. It's this:

Cannot find module dataverse-design-system gdcc/dataverse-ansible#281

pdurbin

Just some quick feedback and a few questions:

Is it valid in DDI to have more than one alternate title?

This pull request was recently merged and now Dataverse produces valid DDI (as far as I know):

Make Dataverse produce valid DDI 3648 #9484

I see there's a test in that PR that calls isValid. Does that test cover changes in this PR? If not, can we please have a test that shows that the DDI is still valid?

pdurbin · 2023-05-12T20:00:02Z

src/test/java/edu/harvard/iq/dataverse/export/ddi/dataset-create-new-all-ddi-fields.json

-            "value": "Alternative Title"
+            "value": ["Alternative Title"]


It looks like API users will encounter backward incompatibility. Is that right? If so, can this please be added to the release note snippet?

Yes, repeatable altTitl is valid DDI. See https://ddialliance.org/Specification/DDI-Codebook/2.5/XMLSchema/field_level_documentation_files/schemas/codebook_xsd/elements/titlStmt.html#a19 It has maxOccurs="unbounded"

I updated dataset-finch xml and json and added multiple alternative titles such that validity test can test it.

I also added comment about incompatibility into release note.

Could you please add a second value here, just like in dataset-finch1.json -

["Alternative Title1", "Alternative Title2"]

and that will need a corresponding change in src/test/java/edu/harvard/iq/dataverse/export/ddi/exportfull.xml that DdiExportUtilTest compares against.
(I'm putting together a list of all the changes that need to be made when fields are made repeatable, for other developers)
Thank you!

@landreev I added ["Alternative Title1", "Alternative Title2"] into dataset-create-new-all-ddi-fields.json and added another title in corresponding exportfull.xml

src/main/java/edu/harvard/iq/dataverse/export/openaire/OpenAireExportUtil.java

src/main/java/edu/harvard/iq/dataverse/export/ddi/DdiExportUtil.java

…l.java Co-authored-by: Philip Durbin <philipdurbin@gmail.com>

…eExportUtil.java Co-authored-by: Philip Durbin <philipdurbin@gmail.com>

coveralls · 2023-08-23T20:05:15Z

coverage: 19.987% (+0.02%) from 19.969% when pulling 85e6661 on lubitchv:9428-alternative-title into 62a9811 on IQSS:develop.

lubitchv · 2023-08-23T21:43:20Z

@pdurbin updated from develop, I see that there is some error, is it integration tests? Compilation on my local machine and unit tests are fine.

pdurbin · 2023-08-24T10:29:42Z

@lubitchv thanks for merging develop.

I just kicked off https://github.com/gdcc/api-test-runner/actions/runs/5962550825

After it finishes, can you see the results?

pdurbin · 2023-08-24T10:56:42Z

@poikilotherm when I open https://github.com/gdcc/api-test-runner/actions/runs/5962550825/job/16173949398 in a private window I can't see the details of the API test failure:

Do you know if we can open this up somehow? Or maybe as a workaround for now we add @lubitchv to that test runner repo?

@lubitchv I don't mean to keep you in suspense. Here's the failure:

@lubitchv I'm happy to help you get set up to run API tests on your laptop. Do you use Docker?

lubitchv · 2023-08-25T16:42:12Z

MetadataBlocksIT seems to work now. I am not sure what does 1 cancelled and 2 skipped checks mean.

pdurbin · 2023-08-25T16:58:23Z

@lubitchv yes! All tests passing now! Please don't worry about the skips and cancels. The skips are related to this:

Document Container Github Actions #9775

The cancel was me, sorry, related to this: #9830 (comment)

scolapasta · 2023-09-11T17:51:46Z

@lubitchv now that we've released 6.0, I'm moving waiting 6.1 issues back to the board.

But could you first handle the 6.0 merge and address any EE10 issues. Thanks.

…e into 9428-alternative-title Merge

lubitchv · 2023-09-11T19:08:43Z

Some integration tests are failing again. can I see which ones?

pdurbin · 2023-09-11T19:26:14Z

@lubitchv the failing tests might be due to Jenkins/Ansible not being updated to 6.0 yet:

bump to 6.0 release gdcc/dataverse-ansible#316

But I'm not sure. Here's the error I see at https://jenkins.dataverse.org/job/IQSS-Dataverse-Develop-PR/job/PR-9440/20/consoleFull

TASK [dataverse : fire off installer] ******************************************
[WARNING]: Module remote_tmp /home/dataverse/.ansible/tmp did not exist and was
created with a mode of 0700, this may cause issues when running as another
user. To avoid this, create the remote_tmp dir with the correct permissions
manually
fatal: [localhost]: FAILED! => {"changed": true, "cmd": "/usr/bin/python3 /tmp/dvinstall/install.py -f --config_file=default.config --noninteractive > /tmp/dvinstall/install.out 2>&1", "delta": "0:00:00.288717", "end": "2023-09-11 18:33:35.911959", "msg": "non-zero return code", "rc": 1, "start": "2023-09-11 18:33:35.623242", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}

Meanwhile, I tried a test run here: https://github.com/gdcc/api-test-runner/actions/runs/6150829101

pdurbin · 2023-09-11T20:11:38Z

Meanwhile, I tried a test run here: https://github.com/gdcc/api-test-runner/actions/runs/6150829101

All the tests passed: Tests run: 210, Failures: 0, Errors: 0, Skipped: 10

landreev

Looks perfect, thank you again!

kcondon · 2023-09-22T16:55:58Z

I'm merging this, thanks! I did find a minor search results display bug relating to fields that have changed from single to multiple in metadata but will open a separate issue since it exists already. Thanks for the help @landreev

pdurbin · 2024-02-28T18:35:44Z

Update JSON to reflect changes on the API dataverse-sample-data#41

lubitchv and others added 8 commits June 6, 2022 09:53

Merge pull request #64 from IQSS/develop

f2946cb

Update

Merge pull request #67 from IQSS/develop

ec31196

Update from IQSS

Merge pull request #68 from IQSS/develop

c5f44d6

Update from develop

Merge branch 'IQSS:develop' into develop

a1f2c78

Merge branch 'IQSS:develop' into develop

af793ec

Alternative Title

9bdcba6

citation

7b8281c

release notes

e5a356a

import

5d81fc7

lubitchv and others added 3 commits March 15, 2023 17:20

prodPlac

991c5f9

restore back

89dbc7a

Merge branch 'IQSS:develop' into 9428-alternative-title

538b815

pdurbin assigned lubitchv Apr 26, 2023

lubitchv and others added 3 commits May 11, 2023 16:38

develop + title

dbd97ff

Merge branch 'IQSS:develop' into 9428-alternative-title

be1e639

add prodPlac

3bb7cbf

pdurbin changed the title ~~9428 alternative title~~ make Alternative Title repeatable May 12, 2023

pdurbin reviewed May 12, 2023

View reviewed changes

lubitchv and others added 3 commits May 15, 2023 12:35

Update src/main/java/edu/harvard/iq/dataverse/export/ddi/DdiExportUti…

b9720c8

…l.java Co-authored-by: Philip Durbin <philipdurbin@gmail.com>

Update src/main/java/edu/harvard/iq/dataverse/export/openaire/OpenAir…

5684140

…eExportUtil.java Co-authored-by: Philip Durbin <philipdurbin@gmail.com>

Update src/main/java/edu/harvard/iq/dataverse/export/openaire/OpenAir…

64f4f1f

…eExportUtil.java Co-authored-by: Philip Durbin <philipdurbin@gmail.com>

Merge branch 'IQSS:develop' into 9428-alternative-title

10f3e1e

alt title multiple

a90aa22

scolapasta removed the Status: Needs Input Applied to issues in need of input from someone currently unavailable label Sep 11, 2023

lubitchv closed this Sep 11, 2023

lubitchv force-pushed the 9428-alternative-title branch from a90aa22 to 62a9811 Compare September 11, 2023 18:06

Merge branch '9428-alternative-title' of github.com:lubitchv/datavers…

85e6661

…e into 9428-alternative-title Merge

lubitchv reopened this Sep 11, 2023

landreev approved these changes Sep 18, 2023

View reviewed changes

landreev removed their assignment Sep 20, 2023

kcondon self-assigned this Sep 22, 2023

kcondon merged commit fa577fb into IQSS:develop Sep 22, 2023
20 checks passed

pdurbin mentioned this pull request Dec 18, 2023

Document the checklist of all the tasks necessary when making an existing single field multiple #9634

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

make Alternative Title repeatable #9440

make Alternative Title repeatable #9440

lubitchv commented Mar 13, 2023

lubitchv commented Mar 13, 2023

pdurbin commented Mar 14, 2023

lubitchv commented Mar 16, 2023

pdurbin commented Mar 16, 2023

lubitchv commented Mar 27, 2023

pdurbin commented Apr 26, 2023

lubitchv commented May 11, 2023

lubitchv commented May 12, 2023

pdurbin commented May 12, 2023

pdurbin left a comment

pdurbin May 12, 2023

lubitchv May 15, 2023

landreev Aug 4, 2023

lubitchv Aug 21, 2023

coveralls commented Aug 23, 2023 •

edited

Loading

lubitchv commented Aug 23, 2023

pdurbin commented Aug 24, 2023

pdurbin commented Aug 24, 2023

lubitchv commented Aug 25, 2023

pdurbin commented Aug 25, 2023

scolapasta commented Sep 11, 2023

lubitchv commented Sep 11, 2023

pdurbin commented Sep 11, 2023

pdurbin commented Sep 11, 2023

landreev left a comment

kcondon commented Sep 22, 2023

pdurbin commented Feb 28, 2024

make Alternative Title repeatable #9440

make Alternative Title repeatable #9440

Conversation

lubitchv commented Mar 13, 2023

lubitchv commented Mar 13, 2023

pdurbin commented Mar 14, 2023

lubitchv commented Mar 16, 2023

pdurbin commented Mar 16, 2023

lubitchv commented Mar 27, 2023

pdurbin commented Apr 26, 2023

lubitchv commented May 11, 2023

lubitchv commented May 12, 2023

pdurbin commented May 12, 2023

pdurbin left a comment

Choose a reason for hiding this comment

pdurbin May 12, 2023

Choose a reason for hiding this comment

lubitchv May 15, 2023

Choose a reason for hiding this comment

landreev Aug 4, 2023

Choose a reason for hiding this comment

lubitchv Aug 21, 2023

Choose a reason for hiding this comment

coveralls commented Aug 23, 2023 • edited Loading

lubitchv commented Aug 23, 2023

pdurbin commented Aug 24, 2023

pdurbin commented Aug 24, 2023

lubitchv commented Aug 25, 2023

pdurbin commented Aug 25, 2023

scolapasta commented Sep 11, 2023

lubitchv commented Sep 11, 2023

pdurbin commented Sep 11, 2023

pdurbin commented Sep 11, 2023

landreev left a comment

Choose a reason for hiding this comment

kcondon commented Sep 22, 2023

pdurbin commented Feb 28, 2024

coveralls commented Aug 23, 2023 •

edited

Loading