Dataset Metadata - clean up titles and other metadata that contain special characters

[raprasad]

(in process)
Review <, and > tags:

https://dataverse.harvard.edu/dataset.xhtml?persistentId=hdl%3A1902.1/20291&version=1.0
https://dataverse.harvard.edu/dataset.xhtml?persistentId=hdl%3A1902.1/20452&version=1.0

[pdurbin]

@raprasad can we close this? It sounds like a data curation issue rather than a code issue.

[pdurbin]

Discussed with @raprasad and we agree it's a data curation issue, not a code issue. @jggautier or @sbarbosadataverse can you please take a look?

[jggautier]

Just came across this dataset with an ampersand in the title: http://dx.doi.org/10.7910/DVN/XJVVQX

In the citation box (and the breadcrumb), & winds up in the dataset's title. (Not sure how related #3845 is.)

I imagined solving this issue could involve querying the database to identify datasets that have unsupported characters in certain metadata fields (searching for those datasets isn't possible now (#2702)). We could curate those datasets more thoroughly, and if we have a better sense of how often unsupported characters are entered in metadata fields, we could make a case for development work that could help with curation (e.g. something that might allow or transform those unsupported characters or add validation to those fields so the depositor is warned about unsupported characters before creating datasets).

I agree this is an ongoing issue with no clear definition of done.

[pdurbin]

@jggautier thanks for bringing this issue to our attention. It's certainly related to #3845 so I grabbed it to at least leave a comment when I move a pull request into code review.

[pameyer]

One suggestion I mentioned to @jggautier is to run a validator on the exported metadata file, rather than querying the database for known problematic strings (with the principle of "asserting goodness" rather than "enumerating badness"

[jggautier]

Closing this, since it's being tracked in Harvard Dataverse's curation github.