Data sets from another repo(s)

[omaralsoudanii]

Is it possible to retrieve data sets metadata from another repository as a harvesting method without having access or api key to said repository?
Regards

[jggautier]

Hi! It sounds like you'd like to be able to harvest metadata from one repository, A, which you said you don't have admin access to, into another repository, B. Do you have admin access to Repository B so that you can its access harvesting settings? If so, you would just need harvesting info from Repository A, and you wouldn't necessarily need admin access to that repository to get that info.

[omaralsoudanii]

Hi @jggautier what i meant is if there is available API end point to get public data sets metadata from another repo that doesn't require a token or any config from that repo, For example :
GET $SERVER_NAME/dataverse/DOI:$DOI_ID
which will return the meta data for said data set.

[pdurbin]

@omaralsoudanii when you say "another repository" do you mean another Dataverse repository or do you mean another non-Dataverse repository like DSpace, Fedora Commons, CKAN, etc.?

[omaralsoudanii]

@pdurbin another dataverse repo ,Example :
want to retrieve meta data of :
https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/TLPDMS
or :
https://dataverse.nl/dataset.xhtml?persistentId=hdl:10411/Y4R7GN
or :
https://data.cipotato.org/dataset.xhtml?persistentId=doi:10.21223/P3/LDMFOM

[pdurbin]

@omaralsoudanii yes. Dataverse's OAI-PMH implementation, which we call "harvesting," does not require any authentication. You can read about this feature at http://guides.dataverse.org/en/4.7/admin/harvestclients.html

Here's a link that might help get you started: https://dataverse.nl/oai?verb=ListIdentifiers&metadataPrefix=oai_ddi

If you are looking for a list of Dataverse servers to harvest from, right now they are listed in a spreadsheet at https://docs.google.com/spreadsheets/d/12cxymvXCqP_kCsLKXQD32go79HBWZ1vU_tdG4kvP5S8/edit?usp=sharing but as mentioned at https://groups.google.com/d/msg/dataverse-community/fqyVxWZSDRA/Iyf2l0MnBgAJ some day we would like to have this list appear on https://dataverse.org

In addition to OAI-PMH, you can pull metadata out of a remote Dataverse installation using the Search API: http://guides.dataverse.org/en/4.7/api/search.html . As installations of Dataverse upgrade to Dataverse 4.7.1 an API token will not be required out of the box (#3900) but some Dataverse installations may choose to revert to the old behavior in Dataverse 4.7 where an API token is required to use the Search API. And example of a service that uses the Search API to pull metadata out of Dataverse is https://share.osf.io/discover?q=*&sources=Harvard%20Dataverse

I hope this helps! Please let us know if you have any questions.

[djbrooke]

I'm going to close this out for now, but feel free to reopen and continue the conversation if needed!