Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2nd Beta Deployment, 5/20 #11

Merged
merged 53 commits into from
May 20, 2014
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
9faa3bd
whoops, in ff4cd45 missed file type card #3961
pdurbin May 8, 2014
51636d0
Added "disabled" style class to the pagination First/Previous links i…
May 9, 2014
5e963a3
Also added "disabled" style class to the bottom pagination First/Prev…
May 9, 2014
373e3d8
Cleaned up the "Add "Replication Data for" to Title" button on the ed…
May 9, 2014
269412f
split out docs for Data Deposit and Data Sharing APIs #3385
pdurbin May 9, 2014
f441258
explain how to set up PageKite
pdurbin May 9, 2014
bd54c4d
Minor layout improvements for spacing and font-size to Draft/Unpublis…
May 12, 2014
4300c1e
Merge origin/master
May 12, 2014
5a3c8d3
re-format file with Netbeans #3433
pdurbin May 12, 2014
20a8bde
start adding SWORD (service document with many todo's) #3385
pdurbin May 12, 2014
5b38010
stub out SWORD dataset creation (many todo's) #3385
pdurbin May 13, 2014
9f3ec0d
fix minimum semi-viable sword dataset creation #3385
pdurbin May 13, 2014
60b37eb
use pom config from https://github.com/swordapp/JavaServer2.0 #3385
pdurbin May 13, 2014
c98655d
Layout improvements for the dataset summary section (above-the-tabs m…
May 13, 2014
f150184
Merge origin/master
May 13, 2014
7a6dc03
added sword list-studies script #3385
pdurbin May 13, 2014
5e1275f
Changed the link URL for About in the header.
May 13, 2014
7ce0a98
Merge origin/master
May 13, 2014
8867b13
Updated footer copyright to be consistent with the copyright in the g…
mheppler May 14, 2014
80a39ac
adding a python wrapper for the api
raprasad May 14, 2014
b1279db
indent output for .md file
raprasad May 14, 2014
39c9770
another readme update
raprasad May 14, 2014
f7d412b
Need to troubleshoot why list datasets api throws
raprasad May 14, 2014
66a3ba4
s/DVN/Dataverse/
pdurbin May 14, 2014
b0c7080
Added logo alignment drop down menu and logo background color field t…
xyang02 May 14, 2014
078c3e9
Added logo alignment drop down menu and logo background color field t…
xyang02 May 14, 2014
bba4e6e
stub out SWORD file upload (many todo's) #3385
pdurbin May 14, 2014
e86e528
Fixed a bug on create mode of dataset pg where labels were displaying…
May 14, 2014
59f37f2
Hide Email from metadata display
sekmiller May 14, 2014
7e187b4
Update appendix.rst
May 15, 2014
d713334
Update appendix.rst
May 15, 2014
03b4507
Update appendix.rst
May 15, 2014
e9128cc
Update appendix.rst
May 15, 2014
abfb004
Applied new customization settings for positioning a rectangle logo, …
May 15, 2014
fa7b36e
Merge origin/master
May 15, 2014
84a69e7
removed log4j from pom.xml
scolapasta May 15, 2014
9cdedf1
stub out SWORD statement (lots of todo's) #3385
pdurbin May 15, 2014
8752cdb
stub out SWORD delete-dataset and show-atom-entry #3385
pdurbin May 16, 2014
97639c5
Removed help text block from top of page and added new help text bloc…
mheppler May 16, 2014
6ef43a5
Merge origin/master
mheppler May 16, 2014
a6d263e
stub out dataset lookup by globalId #3988
pdurbin May 16, 2014
7229e64
rename list-studies script to list-datasets
pdurbin May 19, 2014
5954c74
validate passwords via SWORD #3385
pdurbin May 19, 2014
6721413
log4.properties file - an attempt to get atmosphere runtime to shut up.
landreev May 19, 2014
5dc1855
Round-trip json tests for DatasetFields PASSING
michbarsinai May 20, 2014
06222bf
Merge branch 'master' of github.com:IQSS/dataverse
michbarsinai May 20, 2014
40160f5
get globalId automatically
pdurbin May 20, 2014
4db2f8f
enable citation and persistent URL in deposit receipt #3385
pdurbin May 20, 2014
5d7a99a
DatasetField i/o unit tests passing
michbarsinai May 20, 2014
1bce041
Merge branch 'master' of github.com:IQSS/dataverse
michbarsinai May 20, 2014
3aa80d4
lastupdatetime is always null. fall back to alternatives #3385
pdurbin May 20, 2014
cce8ae2
added Add data button for when you are not logged in
scolapasta May 20, 2014
c6204f4
Cleaned up text in Add Data popup
scolapasta May 20, 2014
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,5 @@ michael-local
ehthumbs.db
Thumbs.db
.vagrant
*.pyc
scripts/api/py_api_wrapper/demo-data/*
8 changes: 8 additions & 0 deletions doc/Sphinx/source/API/data-deposit.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
Data Deposit API
================

Please note that while Dataverse Network (DVN) 3.6 and higher support a
new Data Deposit API documented at
http://thedata.harvard.edu/guides/dataverse-api-main.html#data-deposit-api
the API has not yet been added Dataverse 4.0 Beta. That work will be
completed in https://redmine.hmdc.harvard.edu/issues/3385
Original file line number Diff line number Diff line change
@@ -1,13 +1,5 @@
Dataverse Network APIs
++++++++++++++++++++++++

We strongly encourage anyone interested in building tools to
interoperate with the Dataverse Network to utilize our open source
APIs. Please visit our `website <http://thedata.org/book/apps>`__ for
examples of external apps that have been built to work with our APIs.

Data Sharing API
==================
================

As of version 3.0, a new API for programmatic access to the DVN data and
metadata has been added. The API allows a remote, non-DVN
Expand Down Expand Up @@ -307,17 +299,3 @@ type, file name and such)
| Authorization header supplied, but the authenticated user is not
| authorized to directly access the object protected by Access
| Permissions and/or Access Restrictions (“Terms of Use”).


Data Deposit API
===========

As of version 3.6, a new API for programmatic deposit of data and metadata to the Dataverse Network will be added. The API will allow a remote, non-Dataverse Network archive/application to deposit files and metadata to a Dataverse Network installation.

The latest information on this plugin is available here:
`https://redmine.hmdc.harvard.edu/issues/3108 <https://redmine.hmdc.harvard.edu/issues/3108>`__





10 changes: 8 additions & 2 deletions doc/Sphinx/source/API/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,17 @@
contain the root `toctree` directive.

API Guide
=======================================================
=========

We strongly encourage anyone interested in building tools to
interoperate with the Dataverse Network to utilize our open source
APIs. Please visit http://datascience.iq.harvard.edu/collaborations for
examples of external apps that have been built to work with our APIs.

Contents:

.. toctree::
:maxdepth: 2

dataverse-apis
data-sharing
data-deposit
1 change: 1 addition & 0 deletions doc/Sphinx/source/Developers/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,6 @@ Contents:
:maxdepth: 2

dev-main
tools


20 changes: 20 additions & 0 deletions doc/Sphinx/source/Developers/tools.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
=====
Tools
=====

PageKite
++++++++

PageKite is a fantastic service that can be used to share your
local development environment over the Internet on a public IP address.

With PageKite running on your laptop, the world can access a URL such as
http://pdurbin.pagekite.me to see what you see at http://localhost:8080

Sign up at https://pagekite.net and follow the installation instructions or simply download https://pagekite.net/pk/pagekite.py

The first time you run ``./pagekite.py`` a file at ``~/.pagekite.rc`` will be
created. You can edit this file to configure PageKite to serve up port 8080
(the default GlassFish HTTP port) or the port of your choosing.

According to https://pagekite.net/support/free-for-foss/ PageKite (very generously!) offers free accounts to developers writing software the meets http://opensource.org/docs/definition.php such as Dataverse.
33 changes: 3 additions & 30 deletions doc/Sphinx/source/User/appendix.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,39 +16,12 @@ currently proposed metadata fields for 4.0 (per metadata block):
- `Citation Metadata <https://docs.google.com/spreadsheet/ccc?key=0AjeLxEN77UZodDBaYTFPakhGaEpoa3hqZUJTOWZtclE&usp=sharing>`__ (compliant with `DDI 2.5 <http://www.ddialliance.org/>`__ and `DataCite 3.0 <http://schema.datacite.org/meta/kernel-3/index.html>`__)
- `Social Science & Humanities Metadata (DDI 2.5 compliant) <https://docs.google.com/spreadsheet/ccc?key=0AjeLxEN77UZodEppcTFHT1NnajNLV0tacE10NEdmUnc&usp=sharing>`__
- `Astronomy and Astrophysics Metadata <https://docs.google.com/spreadsheet/ccc?key=0AjeLxEN77UZodEp4Qmp0QURkUWo1S0t4X3hia0FnZUE&usp=sharing>`__
: These metadata elements can be mapped/exported to the International Virtual Observatory Alliance’s (IVOA) Resource Metadata for the Virtual Observatory (VOResource Schema format) and based on `Virtual Observatory (VO) Discovery and Provenance Metadata <http://www.wf4ever-project.org/wiki/download/attachments/1179927/DPmetadata.pdf?version=1&modificationDate=1337186963000>`__)
: These metadata elements can be mapped/exported to the International Virtual Observatory Alliance’s (IVOA)
`VOResource Schema format <http://www.ivoa.net/documents/latest/RM.html>`__ and is based on
`Virtual Observatory (VO) Discovery and Provenance Metadata <http://www.wf4ever-project.org/wiki/download/attachments/1179927/DPmetadata.pdf?version=1&modificationDate=1337186963000>`__
- `Biomedical Metadata <https://docs.google.com/spreadsheet/ccc?key=0AjeLxEN77UZodExsRTB2SEpVWWd1Qmx6M09HSkExd3c&usp=sharing>`__
(based on `ISA-Tab <http://isatab.sourceforge.net/format.html>`__ and `Stem Cell Commons <http://stemcellcommons.org/>`__)

DDI 2.5
--------

The Dataverse metadata is compliant with the `DDI schema
version 2.5 <http://www.ddialliance.org/>`__. The Metadata fields in the Citation and Social Science blocks
associated with each Dataset contain most of the fields
in the study description section of the DDI. That way the Dataverse
metadata can be mapped easily to a DDI, and be exported into XML
format for preservation and interoperability.

DataCite Schema 3.0
--------------------

Dataverse metadata is compliant with the `DataCite Metadata Schema v.3.0 <http://schema.datacite.org/meta/kernel-3/index.html>`__ requirements. That way the Dataverse
metadata can be mapped easily to a DataCite, and be exported into XML
format for preservation and interoperability.

DCMI Terms
-----------

Dataverse metadata is compliant with `Dublin Core Metadata Initiative Terms <http://dublincore.org/documents/dcmi-terms/>`__ (DCMI Terms) requirements. That way the Dataverse
metadata can be mapped easily to DCMI Terms, and be exported into XML
format for preservation and interoperability.


FGDC/CSDGM (imports)
---------------------

For imports only, Dataverse data is compliant with the `Content Standard
for Digital Geospatial Metadata (CSDGM), Vers. 2 (FGDC-STD-001-1998) <http://www.fgdc.gov/metadata>`__ (FGDC).


41 changes: 40 additions & 1 deletion pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,46 @@
<version>3.0.1</version>
<type>jar</type>
</dependency>
<dependency>
<!-- BEGIN Data Deposit API v1 (SWORD v2) -->
<dependency>
<groupId>org.swordapp</groupId>
<artifactId>sword2-server</artifactId>
<version>1.0</version>
<type>jar</type>
<classifier>classes</classifier>
</dependency>
<dependency>
<groupId>org.swordapp</groupId>
<artifactId>sword2-server</artifactId>
<version>1.0</version>
<type>war</type>
</dependency>
<dependency>
<!-- required by org.swordapp.server.sword2-server -->
<groupId>org.apache.abdera</groupId>
<artifactId>abdera-core</artifactId>
<version>1.1.3</version>
</dependency>
<dependency>
<!-- required by org.swordapp.server.sword2-server -->
<groupId>org.apache.abdera</groupId>
<artifactId>abdera-parser</artifactId>
<version>1.1.3</version>
</dependency>
<dependency>
<!-- required by org.swordapp.server.sword2-server -->
<groupId>commons-fileupload</groupId>
<artifactId>commons-fileupload</artifactId>
<version>1.3.1</version>
</dependency>
<dependency>
<!-- required by org.swordapp.server.sword2-server -->
<groupId>xom</groupId>
<artifactId>xom</artifactId>
<version>1.1</version>
</dependency>
<!-- END Data Deposit API v1 (SWORD v2) -->
<dependency>
<groupId>javax</groupId>
<artifactId>javaee-api</artifactId>
<version>7.0</version>
Expand Down
7 changes: 7 additions & 0 deletions scripts/api/data-deposit/create-dataset
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#!/bin/bash
USERNAME=pete
PASSWORD=pete
DVN_SERVER=localhost:8181
DATAVERSE_ALIAS=peteTop
curl --insecure --data-binary "@scripts/api/data-deposit/data/atom-entry-study.xml" -H "Content-Type: application/atom+xml" https://$USERNAME:$PASSWORD@$DVN_SERVER/dvn/api/data-deposit/v1/swordv2/collection/dataverse/$DATAVERSE_ALIAS \
| xmllint -format -
41 changes: 41 additions & 0 deletions scripts/api/data-deposit/data/atom-entry-study.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
<?xml version="1.0"?>
<!--
modified from http://swordapp.github.io/SWORDv2-Profile/SWORDProfile.html#protocoloperations_editingcontent_metadata

comments at https://docs.google.com/document/d/1Rn70XItfA6_mJ4JA3I7m4em0spfdXIIHhwkrgCl40uE/edit?usp=sharing

see also https://redmine.hmdc.harvard.edu/issues/3246
-->
<entry xmlns="http://www.w3.org/2005/Atom"
xmlns:dcterms="http://purl.org/dc/terms/">
<!-- some embedded metadata -->
<dcterms:title>Roasting at Home</dcterms:title>
<dcterms:creator>Peets, John</dcterms:creator>
<dcterms:creator>Stumptown, Jane</dcterms:creator>
<!-- Producer with financial or admin responsibility of the data -->
<dcterms:publisher>Coffee Bean State University</dcterms:publisher>
<!-- related publications -->
<dcterms:isReferencedBy holdingsURI="http://dx.doi.org/10.1038/dvn333" agency="DOI" IDNo="10.1038/dvn333">Peets, J., &amp; Stumptown, J. (2013). Roasting at Home. New England Journal of Coffee, 3(1), 22-34.</dcterms:isReferencedBy>
<!-- production date -->
<dcterms:date>2013-07-11</dcterms:date>
<!-- Other Identifier for the data in this study (or potentially global id if unused) -->
<!--
<dcterms:identifier>hdl:1XXZY.1/XYXZ</dcterms:identifier>
-->
<dcterms:description>Considerations before you start roasting your own coffee at home.</dcterms:description>
<!-- keywords -->
<dcterms:subject>coffee</dcterms:subject>
<dcterms:subject>beverage</dcterms:subject>
<dcterms:subject>caffeine</dcterms:subject>
<!-- geographic coverage -->
<dcterms:coverage>United States</dcterms:coverage>
<dcterms:coverage>Canada</dcterms:coverage>
<!-- kind of data -->
<dcterms:type>aggregate data</dcterms:type>
<!-- List of sources of the data collection-->
<dcterms:source>Stumptown, Jane. 2011. Home Roasting. Coffeemill Press.</dcterms:source>
<!-- restrictions -->
<dcterms:rights>Creative Commons CC-BY 3.0 (unported) http://creativecommons.org/licenses/by/3.0/</dcterms:rights>
<!-- related materials -->
<dcterms:relation>Peets, John. 2010. Roasting Coffee at the Coffee Shop. Coffeemill Press</dcterms:relation>
</entry>
Binary file added scripts/api/data-deposit/data/example.zip
Binary file not shown.
9 changes: 9 additions & 0 deletions scripts/api/data-deposit/delete-dataset
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
#!/bin/bash -x
USERNAME=pete
PASSWORD=pete
DVN_SERVER=localhost:8181
#GLOBAL_ID=hdl:TEST/12345
GLOBAL_ID=`scripts/api/data-deposit/list-datasets | xpath '//id/text()' | cut -d'/' -f11,12`
#curl --insecure -X DELETE https://$DVN_SERVER/api/datasets/$DATABASE_ID?key=$USERNAME
curl --insecure -i -X DELETE https://$USERNAME:$PASSWORD@$DVN_SERVER/dvn/api/data-deposit/v1/swordv2/edit/study/$GLOBAL_ID
#| xmllint -format -
7 changes: 7 additions & 0 deletions scripts/api/data-deposit/list-datasets
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#!/bin/bash
USERNAME=pete
PASSWORD=pete
DVN_SERVER=localhost:8181
DATAVERSE_ALIAS=peteTop
curl --insecure https://$USERNAME:$PASSWORD@$DVN_SERVER/dvn/api/data-deposit/v1/swordv2/collection/dataverse/$DATAVERSE_ALIAS \
| xmllint -format -
2 changes: 2 additions & 0 deletions scripts/api/data-deposit/service-document
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/bin/bash
curl --insecure https://pete:pete@localhost:8181/dvn/api/data-deposit/v1/swordv2/service-document | xmllint -format -
8 changes: 8 additions & 0 deletions scripts/api/data-deposit/show-atom-entry
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/bin/sh
USERNAME=pete
PASSWORD=pete
DVN_SERVER=localhost:8181
#GLOBAL_ID=hdl:TEST/12345
GLOBAL_ID=`scripts/api/data-deposit/list-datasets | xpath '//id/text()' | cut -d'/' -f11,12`
curl --insecure -s https://$USERNAME:$PASSWORD@$DVN_SERVER/dvn/api/data-deposit/v1/swordv2/edit/study/$GLOBAL_ID \
| xmllint -format -
3 changes: 3 additions & 0 deletions scripts/api/data-deposit/show-files
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/sh
#scripts/api/data-deposit/show-statement | xpath "//entry/content/@*[name()='type' or name()='src']"
scripts/api/data-deposit/show-statement | xpath '//entry/id/text()' | cut -d'/' -f11,12
8 changes: 8 additions & 0 deletions scripts/api/data-deposit/show-statement
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/bin/sh
USERNAME=pete
PASSWORD=pete
DVN_SERVER=localhost:8181
GLOBAL_ID=`scripts/api/data-deposit/list-datasets | xpath '//id/text()' | cut -d'/' -f11,12`
curl --insecure -s https://$USERNAME:$PASSWORD@$DVN_SERVER/dvn/api/data-deposit/v1/swordv2/statement/study/$GLOBAL_ID \
| xmllint -format - \
#| xpath '//entry/title'
7 changes: 7 additions & 0 deletions scripts/api/data-deposit/upload-file
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#!/bin/sh
USERNAME=pete
PASSWORD=pete
DVN_SERVER=localhost:8181
GLOBAL_ID=`scripts/api/data-deposit/list-datasets | xpath '//id/text()' | cut -d'/' -f11,12`
curl -s --insecure --data-binary @scripts/api/data-deposit/data/example.zip -H "Content-Disposition: filename=example.zip" -H "Content-Type: application/zip" -H "Packaging: http://purl.org/net/sword/package/SimpleZip" https://$USERNAME:$PASSWORD@$DVN_SERVER/dvn/api/data-deposit/v1/swordv2/edit-media/study/$GLOBAL_ID \
| xmllint -format -
Loading