Skip to content

Commit

Permalink
Merge pull request #1 from IQSS/develop
Browse files Browse the repository at this point in the history
test
  • Loading branch information
xibriz authored Dec 1, 2017
2 parents caf6371 + 4648b6a commit 45288aa
Show file tree
Hide file tree
Showing 17 changed files with 457 additions and 29 deletions.
9 changes: 7 additions & 2 deletions doc/sphinx-guides/source/admin/metadataexport.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,12 @@ Metadata Export
Automatic Exports
-----------------

Unlike in DVN v3, publishing a dataset in Dataverse 4 automaticalliy starts a metadata export job, that will run in the background, asynchronously. Once completed, it will make the dataset metadata exported and cached in all the supported formats (Dublin Core, Data Documentation Initiative (DDI), and native JSON). There is no need to run the export manually.
Publishing a dataset automatically starts a metadata export job, that will run in the background, asynchronously. Once completed, it will make the dataset metadata exported and cached in all the supported formats:

- Dublin Core
- Data Documentation Initiative (DDI)
- Schema.org JSON-LD
- native JSON (Dataverse-specific)

A scheduled timer job that runs nightly will attempt to export any published datasets that for whatever reason haven't been exported yet. This timer is activated automatically on the deployment, or restart, of the application. So, again, no need to start or configure it manually. (See the "Application Timers" section of this guide for more information)

Expand All @@ -28,4 +33,4 @@ Note, that creating, modifying, or re-exporting an OAI set will also attempt to
Export Failures
---------------

An export batch job, whether started via the API, or by the application timer, will leave a detailed log in your configured logs directory. This is the same location where your main Glassfish server.log is found. The name of the log file is ``export_[timestamp].log`` - for example, *export_2016-08-23T03-35-23.log*. The log will contain the numbers of datasets processed successfully and those for which metadata export failed, with some information on the failures detected. Please attach this log file if you need to contact Dataverse support about metadata export problems.
An export batch job, whether started via the API, or by the application timer, will leave a detailed log in your configured logs directory. This is the same location where your main Glassfish server.log is found. The name of the log file is ``export_[timestamp].log`` - for example, *export_2016-08-23T03-35-23.log*. The log will contain the numbers of datasets processed successfully and those for which metadata export failed, with some information on the failures detected. Please attach this log file if you need to contact Dataverse support about metadata export problems.
2 changes: 1 addition & 1 deletion doc/sphinx-guides/source/api/native-api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ Delete the dataset whose id is passed::

GET http://$SERVER/api/datasets/export?exporter=ddi&persistentId=$persistentId

.. note:: Supported exporters (export formats) are ``ddi``, ``oai_ddi``, ``dcterms``, ``oai_dc``, and ``dataverse_json``.
.. note:: Supported exporters (export formats) are ``ddi``, ``oai_ddi``, ``dcterms``, ``oai_dc``, ``schema.org`` , and ``dataverse_json``.

|CORS| Lists all the file metadata, for the given dataset and version::

Expand Down
12 changes: 12 additions & 0 deletions doc/sphinx-guides/source/installation/config.rst
Original file line number Diff line number Diff line change
Expand Up @@ -385,6 +385,13 @@ Once you have the location of your custom header HTML file, run this curl comman

``curl -X PUT -d '/var/www/dataverse/branding/custom-header.html' http://localhost:8080/api/admin/settings/:HeaderCustomizationFile``

If you have enabled a custom header or navbar logo, you might prefer to disable the theme of the root dataverse. You can do so by setting ``:DisableRootDataverseTheme`` to ``true`` like this:

``curl -X PUT -d 'true' http://localhost:8080/api/admin/settings/:DisableRootDataverseTheme``

Please note: Disabling the display of the root dataverse theme also disables your ability to edit it. Remember that dataverse owners can set their dataverses to "inherit theme" from the root. Those dataverses will continue to inherit the root dataverse theme (even though it no longer displays on the root). If you would like to edit the root dataverse theme in the future, you will have to re-enable it first.


Custom Footer
+++++++++++++

Expand Down Expand Up @@ -679,6 +686,11 @@ See :ref:`Branding Your Installation` above.

See :ref:`Branding Your Installation` above.

:DisableRootDataverseTheme
++++++++++++++++++++++++++

See :ref:`Branding Your Installation` above.

:FooterCustomizationFile
++++++++++++++++++++++++

Expand Down
2 changes: 1 addition & 1 deletion doc/sphinx-guides/source/user/dataverse-management.rst
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ Tip: The metadata fields you select as required will appear on the Create Datase
Theme
====================================================

The Theme feature provides you with a way to customize the look of your dataverse. You can decide either to use the customization from the dataverse above yours or upload your own image file. Supported image types are JPEG, TIFF, or PNG and should be no larger than 500 KB. The maximum display size for an image file in a dataverse's theme is 940 pixels wide by 120 pixels high. Additionally, you can select the colors for the header of your dataverse and the text that appears in your dataverse. You can also add a link to your personal website, the website for your organization or institution, your department, journal, etc.
The Theme feature provides you with a way to customize the look of your dataverse. You can decide either to use the theme from the dataverse containing your dataverse (even up to the root dataverse, AKA the homepage), or upload your own image file. Supported image types are JPEG, TIFF, or PNG and should be no larger than 500 KB. The maximum display size for an image file in a dataverse's theme is 940 pixels wide by 120 pixels high. Additionally, you can select the colors for the header of your dataverse and the text that appears in your dataverse. You can also add a link to your personal website, the website for your organization or institution, your department, journal, etc.

.. _dataverse-widgets:

Expand Down
8 changes: 5 additions & 3 deletions src/main/java/Bundle.properties
Original file line number Diff line number Diff line change
Expand Up @@ -749,9 +749,9 @@ dataverse.results.cards.foundInMetadata=Found in Metadata Fields:
dataverse.results.cards.files.tabularData=Tabular Data
dataverse.results.solrIsDown=Please note: Due to an internal error, browsing and searching is not available.
dataverse.theme.title=Theme
dataverse.theme.inheritCustomization.title=Check this to use the existing theme.
dataverse.theme.inheritCustomization.label=Inherit Customization
dataverse.theme.inheritCustomization.checkbox=Inherit customization from {0}
dataverse.theme.inheritCustomization.title=For this dataverse, use the same theme as the parent dataverse.
dataverse.theme.inheritCustomization.label=Inherit Theme
dataverse.theme.inheritCustomization.checkbox=Inherit theme from {0}
dataverse.theme.logo=Logo
dataverse.theme.logo.tip=Supported image types are JPG, TIF, or PNG and should be no larger than 500 KB. The maximum display size for an image file in a dataverse's theme is 940 pixels wide by 120 pixels high.
dataverse.theme.logo.format=Logo Format
Expand Down Expand Up @@ -794,6 +794,7 @@ dataverse.theme.website.title=URL for your personal website, institution, or any
dataverse.theme.website.tip=The website will be linked behind the tagline. To have a website listed, you must also provide a tagline.
dataverse.theme.website.watermark=Your personal site, http://...
dataverse.theme.website.invalidMsg=Invalid URL.
dataverse.theme.disabled=The theme for the root dataverse has been administratively disabled with the :DisableRootDataverseTheme database setting.
dataverse.widgets.title=Widgets
dataverse.widgets.notPublished.why.header=Why Use Widgets?
dataverse.widgets.notPublished.why.reason1=Increases the web visibility of your data by allowing you to embed your dataverse and datasets into your personal or project website.
Expand Down Expand Up @@ -1106,6 +1107,7 @@ dataset.editBtn.itemLabel.deaccession=Deaccession Dataset
dataset.exportBtn=Export Metadata
dataset.exportBtn.itemLabel.ddi=DDI
dataset.exportBtn.itemLabel.dublinCore=Dublin Core
dataset.exportBtn.itemLabel.schemaDotOrg=Schema.org JSON-LD
dataset.exportBtn.itemLabel.json=JSON
metrics.title=Metrics
metrics.title.tip=View more metrics information
Expand Down
11 changes: 10 additions & 1 deletion src/main/java/edu/harvard/iq/dataverse/DatasetPage.java
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@
import edu.harvard.iq.dataverse.engine.command.impl.RestrictFileCommand;
import edu.harvard.iq.dataverse.engine.command.impl.ReturnDatasetToAuthorCommand;
import edu.harvard.iq.dataverse.engine.command.impl.SubmitDatasetForReviewCommand;
import edu.harvard.iq.dataverse.export.SchemaDotOrgExporter;
import java.util.Collections;

import javax.faces.event.AjaxBehaviorEvent;
Expand Down Expand Up @@ -4068,7 +4069,15 @@ public boolean isThisLatestReleasedVersion() {

public String getJsonLd() {
if (isThisLatestReleasedVersion()) {
return workingVersion.getJsonLd();
ExportService instance = ExportService.getInstance(settingsService);
String jsonLd = instance.getExportAsString(dataset, SchemaDotOrgExporter.NAME);
if (jsonLd != null) {
logger.fine("Returning cached schema.org JSON-LD.");
return jsonLd;
} else {
logger.fine("No cached schema.org JSON-LD available. Going to the database.");
return workingVersion.getJsonLd();
}
}
return "";
}
Expand Down
24 changes: 16 additions & 8 deletions src/main/java/edu/harvard/iq/dataverse/DatasetVersion.java
Original file line number Diff line number Diff line change
Expand Up @@ -1211,12 +1211,8 @@ public String getPublicationDateAsString() {
return r;
}

// TODO: Make this more performant by writing the output to the database or a file?
// Agree - now that this has grown into a somewhat complex chunk of formatted
// metadata - and not just a couple of values inserted into the page html -
// it feels like it would make more sense to treat it as another supported
// export format, that can be produced once and cached.
// The problem with that is that the export subsystem assumes there is only
// TODO: Consider moving this comment into the Exporter code.
// The export subsystem assumes there is only
// one metadata export in a given format per dataset (it uses the current
// released (published) version. This JSON fragment is generated for a
// specific released version - and we can have multiple released versions.
Expand All @@ -1240,10 +1236,18 @@ public String getJsonLd() {
for (DatasetAuthor datasetAuthor : this.getDatasetAuthors()) {
JsonObjectBuilder author = Json.createObjectBuilder();
String name = datasetAuthor.getName().getValue();
String affiliation = datasetAuthor.getAffiliation().getValue();
DatasetField authorAffiliation = datasetAuthor.getAffiliation();
String affiliation = null;
if (authorAffiliation != null) {
affiliation = datasetAuthor.getAffiliation().getValue();
}
// We are aware of "givenName" and "familyName" but instead of a person it might be an organization such as "Gallup Organization".
//author.add("@type", "Person");
author.add("name", name);
// We are aware that the following error is thrown by https://search.google.com/structured-data/testing-tool
// "The property affiliation is not recognized by Google for an object of type Thing."
// Someone at Google has said this is ok.
// This logic could be moved into the `if (authorAffiliation != null)` block above.
if (!StringUtil.isEmpty(affiliation)) {
author.add("affiliation", affiliation);
}
Expand Down Expand Up @@ -1341,7 +1345,11 @@ public String getJsonLd() {
if (TermsOfUseAndAccess.License.CC0.equals(terms.getLicense())) {
license.add("text", "CC0").add("url", "https://creativecommons.org/publicdomain/zero/1.0/");
} else {
license.add("text", terms.getTermsOfUse());
String termsOfUse = terms.getTermsOfUse();
// Terms of use can be null if you create the dataset with JSON.
if (termsOfUse != null) {
license.add("text", termsOfUse);
}
}

job.add("license",license);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -249,6 +249,18 @@ public boolean isSignupAllowed() {
return signupAllowed;
}

public boolean isRootDataverseThemeDisabled(Dataverse dataverse) {
if (dataverse == null) {
return false;
}
if (dataverse.getOwner() == null) {
// We're operating on the root dataverse.
return settingsWrapper.isRootDataverseThemeDisabled();
} else {
return false;
}
}

public String getSignupUrl(String loginRedirect) {
String nonNullDefaultIfKeyNotFound = "";
String signUpUrl = settingsWrapper.getValueForKey(SettingsServiceBean.Key.SignUpUrl, nonNullDefaultIfKeyNotFound);
Expand Down
4 changes: 4 additions & 0 deletions src/main/java/edu/harvard/iq/dataverse/SettingsWrapper.java
Original file line number Diff line number Diff line change
Expand Up @@ -155,5 +155,9 @@ public String getSupportTeamName() {
return BrandingUtil.getSupportTeamName(systemAddress, dataverseService.findRootDataverse().getName());
}

public boolean isRootDataverseThemeDisabled() {
return isTrueForKey(Key.DisableRootDataverseTheme, false);
}

}

Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,8 @@
@ViewScoped
@Named
public class ThemeWidgetFragment implements java.io.Serializable {
static final String DEFAULT_LOGO_BACKGROUND_COLOR = "F5F5F5";
static final String DEFAULT_BACKGROUND_COLOR = "F5F5F5";
static final String DEFAULT_LOGO_BACKGROUND_COLOR = "FFFFFF";
static final String DEFAULT_BACKGROUND_COLOR = "FFFFFF";
static final String DEFAULT_LINK_COLOR = "428BCA";
static final String DEFAULT_TEXT_COLOR = "888888";
private static final Logger logger = Logger.getLogger(ThemeWidgetFragment.class.getCanonicalName());
Expand Down
4 changes: 0 additions & 4 deletions src/main/java/edu/harvard/iq/dataverse/api/Dataverses.java
Original file line number Diff line number Diff line change
Expand Up @@ -97,10 +97,6 @@ public class Dataverses extends AbstractApiBean {
@Deprecated
private static final Logger LOGGER = Logger.getLogger(Dataverses.class.getName());
private static final Logger logger = Logger.getLogger(Dataverses.class.getCanonicalName());
// static final String DEFAULT_LOGO_BACKGROUND_COLOR = "F5F5F5";
// static final String DEFAULT_BACKGROUND_COLOR = "F5F5F5";
// static final String DEFAULT_LINK_COLOR = "428BCA";
// static final String DEFAULT_TEXT_COLOR = "888888";

@EJB
ExplicitGroupServiceBean explicitGroupSvc;
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
package edu.harvard.iq.dataverse.export;

import com.google.auto.service.AutoService;
import edu.harvard.iq.dataverse.DatasetVersion;
import edu.harvard.iq.dataverse.export.spi.Exporter;
import edu.harvard.iq.dataverse.util.BundleUtil;
import java.io.IOException;
import java.io.OutputStream;
import java.io.StringReader;
import java.util.logging.Logger;
import javax.json.Json;
import javax.json.JsonObject;
import javax.json.JsonReader;

@AutoService(Exporter.class)
public class SchemaDotOrgExporter implements Exporter {

private static final Logger logger = Logger.getLogger(SchemaDotOrgExporter.class.getCanonicalName());

public static final String NAME = "schema.org";

@Override
public void exportDataset(DatasetVersion version, JsonObject json, OutputStream outputStream) throws ExportException {
String jsonLdAsString = version.getJsonLd();
StringReader stringReader = new StringReader(jsonLdAsString);
JsonReader jsonReader = Json.createReader(stringReader);
JsonObject jsonLdJsonObject = jsonReader.readObject();
try {
outputStream.write(jsonLdJsonObject.toString().getBytes("UTF8"));
} catch (IOException ex) {
logger.info("IOException calling outputStream.write: " + ex);
}
try {
outputStream.flush();
} catch (IOException ex) {
logger.info("IOException calling outputStream.flush: " + ex);
}
}

@Override
public String getProviderName() {
return NAME;
}

@Override
public String getDisplayName() {
return BundleUtil.getStringFromBundle("dataset.exportBtn.itemLabel.schemaDotOrg");
}

@Override
public Boolean isXMLFormat() {
return false;
}

@Override
public Boolean isHarvestable() {
// Defer harvesting because the current effort was estimated as a "2": https://github.com/IQSS/dataverse/issues/3700
return false;
}

@Override
public Boolean isAvailableToUsers() {
return true;
}

@Override
public String getXMLNameSpace() throws ExportException {
throw new ExportException(SchemaDotOrgExporter.class.getSimpleName() + ": not an XML format.");
}

@Override
public String getXMLSchemaLocation() throws ExportException {
throw new ExportException(SchemaDotOrgExporter.class.getSimpleName() + ": not an XML format.");
}

@Override
public String getXMLSchemaVersion() throws ExportException {
throw new ExportException(SchemaDotOrgExporter.class.getSimpleName() + ": not an XML format.");
}

@Override
public void setParam(String name, Object value) {
// this exporter doesn't need/doesn't currently take any parameters
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -309,6 +309,12 @@ Whether Harvesting (OAI) service is enabled
// Option to override multiple guides with a single url
NavbarGuidesUrl,

/**
* The theme for the root dataverse can get in the way when you try make
* use of HeaderCustomizationFile and LogoCustomizationFile so this is a
* way to disable it.
*/
DisableRootDataverseTheme,
// Limit on how many guestbook entries to display on the guestbook-responses page:
GuestbookResponsesPageDisplayLimit,

Expand Down
10 changes: 6 additions & 4 deletions src/main/webapp/dataverse_header.xhtml
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@
<label class="sr-only" for="navbarsearch">#{bundle.search}</label>
<input id="navbarsearch" type="text" class="search-input ui-inputfield ui-inputtext ui-widget ui-state-default ui-corner-all" size="28" value="" placeholder="#{bundle['header.search.title']}"/>
</div>
<button type="submit" class="btn btn-default" onclick="window.location = '/dataverse/#{dataverseServiceBean.findRootDataverse().alias}&#38;q=' + document.getElementById('navbarsearch').value;
<button type="submit" class="btn btn-default" onclick="window.location = '/dataverse/#{dataverseServiceBean.findRootDataverse().alias}&#63;q=' + document.getElementById('navbarsearch').value;
return false;">
<span class="glyphicon glyphicon-search"/> #{bundle.find}
</button>
Expand Down Expand Up @@ -154,11 +154,13 @@
</div>
</nav>
<!-- Header Panel -->
<!-- FIXME: Rather than hard-coding "FFFFFF", get it from the backend (ThemeWidgetFragment.DEFAULT_BACKGROUND_COLOR). -->
<div id="dataverseHeader" class="container bg-muted"
style="background:##{!empty dataverse.dataverseTheme.backgroundColor ? dataverse.dataverseTheme.backgroundColor : 'F5F5F5'};"
jsf:rendered="#{showDataverseHeader and !widgetWrapper.widgetView}">
style="background:##{!empty dataverse.dataverseTheme.backgroundColor ? dataverse.dataverseTheme.backgroundColor : 'FFFFFF'};"
jsf:rendered="#{showDataverseHeader and !widgetWrapper.widgetView and !dataverseHeaderFragment.isRootDataverseThemeDisabled(dataverse)}">
<o:importFunctions type="org.apache.commons.lang.StringEscapeUtils"/>
<div class="dataverseHeaderLogo" style="text-align:#{!empty dataverse.dataverseTheme.logoAlignment ? dataverse.dataverseTheme.logoAlignment : 'center'};background:##{!empty dataverse.dataverseTheme.logoBackgroundColor ? dataverse.dataverseTheme.logoBackgroundColor : 'F5F5F5'};"
<!-- FIXME: Rather than hard-coding "FFFFFF", get it from the backend (ThemeWidgetFragment.DEFAULT_LOGO_BACKGROUND_COLOR). -->
<div class="dataverseHeaderLogo" style="text-align:#{!empty dataverse.dataverseTheme.logoAlignment ? dataverse.dataverseTheme.logoAlignment : 'center'};background:##{!empty dataverse.dataverseTheme.logoBackgroundColor ? dataverse.dataverseTheme.logoBackgroundColor : 'FFFFFF'};"
jsf:rendered="#{!empty dataverse.dataverseTheme.logo and dataverse.dataverseTheme.logoFormat == 'RECTANGLE'}">
<img src="/logos/#{dataverse.logoOwnerId}/#{dataverse.dataverseTheme.logo}" alt="#{dataverse.name}"/>
</div>
Expand Down
Loading

0 comments on commit 45288aa

Please sign in to comment.