Releases: GetDKAN/dkan
1.13
DKAN 1.13 Release notes
Read up on our latest release, 1.13! This is a huge release and we have new features and enhancements that will make using DKAN better than ever. Major new features include a Harvester, API improvements, new data previews, and DCAT-AP support.
Have questions or thoughts? Let us know on our public DKAN Repo in the issues queue or chat with us in our Gitter room.
What's new
DKAN Harvest
DKAN's new Harvest module gives DKAN administrators and site managers the ability to harvest datasets from other sources for their DKAN site. Built on the widely-used Drupal Migrate system, the Harvest module can fetch data and publish it to a DKAN site.
DKAN Harvest is fully configurable via the DKAN web interface. Harvest sources can be added like any other type of content, and data can be overridden or manipulated before being published. Migrations can be managed via an intuitive dashboard interface or triggered with a cron job.
Get more details on DKAN Harvest by reading its documentation pages.
DKAN Dataset REST API
Previous DKAN APIs have been read-only and focused on CKAN compatibility. A new RESTful API for DKAN datasets provides an endpoint that allows other applications to interact with DKAN, including creating and modifying datasets and resources.
Get more details on the Dataset REST API.
DCAT-AP Compliance
DKAN's DCAT RDF publishing has been reengineered to be more consistent with its other open-data standards implementation, and, more importantly, to implement the new DCAT-AP protocol. DKAN's fields can be easily mapped to DCAT-AP fields using the Open Data Schema Mapper. Now, DKAN meets US standards (Project Open Data) as well as European Union standards (DCAT-AP).
New Default Content
On a basic install of DKAN, the site came with only a few example Datasets and Resources. We added more default content in order to demonstrate the full capabilities on DKAN and give new users a better experience in testing out various DKAN features. The default content module is part of core DKAN, so all new installs of DKAN contain the content. Simply disabling the "DKAN Default Content" module will remove all the default content to give you a blank slate to start building.
What we've improved
The functionality isn't new, but it's working even better than before. These improvements make DKAN better on the whole including user experience, efficiency, and scale.
Site Managers can assign roles
The Site Manager role is part of a core group of user roles on DKAN. Site Managers have a high degree of control over the entire site, but are expected to be less technical than a full admin. Previously, it wasn't possible for Site Managers to assign roles to new users. By adding the RoleAssign module we have provided this more fine-grained permission.
Site Managers can add pages
One of the most basic content types on DKAN is a page. And though the content type is straightforward it can have implications for the structure and appearance of the site. Originally, Site Managers were not able to add or manage Pages; in DKAN 1.13 Site Managers now have this permission.
Even more permissions for Site Managers
- Theme Settings: Includes page elements and ability to add custom logo and favicon from "Site Configuration > Theme Settings" at /admin/appearance/settings
- Colorizer: Ability to create custom color schemes for a DKAN site from "Site Configuration > Colorizer" at /admin/appearance/colorizer
- Open Data Schema Mapper (ODSM): Ability to add, edit, and delete APIs and their mappings to DKAN fields from "Site Configuration > Open Data Schema Mapper" at /admin/config/services/odsm
- Menus: Permission to manage the main menu links by adding, editing, and deleting links from "Site Configuration > Menu" at /admin/structure/menu
- Topics Icons: When adding or editing terms in the Topics taxonomy at path /admin/structure/taxonomy/dkan_topics, site managers can choose an "Icon Type" of either "Font Icon" or "Image." If they choose "Font Icon," the edit form displays a list of available icons from which to choose. If they choose "Image," the form lets them upload an image.
- Enabling External Previews: Site Managers can enable previews so that site visitors can look at Resource contents with visualization tools, Carto and ArcGIS. To enable External Previews, go "DKAN > Data Previews" at /admin/dkan/dataset_preview
- DCAT and POD validation: Site Managers can select the settings for how Datasets are validated against Project Open Data and DCAT-AP standards.
- Site managers can confirm that the website's data.json feed is working correctly by using the POD online validator at https://labs.data.gov/dashboard/validate
- Site managers can confirm that the website's DCAT feed is working correctly from Site Configuration > Open Data Schema Mapper > DCAT Validation at /admin/config/services/odsm/validate/dcat or by using the DCAT online validator at http://www.dcat.be/validator/
- Order of Groups if using the featured groups block: Site Managers can arrange the order in which Groups appear in the featured group block from "DKAN > Featured Groups Sort Order" at /admin/dkan/featured-groups-sort-order
- Recline size configuration: Manage size constraints for Recline, which powers internal previews, from "DKAN > Recline Configuration" at /admin/dkan/recline
Open Data Federal Extras (ODFE) is now part of core DKAN
Project Open Data (POD) sets a standard for the information about datasets (metadata) included when the data is published. DKAN collects metadata with fields on the Dataset form when the data is published. For most agencies, the fields reflect basic requirements. However US federal agencies are required to provide additional information about the data published. The extra fields appear as part of the Dataset form with our ODFE module.
We've moved this module into core DKAN so that it's part of every install, though not enabled by default. When enabled, the ODFE module makes federal agencies compliant with POD standards.
Get more details on Federal Extras on DKAN.
In-place Editor for Site Managers + look and feel
Previously, the In-place Editor (IPE, an interface for changing page layouts) for Site Managers included several options that weren't appropriate for their level of permissions. This UI was confusing, so we improved the interface to only include options that are relevant to non-admin user roles. We also improved the overall look and feel of the IPE to be more user-friendly.
Improved Data Previews
The Data Previews feature is designed to let site visitors take a sneak peek at the content of a Resource before downloading the file. In this release, we've improved Data Previews to support more file formats beyond CSV and XLS. Now previews also support JSON, geojson, XML, ArcGIS REST, WMS, images, PDF, and ZIP data formats.
Improvements to Data Previews also offer better support for Resources that are hosted remotely. Previously, Resources that were linked to a web source could only be previewed if they were first imported into the DKAN Datastore (which only supported CSV files). In DKAN 1.13, linked Resources can be previewed with the Data Previews feature.
Improved Datastore API
The DKAN Datastore API makes it possible to query for contents of Resources uploaded to the Datastore as detailed as a cell within a table. The improved Datastore API is enhanced to open greater possibilities of requesting complex queries from contents within a Datastore on a DKAN site as well as multiple queries in a single request.
Get more details on the Datastore API
New Dataset fields added
A number of metadata fields that are common requirements for open data standards like Project Open Data and DCAT-AP have been added to the basic dataset content type in DKAN. These fields are:
- Data Standard
- Data Dictionary Type
- Homepage URL
- Data Standard (conformsTo)
- Rights
- Language
The following fields are also included but hidden. This allows the harvester to ingest metadata with these fields and present them in data.json but they are not yet supported in the UI for locally-created datasets:
See the Project Open Data schema for more information on these fields.
NuBoot Radix now in core DKAN
Previously, the NuBoot Radix theme on DKAN was not part of the core repository. The old Omega-based theme and its tools (the Delta module) have been removed, and NuBoot Radix is no longer maintained in a separate repository. Legacy themes will not be maintained or shipped with DKAN and require manual installation if you want to use the Omega theme. Alternatively, you can make customizations to the default theme.
Additionally, the admin theme setting is set to use the default theme rather than the NuBoot Radix theme. Because the Colorizer tool requires both the admin and default to be the same, setting the admin theme setting to use the default theme enables Colorizer to work normally for both the NuBoot subtheme and other custom subthemes without admins needing to manually override this setting.
Added POD-based...
7.x-1.12.13
This is a "patch" release of DKAN, containing bug fixes and minor updates to DKAN 7.x-1.12, but adding no new functionality. Upgrading should be straightforward.
Improvements in this release
- Fixed broken recline (resource CSV) preview embeds caused by how recline module loads bootstrap.
- Added patch to remote_stream_wrapper to avoid memory exhausted errors on big files.
- Fixed publisher token in open_data_schema_map_dkan. It was showing only URL rather than publisher name.
- Remote linked files are now proxied through a DKAN site so that we don't run into CORS issues for the visualizations. That can cause memory issues with large files. This only proxies files smaller than 50MB.
7.x-1.12.12
This is a "patch" release of DKAN, containing bug fixes and minor updates to DKAN 7.x-1.12, but adding no new functionality. Upgrading should be straightforward.
Group assignment cleanup update
This release contains a database update that may take some time to complete on larger sites. As described in #1267, on older DKAN sites it was very easy to add resources to datasets that would not be added to the parent dataset's group.
Many sites may have this issue and not even realize, but it will create problems for organizations that use DKAN's out-of-box group roles and permissions; it give users permission to edit content in their own group but not content outside of it. If a resource is not assigned to a group, this could block a group user from editing their groups' own datasets.
This patch release contains a database update that will identify datasets with resources missing their group assignments, and correct them. See pull request #1491 for the fix itself.
Other improvements in this release
- Data previews can be captured in a certain state for embedding into other web pages as an iframe, but these previews were not cached by Drupal. This has been fixed; embedded previews are now cached, which is a significant performance boost for some websites.
- Upgraded Drupal core to 7.52, ctools to 1.11, and Media to 2.0-beta13.
- Fixed links to group pages from group node teasers that broke when site has clean urls disabled
- Fixed some issues with certain filters/facets on the search page being unexpectedly collapsed.
- Fixed a bug introduced on a previous patch release that hid the body field on page nodes.
- Improved the access check and security on datastore pages - unauthorized users could perform certain datastore functions by guessing URLs.
- Fixed "page not found" errors when clicking certain topics links with spaces or special characters in them.
7.x-1.12.11
This is a "patch" release of DKAN, containing bug fixes and minor updates to DKAN 7.x-1.12, but adding no new functionality. Upgrading should be straightforward.
Improvements in this release
- Fixed a bug in Recline regarding file objects and node forms that caused errors when using the "view changes" button on Dataset or Resource edit form
- Center group images in "Groups" page and group node page sidebar
- Add "2x," "3x" etc to Dataset teasers when more than one resource of a particular format present.
- Update the default jquery library setting from 1.7 to 1.10
- Fixed topics menu and facet links if special characters are used in topic terms.
- Fixed dataset form redirect when validation fails, was sending user to node/add/dataset rather than node/%/edit
- Patch fontyourface to remove from the "standard text" selector, to prevent unpredictable results from this option
- Fixed issue in open_data_schema_map with "Data Dictionary" field not displaying URL in data.json file
7.x-1.12.10
This is a "patch" release of DKAN, containing bug fixes and minor updates to DKAN 7.x-1.12, but adding no new functionality. Upgrading should be straightforward.
Improvements in this release
- Fixes an issue causing JavaScript errors in IE browsers, preventing recline previews from displaying. Bug was introduced in 7.x-1.12.7.
- Provides a better upgrade path for Markdown text format and bueditor improvements added in #1085
7.x-1.12.9
This is a "patch" release of DKAN, containing bug fixes and minor updates to DKAN 7.x-1.12, but adding no new functionality. Upgrading should be straightforward.
Improvements in this release
- Fix to a bug introduced in the Recline module in 1.12.7. Previews were not displaying if the resource file was smaller than max preview size, even if loading the resource from datastore.
- Fix a problem with paths for image icons for topics. Uploaded image icons will now display correctly in the Topics menu.
- Fixed the "data dictionary" field to use HTML input format. This field, at least in U.S. Project Open Data, is really intended to have only URL values, and using HTML ensures that links will be clickable. Also made improvements to the field's display and help text.
- Updated the DKAN Workflow content type legend to include all content types with correct icons. This is mostly to prepare for future content types' inclusion in Workflow but also to support sites that already have additional content types in Workflow.
7.x-1.12.8
Special note: This release contains security updates and should be applied immediately.
This is a "patch" release of DKAN, containing bug fixes and minor updates to DKAN 7.x-1.12, but adding no new functionality. Upgrading should be straightforward.
Improvements in this release
- Updates the Panels and Panelizer modules following critical security releases. See Drupal security advisories DRUPAL-SA-CONTRIB-2016-047 and DRUPAL-SA-CONTRIB-2014-048.
- Added validation to groups node form to prevent user from creating duplicate groups. Saving a group will now fail if there is an existing group with the same name.
7.x-1.12.7
This is a "patch" release of DKAN, containing bug fixes and minor updates to DKAN 7.x-1.12, but adding no new functionality. Upgrading should be straightforward.
Improvements in this release
- Updates contrib modules (uuid, services, manualcrop, markdown, panelizer, panopoly_widgets and panopoly_images).
- Fixes a bug that produced a "Cannot use a scalar value" error when users trying to add a content pane to a data story node in panels.
- Changes theming on resources to remove iframe to "API or link URL" if remote file field also populated. The only real use-case this applies to is when harvesting datasets from a data.json that includes both an accessURL and a downloadURL for a single distribution; both these fields should not be used at once on resouces created using the node form.
- Updates the options in the Dataset "frequency" field to reflect ISO 8601 standards for DCAT and Project Open Data
- Provides a simple file proxy to serve remote CSVs through local domain and resolve cross-origin issues with previews n these resources
- Somes changes to the Resource node form to improve the UX of links and file attachments, especially by replacing the label "Link to an API" to "API or Website URL."
- Fix bug on relative paths for links in theme template files, which caused some links to break when DKAN lives at a subdirectory rather than a website's domain root.
- Fixes a minor typo in the DKAN topics menu link that was exposed by the XSS fix in the 7.x-1.12.6 release.
7.x-1.12.6
Special note: This release contains security updates and should be applied immediately.
This is a "patch" release of DKAN, containing bug fixes and minor updates to DKAN 7.x-1.12, but adding no new functionality. Upgrading should be straightforward.
Improvements in this release
- Adds validation to search parameters. Without this, text can be passed into search parameters and passed unsanitized to the browser. The way the facets are themed allowed Javascript to be injected into the attributes of any facet items with icons, (such as content types). The fix validates all facet input and returns a "page not found" error if a search parameter passed in the URL arguments does not match available options. See #1271
- Starts sanitizing output to the facets on the search page, as an additional safeguard against malicious input from the search parameters.
- Adds icons to the topics drop-down menu rather than just the title of the topic.
- Added a new module called Role Assign, which gives site managers the ability to assign roles to other users without giving them access to the entire Permissions module, which meant previously only admins could assign roles.
- Made links and emails in dataset metadata clickable in certain places that they hadn't been for a better user experience.
7.x-1.12.5
This is a "patch" release of DKAN, containing bug fixes and minor updates to DKAN 7.x-1.12, but adding no new functionality. Upgrading should be straightforward.
This release contains critical security updates and should be applied immediately.
Improvements in this release
- Updates the RESTFull Web Services module to 7.x-2.6, fixing a highly critical security issue announced by the Drupal Security team on July 13.
- Updates the version of recline.js used in the Recline module to point to a new source for map tiles. Previously map tiles were served by MapQuest, but a change in policy has broken map previews on all applications using recline.js. See the original PR on recline.js: datopian/portaljs#501
- Updates Drupal core to version 7.50.
Special Note
This is a fairly big update to Drupal and you may want to familiarize yourself with it by reading the release announcement.
Of note to DKAN users is that "Administer fields" is now a separate permission. This means, for instance that site managers and editors, who by default have "administer taxonomy vocabularies" permissions, will no longer be able to add, remove, reorder or otherwise modify fields in a taxonomy vocabulary.
Only administrators will be able to do administer fields. We see this as a welcome improvement, as making minor changes to, say, the topics vocabulary should not require giving a site manager permission to alter the entire data architecture.