ISO 19115 metadata is converted to lossy ISO 19139 format when ingested by CKAN

[adborden]

Geospatial datasets in ISO 19115 lose metadata when CKAN harvests them and converts them to ISO 19139.

How to reproduce

Expected behavior

Actual behavior

[adborden]

Would be good to grab a specific example and list out the metadata fields that are lost.

[adborden]

Some background from @JJediny

Thread on FGDC call https://github.com/ckan/ckanext-spatial/tree/master/ckanext/spatial/validation/xml

johnjediny 3 hours ago
current ckan support for xml harvesting metadata

johnjediny 3 hours ago
think the current issue is how we’re handling our harvest sources transformations into ckan tables that that common schema we conform to should be updated to 19115-3 as a new planned common core that would be compatible with pyCSW without having to risk losing fields in our current transformation.

johnjediny 3 hours ago
im not so up to speed on the best ways to go about approaching that re flask/spatial-ext/scheming-ext

johnjediny 3 hours ago
if they want us to temporarily update the xlst(s) we current install with the current java transformation that might be the least-work fix

johnjediny 3 hours ago
Here is the 19115 v3 schema project https://github.com/ISO-TC211
ISO/TC 211
Repositories
27
@ISO-TC211 | May 26th, 2014 | Added by GitHub

johnjediny 3 hours ago
artic metadata project re existing mapping work https://github.com/adiwg/mdJson-schemas
adiwg/mdJson-schemas
JSON schemas, examples, and templates for ADIwg metadata standards
Website
http://www.adiwg.org/projects/
Stars
7
adiwg/mdJson-schemas | Feb 18th, 2014 | Added by GitHub

johnjediny 3 hours ago
This was created by Micah.Wengren@noaa.gov https://docs.google.com/spreadsheets/u/1/d/19L89sgSijpB9nvaWjgGTSo3m3r06zuXKq9C5QwcbgRs/edit

[JJediny]

Best long term solution may be to convert to using 19115-3 as CKAN's core schema. Which could be accomplished using https://github.com/ckan/ckanext-scheming w/ JSON schemas that have already been prepared for the 19115-3 FGDC profile - https://github.com/adiwg/mdJson-schemas/tree/master/schema as there is a good amount of work done mapping DCAT to ISO 19115 it might be easier to map data.json into 19115 - https://www.w3.org/2015/spatial/wiki/ISO_19115_-_DCAT_-_Schema.org_mapping

[nickumia-reisys]

Is this still relevant? @GSA/data-gov-team (specifically @FuhuXia and @jbrown-xentity)

[jbrown-xentity]

This is still relevant. Related to the transformation from CSDGM (FGDC standard) into ISO, done in ckanext-geodatagov. Hopefully this will be addressed with the https://github.com/adiwg/mdTranslator tool, but still TBD.

[hkdctol]

@jbrown-xentity @nickumia-reisys I will put this back in icebox then?

[nickumia-reisys]

I had brought this back up because of the harvesting requirement derivation/documentation. I wasn't sure if this was still a consideration we needed to address. Since it is, I'll pull the relevant aspects and we can move it back to the icebox.

[gujral-rei]

@jbrown-xentity will we address this with #4639 and #4564 ?

[jbrown-xentity]

@jbrown-xentity will we address this with #4639 and #4564 ?

That is the hope. However, I always believed this "error" was actually just a bad UI representation of the data; all of the source data was accessible "as is" without transform, as well as the CKAN view.