Determine implications of OGC API - Common - Part 2

[ghobona]

The SWG today (2020-05-27) discussed whether the development of OGC API - Common - Part 2: Collections might have implications for OGC API - Coverages. The SWG agreed to revise the document, if possible, to be consistent with OGC API - Common - Part 2: Collections.

[ghobona]

The SWG on 2020-05-27 agreed that @jerstlouis would review the specifications and estimate the changes needed and how long they might take to apply to the document.

@jerstlouis will report back to the SWG on 2020-06-03.

[ghobona]

Cc: @cmheazel @Schpidi

[jerstlouis]

An important on-going discussion has been happening in Common ( opengeospatial/ogcapi-common#140 ), and it seems like we are finally converging towards solutions to settle the big Collections issue.

1. Relation type
   So far, one important change is that the "items" relation type would not be appropriate for coverage. The exact meaning of an "items" relation type is still being discussed (e.g. whether it implies that the link follows the /items semantics of the Features API), but it seems now that the relation type would be the primary mechanism by which a client recognizes the access method provided by a link. This enables supporting more than one access method for a single collection (e.g. the same data available through SensorThings, Features and Coverage), while identifying which link provides which access method. Therefore I suggest to change this relation type from "items" to "coverage".

2. Extensible /collections/{collectionId} resource
   It has also been clarified that by default, a JSON schema is extensible unless "additionalProperties" is explicitly set to false. So it would already be perfectly okay to add additional information directly at /collections/{collectionId}, and part of my suggestion was that some coverage resources which are not going to be overly lengthy could be provided as properties of the /collections/{collectionID} resource. I believe this could generally include the DomainSet, RangeType and Metadata. As the Metadata seems to be currently described as application-specific, perhaps this could be a general Common concept rather than coverage-specific (and preferably as a linked resource). For the DomainSet, it might be that non-gridded coverages require a more verbose description. In this case, perhaps it could be specified that as an alternative to fully describe the whole DomainSet, a link to a DomainSet resource could be provided instead. What is currently being returned from /coverage would nicely merge into that {collectionID} resource. The envelope could be an extra property with more complex coverage-semantics which the {collectionID}'s spatial and geospatial extent do not support describing adequately.

3. /coverage, /coverage/description, /coverage/all and /coverage/rangeset
   Currently, /coverage/all is indicated as returning "everything", i.e. both the coverage description and the coverage data itself (the RangeSet). One of my concerns with this is that some formats (media types) may either:

   • not support specifying the data alone without the description (netCDF? GeoTIFF?)
   • while other RangeSet formats may not support specifying the description at all

   To keep things simple, I would suggest that:

   • /collections/{collectionId} embeds the description as properties, except when the DomainSet is too bulky in which case the domainSet property could be omitted, and a link with "rel" : "domainSet" could be included instead linking to /coverage/domainset.
   • /collections/{collectionId}/coverage returns the data (RangeSet), and if the selected media type defines a mechanism for it, the description components as well.
   • /coverage/rangeset could still optionally be supported to return only the data without the description for formats that support this.

4. Formats recommendations
   Like Features and Tiles, no specific media types is mandated by Coverage. This is good, but Features and Tiles make some recommendations on specific formats that are commonly implemented. I could not find any such recommendation in the current draft specs. I would suggest including at least mentions of CIS JSON, CIS RDF, GeoTIFF, netCDF and CoverageJSON.

5. Missing example responses
   None of the examples responses at https://github.com/opengeospatial/ogc_api_coverages/tree/master/standard/examples/JSON seem to have any proper content. I think the lack of proper examples is quite hindering for developers wanting to implement clients & services. Some of the examples likely require selecting a specific media type, and text-based formats are probably better suited for example responses (e.g. CIS JSON).

[tomkralidis]

Thanks @jerstlouis for summarizing this important issue. I have chimed in a bit on opengeospatial/ogcapi-common#140 as much as I can as time permits, so thanks very much for working on this one and reporting back. Thoughts related to OACov:

• relation type could be ogc-coverage (see related queryables link relation type ogcapi-features#383)
• +1 to recommend CoverageJSON, GeoTIFF, NetCDF as media types
• we need some effort to start adjusted the various schemas we have and come up with examples. I'm happy to sprint on this with others. Would also suggest a once over on the spec and the various JSON/OpenAPI/yaml (even renaming this repo to ogcapi-coverages for consistency with the other standards here on GitHub)

[jerstlouis]

@tomkralidis Thanks!

Would also suggest a once over on the spec and the various JSON/OpenAPI/yaml (even renaming this repo to ogcapi-coverages for consistency with the other standards here on GitHub)

This has also been bugging me a lot. Currently there are all sorts of naming for the repos and it makes them hard to find, especially when switching between them:

• ogcapi-features
• ogcapi-records
• oapi_common
• ogc_api_coverages
• wps-rest-binding
• OGC-API-Tiles
• OGC-API-Maps

It would be nice to standardize this @ghobona .

[tomkralidis]

Note we changed ogcapi-records at some point for the consistency.

[pvretano]

@jerstlouis don't forget ogcapi-records! ;)

[jerstlouis]

@pvretano well at least that is consistent with Features ;)

[chris-little]

And EDR API has not yet got a new branding (OGC API - Environmental Data Retrieval?)

[jerstlouis]

To summarize points 2 & 3 of the proposal:

• Merge current "coverage offering" ({collectionID}/coverage) directly into {collectionID}, e.g. envelope property, and links and/or properties for the coverage description
• Get rid of {collectionID}/coverage/description (integrated in {collectionID})
• Get rid of {collectionID}/coverage/all (now returned by /coverage)
• Get rid of {collectionID}/coverage/rangetype (always a property in {collectionID}, following CIS JSON encoding)
• Make {collectionID}/coverage/domainset optional (Can either be a property in {collectionID}, or a link in the case of complex domainset, which may also support subsetting) -- NOTE: Either a property or a link is still required, and always encoded as CIS JSON, though in the case of a link it could additionally be provided in other formats.
• Make {collectionID}/coverage/rangeset optional (only present when format supports providing the rangeset without the description, i.e. a raw naked set of values)
• Make {collectionID}/coverage/metadata optional, linked from {collectionID}

The advantages I see from this is that while it preserves full compatibility with the CIS model, it greatly reduces the entry barrier for non-coverage experts, and facilitates a simple implementation for clients & services, including serving simple coverage data by a static server.

e.g. only two resources are required for a simple one-file gridded coverage GeoTIFF or netCDF:

• {collectionID} (which will feature rangetype and domainset properties, as well as optional link to metadata)
• {collectionID}/coverage (which will contain the rangeset along with the description supported by the format) -- NOTE: Like items the path should not be fixed, so this could be e.g. /coverage?f=cdf or /coverage.cdf, following whatever the link (based on rel and type) points to.

(This also happens to nicely mirror Features with {collectionID} and {collectionID}/items)

[jerstlouis]

This ( opengeospatial/ogcapi-common#140 (comment) ) is the comment from the big Collections discussion in Common which is stirring us towards using more specific relation types, and why I was suggesting "coverage" rather than "items" for the {collectionID}/coverage resource.

[pebau]

hm, I see the long and involved description, and the different handling in different branches - now I fail to see how I just can get a coverage. How would I do the equivalent of a vanilla GetCoverage, even without subsetting?

[jerstlouis]

@pebau {collectionID}/coverage

{collectionID}/coverage/rangeset , if available, in the case the format (media type) supports both including or omitting the description, and you want to omit the description (but this is more useful together with subsetting, to avoid the description overhead)

And {collectionID} is much the equivalent of DescribeCoverage.
A coverage with a complex DomainSet requires an extra fetch of {collectionID}/coverage/domainset, linked from {collectionID}.

[pebau]

@tomkralidis: just a small fix concerning

+1 to recommend CoverageJSON, GeoTIFF, NetCDF as media types

CIS is the standardized JSON encoding, so first candidate. We might also add CIS's RDF, BTW.
CoverageJSON is not compatible with CIS.

[jerstlouis]

@pebau Would you have a link to CIS JSON encoding?
(Is this it: https://docs.opengeospatial.org/is/09-146r6/09-146r6.html#46 ?)

Googling for "cis json encoding", first hit is for CoverageJSON :/ ( https://www.w3.org/TR/covjson-overview/ ).
I don't see any issue with adding any formats which existing/initial implementations will support.

So I would suggest we use extracts from the CIS standardized JSON encoding for the properties in {collectionID}, as well as for the optionally linked /coverage/domainset.

CIS standard JSON could also be the format of /coverage and /coverage/rangeset if a link is included identified by media type simply "application/json".

[pebau]

@jerstlouis:

Googling for "cis json encoding", first hit is for CoverageJSON :/ ( https://www.w3.org/TR/covjson-overview/ ).

duckduckgoin for "feature" I get a high-ranked

Sneakers, high-end apparel and accessories - Feature
[Search domain feature.com] https://feature.com
Feature was created as a venue where fashion, music and art could co-exist. Since then, it has been our mission to provide the highest quality in footwear, apparel and accessories that street wear and high end fashion culture have to offer.

I don't see any issue with adding any formats which existing/initial implementations will support.

If compatible, yes. And, again, IMHO there are further interesting candidates, such as RDF.

BTW, as I often have expressed I am not against having a compatible CoverageJSON encoding as a CIS encoding. It just is not there yet, whereas OGC coverage JSON is standard, compatible, and therefore makes sense as primary choice.

[pebau]

...just for the sake of avoiding misinterpretations: please read previous search results as tongue-in-cheek!

[pebau]

sorry,today is my day of delivering slicewise:

@pebau Would you have a link to CIS JSON encoding?
(Is this it: https://docs.opengeospatial.org/is/09-146r6/09-146r6.html#46 ?)

yes, exactly - conformance class json-coverage. (and to give proper credit: done by Joan Maso)
Examples you find in http://schemas.opengis.net/cis/1.1/json/examples/

[jerstlouis]

@pebau Well I always complain about the term feature, especially when used meaning vector data and unqualified as such ;) Still, it would be nice to do some SEO to improve search results on CIS JSON :)

I added CIS JSON and CIS RDF at the head of the list of formats to recommend.
Even if a coverage representation is deemed "incompatible" with CIS, I think we still need to acknowledge the possibility that implementations may be supporting those formats (e.g. @tomkralidis 's implementation was one of the first for OGC API - Coverage, at the coverage & analytics code sprint, and CoverageJSON is the one encoding that he decided to implement (unless I'm mistaken and made some wrong assumptions?). Generally speaking CoverageJSON seems to be quite popular.).

If it is possible to convert between CoverageJSON and a CIS-compliant encoding, is "not compatible" really accurate?

[jerstlouis]

From CoverageJSON (https://www.w3.org/TR/covjson-overview/#cis):

The main points of difference are:

• CIS uses a different set of rules for gridded and non-gridded data, whereas CoverageJSON uses a single consistent set.
• CIS allows each Coverage to have exactly one CRS, whereas CoverageJSON allows different CRSs to be applied to different sets of coordinates in the domain (e.g. one CRS for x and y, and another CRS for z).
• Version 1.1 of CIS defines a JSON encoding that uses a near-literal translation of GML structures into JSON. CoverageJSON does not use GML as its starting point.

I think it will be necessary to accommodate people in both camps :)

[pebau]

@pebau Well I always complain about the term feature, especially when used meaning vector data and unqualified as such ;) Still, it would be nice to do some SEO to improve search results on CIS JSON :)

right, calls for OGC's marketing office ;-)

I added CIS JSON and CIS RDF at the head of the list of formats to recommend.

thanks, appreciated!

Even if a coverage representation is deemed "incompatible" with CIS, I think we still need to acknowledge the possibility that implementations may be supporting those formats (e.g. @tomkralidis 's implementation was one of the first for OGC API - Coverage, at the coverage & analytics code sprint, and CoverageJSON is the one encoding that he decided to implement (unless I'm mistaken and made some wrong assumptions?). Generally speaking CoverageJSON seems to be quite popular.).

well...I could bore you with a list of agencies, businesses, research centers, data providers using CIS JSON, with 2-digit Petabytes of offerings. But me engineer would rather go for the technical merits :-)

If it is possible to convert between CoverageJSON and a CIS-compliant encoding, is "not compatible" really accurate?

definitely that would change it: if both can be converted into each other in both directions without loss of information and with all the coverage types, we have a win.

FWIW, earlier we have worked with encoding writers wanting to bring in their formats into coverage world, such as NetCDF and GRIB2 and JPEG2000, helping to get this correspondence established.

[pebau]

@jerstlouis: thanks for sharing - interesting, so that is their view - let's inspect:

From CoverageJSON (https://www.w3.org/TR/covjson-overview/#cis):

The main points of difference are:

* CIS uses a different set of rules for gridded and non-gridded data, whereas CoverageJSON uses a single consistent set.

actually, CoverageJSON only describes grids, and even only regular grids, so a small subset of CIS JSON. (Note: a bit hard to discover as they have only JSON examples, could not find a comprehensive specification.)

* CIS allows each Coverage to have exactly one CRS, whereas CoverageJSON allows different CRSs to be applied to different sets of coordinates in the domain (e.g. one CRS for x and y, and another CRS for z).

in CIS, this CRS is the common bracket, composed from, say, EPSG for x/y, something else for z, something else for t - so same principle, but one common, coherent mechanism (information not distributed over several places.

* Version 1.1 of CIS defines a JSON encoding that uses a near-literal translation of GML structures into JSON. CoverageJSON does not use GML as its starting point.

so....? but actually it does not start from GML, but from UML,. CIS addresses (and integrates) a far larger ecosystem than CoverageJSON, not just JSON.

I think it will be necessary to accommodate people in both camps :)

so we talk politics now ;-)

[jerstlouis]

@pebau

Thanks for sharing your thoughts on that comparison. I will definitely need to dig into these further at some point.

so we talk politics now ;-)

No, we explicitly try to stay away from politics :)
(I myself don't belong to either camps, and would be inclined to implement both in our software).

This is partly why I think the OGC API specifications do not mandate a specific media type, recognizing the different needs and preferences of different communities.

And the recommendation is just to provide a sample of commonly used coverage formats, so I think it would be a fair compromise to include both CIS JSON and CoverageJSON in there.

[pebau]

@jerstlouis 👍

This is partly why I think the OGC API specifications do not mandate a specific media type,
recognizing the different needs and preferences of different communities.

couldn't agree more! In the WCS.SWG we had years (literally!) of discussion on "the" default format, every incoming community brought their own and relaunched discussion. In the end we had to give up and accept that "no one size fits all".

And the recommendation is just to provide a sample of commonly used coverage formats, so I think it would be a fair compromise to include both CIS JSON and CoverageJSON in there.

As said, no objection, as soon as CoverageJSON is compatible in the sense you defined it: translating back& forth retains the coverage.

[chris-little]

@jerstlouis There is a CoverageJSON standardisation effort, hosted in the WCS SWG, to be started soon. The one technical change that may help compatibility with CIS is the evolution of JSON-LD from 1.0 to 1.1, so there is some work to do besides just editorially producing an OGC standard.

The WCS SWG agreed to host the work last year, rather than have a separate, format only, SWG established. It was put back on the WCS agenda a month of so ago. As you say, there are plenty of implementations, and the user community is supportive of an OGC standardisation effort. I just need to get some money and contractual stuff sorted.

There is even a moribund GitHub repo in waiting: https://github.com/opengeospatial/CoverageJSON

[ghobona]

All,

• OGC API - Coverages - Part 1: Core is due to be completed by the end of 2020.
• CIS is well established within OGC and it has a JSON encoding.
• Reading the current version of the OGC API - Coverages spec, the SWG had already selected CIS to work with.

Those are multiple reasons to proceed with CIS as mandatory. CoverageJSON can be optional.
We need to aim to complete OGC API - Coverages - Part 1: Core soon, so that it can be sent to the OAB and the Public for Comment.

[jerstlouis]

@ghobona In Features, no encoding is mandatory.
Was there a decision to have a mandatory encoding in OGC API - Coverage?
The data model behind an OGC API - Coverage can be assumed to be CIS without mandating any particular encoding.

I do see:

The OGC JSON Schema for CIS standard addresses that need by defining a JSON schema for the CIS standard. This format should be used for all JSON encodings from the {root}/collections/{coverageid}/coverage/* paths

currently in the specs.

[ghobona]

... and there is clause_8_media_types.adoc which says "This API-Coverages standard is built around the OGC Coverage Implementation Schema (CIS). CIS content often includes multi-dimensional coordinates and coordinate reference systems in sensor and analytic space. "

[jerstlouis]

@ghobona The way I read the specs right now:

• Support for JSON is recommended.
• This [CIS JSON] format should be used for all JSON encodings from the {root}/collections/{coverageid}/coverage/* paths.

This would mean (or might be interpreted in this way) that support for CIS JSON is not mandatory, but recommended (and I think that's a good thing).
The fact that it mentions all JSON encodings seems in direct conflict with the ability to support additional JSON-based formats, but is not strong enough either (should rather than shall) to ensure that application/json media type implies CIS JSON.

So I would like to suggest that:

• As per this proposal to update the resources, it is clarified that in {collectionId} properties (rangetype, domainset) and alternate linked resource for domainset, CIS JSON is what must be used.
• For the coverage itself (this proposal suggests to move this to /coverage, with optional /coverage/rangeset for rangeset only), it is strongly recommended (SHOULD) that the API supports at least CIS JSON
• If the media type "application/json" is used for those resources, this implies (SHALL be) CIS JSON.
• Alternative JSON encodings such as CoverageJSON can be used, but must be identified by a more specific media type (e.g. something like application/json+covjson).
• The references to GeoJSON are very confusing, as GeoJSON is not really used anywhere, and should be moved to a more informal section, rather than come before mentioning the CIS JSON the implementer is ideally expected to be implementing.

[Schpidi]

20200902: Coverages SWG call: Embedding vs. referencing of DomainSet and RangeType. It was decided to recommend to Common to explicitly allow relative links in the same document in the href element like "href": "#domainSet". This way we would make the link mandatory which could be a relative link in the same document.

[jerstlouis]

PR #87 should address the last remaining aspects of this issues.

[Schpidi]

20200909: Coverages SWG call: PR #87 was merged and it was agreed to close this issue.