filter on associations (links) in CQL

[cportele]

As discussed in the 2020-03-04 meeting:

The queryables currently include links to related resources, but the CQL grammar does not support this or at least it has no explicit mechanism how property names should map to links. Either we just need additional guidance how this is represented in the queryables or we need an extension to CQL.

The predicates should probably be restricted to the information included in the link (in particular: title, rel, href).

As CQL has no mechanisms how to process properties with multiple values we probably also need to clarify the requirements how the predicates on links work for the standard case that there are multiple links.

[cportele]

Proposal for the extension to CQL

We need the following capabilities:

• For queryables that are objects we need a capability to reference a sub-property. The proposal is to use a dot as a separator consistent with the <compund attribute name> CQL rule of CSW 3.0.
• For queryables that are arrays we need a rule how the expression is evaluated since the property has multiple values, not just one. I don't think that CQL has specified the rule for such cases. In Filter Encoding 2.0 the default is to evaluate to true, if at least one of the values matches the expression. The proposal is to use this here, too.
• An additional capability would be beneficial to specify a sub-query. The proposal is to inject the sub-query expression in square brackets at the end of a segment of a compound property name.

See the examples below.

In addition, we need to clarify how links / associations are encoded in the JSON encoding.

The general links of a resource that are specified in the OGC API standards are in a top level links member of the JSON encoding with a straightforward encoding of RFC 8288. For example:

{
  "links" : [ {
    "rel" : "self",
    "type" : "application/json",
    "title" : "This document",
    "href" : "https://example.com/path.json"
  }, {
    "rel" : "alternate",
    "type" : "text/html",
    "title" : "This document as HTML",
    "hreflang" : "de",
    "href" : "https://example.com/path.de.html"
  }, {
    "rel" : "alternate",
    "type" : "text/html",
    "title" : "This document as HTML",
    "hreflang" : "de",
    "href" : "https://example.com/path.en.html"
  }, {
    "rel" : "collection",
    "type" : "text/html",
    "title" : "The catalogue of this record",
    "href" : "https://example.com/collection-path"
  } ]
}

We have not discussed yet how additional links / associations should be included in the records. I see three general options and I don't think we need to mandate one, except for the properties that are specified in the Core.

Option 1

Add them to the links member.

{
  "links" : [ ..., 
  {
    "rel" : "related",
    "title" : "A related record",
    "href" : "https://example.com/records-path/related-record-id"
  }, {
    "rel" : "license",
    "title" : "CC BY 2.0",
    "href" : "https://creativecommons.org/licenses/by/2.0/"
  } ]
}

Sample filter expression: links[rel='related'].href LIKE '%/related-record-id' AND links[rel='license'].title LIKE 'CC%'

Note that we need the sub-query for these cases as links.rel='related' AND links.href LIKE '%/related-record-id' would also match cases were both expressions are true, but for different link objects.

Option 2

Add them as normal properties. The rel value would be the JSON member name and not be included in the link object.

{
   "type":"Feature",
   ...
   "properties":{
      ...,
      "related": [ {
		"title" : "A related record",
		"href" : "https://example.com/related-record-path"
	  } ],
      "license": {
		"title" : "CC BY 2.0",
		"href" : "https://creativecommons.org/licenses/by/2.0/"
	  }
   }
}

Sample filter expression: related.href LIKE '%/related-record-id' AND license.title LIKE 'CC%'

Option 3

Similar to option 2, but just the href value.

{
   "type":"Feature",
   ...
   "properties":{
      ...,
      "related": [ "https://example.com/related-record-path" ],
      "license": "https://creativecommons.org/licenses/by/2.0/"
   }
}

Sample filter expression: related LIKE '%/related-record-id' AND license LIKE 'https://creativecommons.org/licenses/%'

[m-mohr]

@cholmes @matthewhanson These kind of filters could be something we use in STAC for querying the bands etc, right?

[pvretano]

07-DEC-2020: The consensus seems to be with option 1 with the following caveats: (1) The key can be anything, "links", "associations", "assets". (2) The structure has to be an array of links with the structure displayed in option 1 (i.e. rel, title, href, etc.) as per features (and an IETF draft).
It seems that there are 2 linking patterns that might emerge. Option 1 and Option 2 (option 3 is just a degenerate case of 2). It might be good it CQL covered both emerging linking patterns.

[MartinePaepen]

You could check https://docs.opengeospatial.org/is/17-003r2/17-003r2.html#23
OGC EO Dataset Metadata GeoJSON(-LD) Encoding Standard
Section 7.1.2
Is seems like a combination of option 1 and option 2.

Example 4: GeoJSON encoding example
{
"type": "Feature",
"id": "http://fedeo.esa.int/opensearch/request/?parentIdentifier=SEA_GEC_1P&uid=SE1_OPER_SEA_GEC_1P_19780927T010430_19780927T010445_001316_0000_2267_9B4F",
"geometry": { … },
"properties": {
"links": {
"data": [
{
"href": "http://tpm-ds.eo.esa.int/products/SEA_GEC_1P/1978/09/27/SE1_OPER_SEA_GEC_1P_19780927T010430_19780927T010445_001316_0000_2267_9B4F.ZIP",
"type": "application/binary",
"title": "Download"
}
],
"previews": [
{
"href": "http://tpm-ds.eo.esa.int/metadata/SEA_GEC_1P/1978/09/27/SE1_OPER_SEA_GEC_1P_19780927T010430_19780927T010445_001316_0000_2267_9B4F.BI.PNG",
"type": "image/png",
"title": "Quicklook"
}
],
"alternates": [
{
"href": "http://fedeo.esa.int/opensearch/request/?httpAccept=application/atom%2Bxml&parentIdentifier=SEA_GEC_1P&uid=SE1_OPER_SEA_GEC_1P_19780927T010430_19780927T010445_001316_0000_2267_9B4F",
"type": "application/atom+xml",
"title": "Atom format"
}
]

    }
}

}

[pvretano]

28-DEC-2020: Peter: mostly documented both approaches in records specification. Mood of SWG however is to not allow that many degrees of freedom and so we should pick 1 and Option 1 is it because that is the linking style used is features and in common. Peter will leave both approach in the specification for now but segregate option 2 as an option to consider but clearly state that option 1 is the preferred approach.

[pvretano]

25-JAN-2021: @pvretano to add text to specification for review. Also related to CQL extension because CQL requires extensions for this too.

[pvretano]

SWG MEETING 08-FEB-2021: Don't have a clear description in the document yet and we need some extensions to CQL. We will define the necessary extensions here in Records in a separate conformance class with the understanding that the extension may eventually move to CQL or common if/when CQL moves to common.

[pvretano]

09-AUG-2021: There have been a lot of changes to the document recently. @cportele will review the recent changes to the document and determine if this issue needs to be addressed here or in CQL in Features and also the JSON-FG SWG.

[pvretano]

07-FEB-2022: Pausing work on this issue for now to allow the work on CQL2 to finish. This particular requirement would end up as either a revision or extension of CQL2.

[pvretano]

16-MAY-2022: Since the Sept. code sprint with be covering Records, JSON-FG, ISO19115 and STAC, it would be good to discuss this issues at the code sprint.

[cportele]

OGC Code Sprint 2022-09-16:

• We agreed to work on a CQL2 extension that supports queryables that have objects and arrays as values. CQL2 currently has support for arrays, but only with simple values (string, number, boolean).
• This will be required for links, but will also be needed for other cases including querying the "theme" property in Records or the "assets" member in STAC.
• The property with structure like "links", "theme" or "assets" would be published as a queryable with their schema.
• This is complementary to the approach to define a queryable to make filtering data structures like "assets" simpler as described in OGC API Features - Part 3: Filtering.
• @cportele will add an example for filtering the "theme" property and the "assets" property.
• We will continue the discussion here to identify the requirements from Records and then move the discussion to the Features API where the CQL2 extension would be drafted.

[cportele]

Example for a filter on "themes" (using a live API):

A_CONTAINS(theme[scheme='profile'].concept, ['DLKM', 'Basis-DLM', 'DLM50'])

This filter queries on the "theme.concept" values where the "scheme" is "profile". This uses the array operator A_CONTAINS since there are multiple values "theme.concept" ("theme" is an array of objects where each object has 0 or 1 "scheme" and an array of strings in "concept").

In the example, all records (in this case values of a code list with building functions) are matched that at least are part of the profiles "DLKM", "Basis-DLM" and "DLM50".

[m-mohr]

A question that is highly relevant for STAC: Is this syntax also working for objects (i.e. STAC Assets)?
Can I search through all assets and find a band with a specific common name?

I'd hope that it would be something like this:
assets[*].eo:bands[common_name=="red"].name orassets[*].eo:bands[*].common_name == 'red'

[cportele]

@m-mohr - Stay tuned, I will work on the asset example in the next days and a rough draft of the extension, but as we agreed in the discussion, it will/has to support objects. Maybe the example in Features Part 3 could be written an A_EQUALS(assets.*.eo:bands.common_name,["nir","blue"]), but I need to spend a bit more time on this.

[pvretano]

15-MAR-2023: Might be related to opengeospatial/ogcapi-features#740

[pvretano]

30-OCT-2023: We need to revist this in light of the new Schema in OAFeat. We will do that and make a determination if this issue can be close or drafted based on the new work.

[pvretano]

08-JUL-2024: This is still a topic that needs to be addressed but since all the API elements of OAPIR are part of features it is a topic that should/will be handled there. Closing.