Harmonize the "queryables" resource with OGC API Styles

[cportele]

See http://docs.opengeospatial.org/DRAFTS/20-009.html#rc_queryables. Both resources have the same role. There should only be a single requirements class for queryables.

[cportele]

2020-08-03: @cportele to make a proposal, also considering the ongoing discussion about the tileset resource in OGC API Tiles.

[cportele]

Link to the section in the document: http://docs.opengeospatial.org/DRAFTS/19-079.html#filter-queryables

Proposal for the schema of the queryables:

type: object
required:
- queryables   
properties:
   queryables:
      type: array
      items: 
         type: object
         required:
         - id   
         - type
         properties:
            id:
               description: the token that may be used in a query predicate
               type: string
            title:
               description: a human readable title for the queryable
               type: string
            description:
               description: a human-readable narrative describing the queryable
               type: string
            language:
               description: the language used for the title and description
               type: string
            type:
               description: the data type of the queryable
               type: string
               enum:
               - string
               - number
               - integer
               - boolean
               - spatial
               - temporal
            spatial-types:
               description: the list of allowed spatial geometry types, if known
               type: string
               enum:
               - point
               - multi-point
               - curve
               - multi-curve
               - surface
               - multi-surface
               - solid
               - multi-solid
               - aggregate
               - any
            temporal-types:
               description: the list of allowed temporal geometry types, if known
               type: string
               enum:
               - instant
               - interval
            links:
               description: additional links, e.g. to documentation or to the schema of the values
               type: array
               items:
                  $ref: 'https://raw.githubusercontent.com/opengeospatial/ogcapi-features/master/core/openapi/ogcapi-features-1.yaml#/components/schemas/link'

So the example would change to:

{   
   "queryables": [
      {
         "id": "elevation",
         "title": "Elevation",
         "description": "The average distance of the road segment above sea level in meter.",
         "type": "number"
      },
      {
         "id": "nlanes",
         "title": "Number of lanes",
         "description": "The total number of lanes in all directions.",
         "type": "integer"
      },
      {
         "id": "geom",
         "title": "Segment Geometry",
         "description": "The geometry of the road segment",
         "type": "spatial",
         "spatial-type": "curve"
      },
      {
         "id": "name",
         "title": "Segment Name",
         "description": "The common name of the road segment.",
         "type": "string"
      }
   ]
}

Changes:

• Changed queryable to id to make it consistent with other schemas, e.g. in the Collections resource where id is used.
• Provided enums for type. The values can be debated, but at least we need to distinguish strings, numbers, boolean, spatial geometries and temporal geometries as these are relevant for filters.
• Added spatial-type and temporal-type as hints. Maybe point/curve/surface/solid/any is sufficient for the spatial types and we do not need to distinguish the aggregates?
• Added a links section so that one can add a link to a JSON schema (rel: describedby, type: application/schema+json) or human readable documentation, where available.

We should extend the example to cover also a temporal property and links.

[jeffharrison]

Styles SWG mtg discussed this Issue. Group assessed that such a 'schema' is needed Visual Style Editors. Proposal will be reworked and posted.

[cportele]

To add to the comment above, for styles there are different requirements depending on where the data is styled. For styling in the client (vector tiles), one needs the schema of the data in the vector tiles (same as in the tileset resource), for styling on the server (map images), one needs the properties that may be used in filters evaluated on the server, i.e., the queryables.

I will have a look at this again and try to come up with a proposal that takes that into account. Probably we need two resources, the schema and the list of queryables (with their schema). Both should be consistent with each other.

[cportele]

Meeting 2020-10-26:

We want to use the same approach for queryables, returnables and the full schema. They should be defined in the same spec (probably a separate schema extension), but in separate requirements classes. Use JSON Schema direct as far as possible for JSON encodings (likely a profile), without OGC-specific extensions / sugar.

The queryables response would be a single JSON object where each property would be a queryable. The same would be for the full schema and the returnables. Where the response uses GeoJSON, the returnables schema (and the full schema, too) would describe the GeoJSON objects. The returnables schema is a subset of the full schema. The full schema backend datastore schema, the returnables is a defined profile/view for a specific use.

@cportele to draft an example. @cportele and @pvretano will try to update their software with live examples.

We would need specific link relation types for each of the schema types. We could still use fixed paths for the resources under the Collection resource.

[cportele]

I have updated the ldproxy demo server with the approach that we have discussed

Queryables

A JSON object, a property for every queryable.

Link

{
  "$schema" : "https://json-schema.org/draft/2019-09/schema",
  "$id" : "https://demo.ldproxy.net/daraa/collections/CulturePnt/queryables",
  "type" : "object",
  "title" : "Cultural (Points)",
  "description" : "Cultural: Information about features on the landscape that have been constructed by man.",
  "properties" : {
    "F_CODE" : {
      "type" : "string",
      "title" : "Feature Type",
      "enum" : [ "AK121", "AL012", "AL030", "AL130", "BH075" ]
    },
    "ZI001_SDV" : {
      "type" : "string",
      "title" : "Last Change",
      "format" : "date-time,date"
    },
    "ZI037_REL" : {
      "type" : "integer",
      "title" : "Religious Designation",
      "enum" : [ -999999, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14 ]
    },
    "geometry" : {
      "$ref" : "https://geojson.org/schema/Point.json"
    }
  }
}

Response schema

The schema of the GeoJSON object returned by the API.

Link

{
  "$schema" : "https://json-schema.org/draft/2019-09/schema",
  "$id" : "https://demo.ldproxy.net/daraa/collections/CulturePnt/schema?f=json",
  "type" : "object",
  "title" : "Cultural (Points)",
  "description" : "Cultural: Information about features on the landscape that have been constructed by man.",
  "required" : [ "type", "geometry", "properties" ],
  "properties" : {
    "type" : {
      "type" : "string",
      "enum" : [ "Feature" ]
    },
    "id" : {
      "type" : "integer"
    },
    "links" : {
      "type" : "array",
      "items" : {
        "$ref" : "#/$defs/Link"
      }
    },
    "geometry" : {
      "$ref" : "https://geojson.org/schema/Point.json"
    },
    "properties" : {
      "type" : "object",
      "properties" : {
        "FCSUBTYPE" : {
          "type" : "integer",
          "title" : "Feature Subtype Code"
        },
        "F_CODE" : {
          "type" : "string",
          "title" : "Feature Type",
          "enum" : [ "AK121", "AL012", "AL030", "AL130", "BH075" ]
        },
        "UFI" : {
          "type" : "string",
          "title" : "Unique Entity Identifier"
        },
        "ZI001_SDP" : {
          "type" : "string",
          "title" : "Source Description"
        },
        "ZI001_SDV" : {
          "type" : "string",
          "title" : "Last Change",
          "format" : "date-time,date"
        },
        "ZI005_FNA" : {
          "type" : "string",
          "title" : "Name"
        },
        "ZI006_MEM" : {
          "type" : "string",
          "title" : "Memorandum"
        },
        "ZI037_REL" : {
          "type" : "integer",
          "title" : "Religious Designation",
          "enum" : [ -999999, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14 ]
        }
      }
    }
  },
  "$defs" : {
    "Link" : {
      "type" : "object",
      "required" : [ "href" ],
      "properties" : {
        "href" : {
          "type" : "string",
          "format" : "uri-reference"
        },
        "rel" : {
          "type" : "string"
        },
        "type" : {
          "type" : "string"
        },
        "title" : {
          "type" : "string"
        }
      }
    }
  }
}

Mutation schema

The APIs on that server do not support transactions, so there is no mutation schema (or however we will call it), but it would be the response schema without links and id as these, if present, would be ignored in a POST/PUT request.

JSON Schema profile

We discussed that we would recommend a profile of JSON Schema to simplify parsing by clients. Additional elements would be allowed, but providers should be aware that clients may ignore them. Here is a proposal based on the current drafts on schemas in various OGC API drafts:

• The following assumes we reference the current version (2019-09).
• A feature schema is always an object.
  • The schema should include $schema and $id.
  • The following discusses only the schema for the feature properties. For the response schema and the mutation schema additional schema elements for the GeoJSON properties like id, geometry, properties have to be added.
• requried can be used to identify the required properties
  • required would not be used in the queryables schema
• properties
  • type: all values (string, number, integer, boolean, null, object, array)
    • object would not be used in the queryables schema
    • should support for objects we in a separate conformance class
    • do we want to allow an array, e.g. [ 'string', 'null' ]?
  • For strings: minLength, maxLength, pattern, format with date-time/date/time/uri-reference/email, enum.
  • For numbers: multipleOf , minimum , exclusiveMinimum, maximum, exclusiveMaximum
  • For integers: same as for numbers plus enum
  • For arrays: items, minItems, maxItems, $ref (to #/$defs/{typename} for object valued properties)
  • For objects: properties, required, $ref (to #/$defs/{typename} for object valued properties)
• $defs for cases where objects as values of feature properties are supported

Additional questions

• Do we also want to have a way to indicate which queryables map to bbox and datetime?

[jeffharrison]

@cportele Great example. I've sent this out to Styles SWG as well...

Thanks and Regards,
Jeff

[cportele]

Meeting 2020-11-09: The example looks good and @pvretano will implement the approach soon. If any issue is identified in the process we will resolve the problem and otherwise, if there are no other comments, plan to move forward with the approach.

The question, if we want to indicate the properties that are used for datetime / geometry in the queryables: For Core users the information should not be necessary. For users that want and need more control, the CQL/filter extension can and should be used. So the current thinking is to not include that indication in the queryables schema. If the level of control is needed, use filter/CQL, if not use datetime from Core.

We still need to discuss the JSON Schema profile.

If anyone has any comments, please let us know.

[cportele]

Reopen for review in SWG meeting on 2020-12-07

[cportele]

For now, the issue has been addressed, but we will keep this issue open since the schema discussion is broader than "just" queryables and Part 3.

[aaime]

I've implemented queryables representation as json-schema for GeoServer as well, while reviewing the CQL spec (available in GeoServer 2.20.x nightly builds, I don't have an online server supporting that version that I could link to, unfortunately).

That said, using schema covers all the technical needs for listing queryables, presentables and the like. But semantically, I'm a bit torn... people can look at it and say "ah it's a schema" so wonder "the schema of what". Of nothing, in fact.
I believe this could be better stressed in the standard, that schema is being used as a way to enumerate properties, but the docuemnt is not really describing the schema of any information returned by the service (or at least, not necessarily doing so).