Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#260: Rename 'Dataset extension' to 'Collection extension' + other improvements #280

Merged
merged 19 commits into from
Oct 13, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
31c2209
Renamed dataset to collection in texts and some examples.
m-mohr Oct 10, 2018
1f848bc
Renamed dataset-spec folder and spec file to collection-spec.
m-mohr Oct 10, 2018
be5b769
Additions to PR #275 (removing old collection extension document and …
m-mohr Oct 11, 2018
295e049
Merge branch 'dev' into dataset-rename
m-mohr Oct 11, 2018
36ed23c
Fixed issues mostly introduced by other PRs, some restructuring and o…
m-mohr Oct 11, 2018
7e26149
Read through the texts and examples and fixed inconsistencies and oth…
m-mohr Oct 11, 2018
aab3f40
Changed scope of the scientific extension.
m-mohr Oct 12, 2018
159757c
Renamed the datetime-range README file to be consistent with other ex…
m-mohr Oct 12, 2018
0088537
Fixed/improved a bunch of links and solved several issues raised in t…
m-mohr Oct 12, 2018
0927b0c
Moved around recommendations.
m-mohr Oct 12, 2018
90e5df0
Merge remote-tracking branch 'origin/dev' into dataset-rename
m-mohr Oct 12, 2018
a5f8bc9
Updated link structure that is used for pagination.
m-mohr Oct 12, 2018
f43f6be
Forgot to commit the transaction extension changes.
m-mohr Oct 12, 2018
c5ebad1
Removed recommendation section
cholmes Oct 12, 2018
a786c93
Added more prominent link to collection spec
cholmes Oct 12, 2018
c41564a
Merge pull request #287 from radiantearth/fix-broken-recommendation-link
m-mohr Oct 13, 2018
dc90f08
Merge pull request #288 from radiantearth/link-to-spec
m-mohr Oct 13, 2018
5f3c563
Made a set of changes proposed by @cholmes, mainly regarding collecti…
m-mohr Oct 13, 2018
5d58af4
Added information about collections and commons extension to the scie…
m-mohr Oct 13, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ jobs:
command: |
cd validation
npm run validate_catalog -- -d ../catalog-spec/examples/catalog.json
npm run validate_dataset -- -d ../dataset-spec/examples/sentinel2.json
npm run validate_collection -- -d ../collection-spec/examples/sentinel2.json
npm run validate_extension -- -s ../extensions/scientific/schema.json -d ../extensions/scientific/example-merraclim.json
npm run validate_item -- -d ../item-spec/examples/sample.json
# ToDo: More files should be validated
File renamed without changes.
43 changes: 21 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,9 @@ The specification is currently still an early version, with the potential for so
fleshed out, so implementors are encouraged to try it out and give feedback. But the goal is to actually be able to act
on that feedback, which will mean changes are quite possible.

But efforts will be made to maintain the core fields established in the central [Item spec](item-spec/) and
[Catalog spec](catalog-spec). The minimal amount is specified right now, but best practices should emerge with
implementation and more will likely be specified.
But efforts will be made to maintain the core fields established in the central [Item spec](item-spec/),
[Catalog spec](catalog-spec/) and [Collection spec](collection-spec/).
The minimal amount is specified right now, but best practices should emerge with implementation and more will likely be specified.

## Current version and branches

Expand All @@ -40,7 +40,7 @@ not require lots of updating.
For any questions feel free to jump on our [gitter channel](https://gitter.im/SpatioTemporal-Asset-Catalog/Lobby) or email
our [google group](https://groups.google.com/forum/#!forum/stac-spec). The majority of communication about the evolution of
the specification takes place in the [issue tracker](https://github.com/radiantearth/stac-spec/issues) and in
[pull requests](https://github.com/radiantearth/stac-spec/pulls)
[pull requests](https://github.com/radiantearth/stac-spec/pulls).

## In this Repository

Expand All @@ -50,24 +50,25 @@ README explaining the layout of the folder, the main specification document, exa
documents (if relevant). The four specifications detailed are meant to be used together, but are designed so each piece
is small, self-contained and reusable in other contexts.

**[item-spec/](item-spec/)** defines a SpatioTemporal Asset Catalog `Item`, which is a [GeoJSON](http://geojson.org) Feature
**[item-spec/](item-spec/)** defines a STAC Item, which is a [GeoJSON](http://geojson.org) Feature
with additional fields for things like time, links to related entities and assets (including thumbnails). This is the
atomic unit that describes the data to be discovered.

**[catalog-spec/](catalog-spec/)** specifies a structure to link various `Items` together to be crawled or browsed. It is a
simple, flexible JSON file of links to `Item`s or other `Catalogs` that can be used in a variety of ways.
**[catalog-spec/](catalog-spec/)** specifies a structure to link various STAC Items together to be crawled or browsed. It is a
simple, flexible JSON file of links to Items, Catalogs or Collections that can be used in a variety of ways.

**[dataset-spec/](dataset-spec/)** provides additional information about a set of `Item`s that are made available by a
data provider. It includes things like the spatial and temporal extent of the set of data, the license, keywords, etc. It
enables discovery at a higher level than individual items, providing a simple way to describe sets of data.
**[collection-spec/](collection-spec/)** provides additional information about a spatio-temporal collection of data.
In the context of STAC it is most likely a collection of STAC Items that is made available by a data provider.
It includes things like the spatial and temporal extent of the data, the license, keywords, etc.
It enables discovery at a higher level than individual items, providing a simple way to describe sets of data.

**[api-spec/](api-spec/)** extends the core publishing capabilities of STAC with an active REST search endpoint that returns
just the `Item`s a user requests in their query. It is specified as a couple [OpenAPI](http://openapis.org) documents, one
just the Items a user requests in their query. It is specified as a couple [OpenAPI](http://openapis.org) documents, one
[standalone](api-spec/STAC-standalone.yaml) and one that is [integrated with WFS3](api-spec/WFS3core%2BSTAC.yaml)
(see [wfs github](https://github.com/opengeospatial/wfs_fes) for info on it). The documents also include the `/stac/`
endpoint which is a way for a dynamic server to provide catalog and dataset browsing.
(see [WFS3 on GitHub](https://github.com/opengeospatial/wfs_fes) for info on it). The documents also include the `/stac/`
endpoint which is a way for a dynamic server to provide catalog and collection browsing.

**Extensions:** The *[extensions/](extensions/)* folder is where extensions (profiles) live. Extensions can extend the
**Extensions:** The *[extensions/](extensions/)* folder is where extensions live. Extensions can extend the
functionality of the core spec or add fields for specific domains.

**Additional documents** include the current [roadmap](roadmap.md) and a complementary [how to help](how-to-help.md)
Expand All @@ -78,7 +79,8 @@ and a discussion of the collaboration [principles](principles.md) and specificat

An important core principle of the STAC design is to embrace best practices of making data available on the web (like
[HATEOAS](https://en.wikipedia.org/wiki/HATEOAS) and [W3C Spatial Data on the Web](https://www.w3.org/TR/sdw-bp/)), and
to leverage the reliability of flat files on object stores like [AWS S3](https://aws.amazon.com/s3/) and [Google Cloud Storage](https://cloud.google.com/storage/). This leads to putting the 'static catalog' at the core of the STAC spec.
to leverage the reliability of flat files on object stores like [AWS S3](https://aws.amazon.com/s3/) and [Google Cloud Storage](https://cloud.google.com/storage/).
This leaded to designing a static catalog at the core of the STAC spec.

#### Static Catalog

Expand All @@ -99,17 +101,14 @@ or at least may have a 'backup' of its fields stored as a cached static catalog.

#### Core Metadata and Extensions

The [SpatioTemporal Asset Metadata](https://github.com/radiantearth/stam-spec) specification defines the core fields that all
assets must make available for searching in a catalog. Vendors can extend those core fields for the metadata they want to
make available, and the community is starting to define shared extensions, with 'electro-optical' being
the first one. This repo contains the [STAC Item](item-spec/item-spec.md) definition, which is the primary json extension of STAM,
so it can evolve with the rest of the STAC spec more easily. The STAM repo retains the abstract definition, and may evolve
to contain other extensions.
The Item specification defines the core fields that all assets must make available for searching in a catalog.
In addition there are some basic fields for describing collections of data.
Vendors can extend those core fields for the metadata they want to make available, and the community has started to define shared extensions.

## Contributing

Anyone building software that catalogs imagery or other geospatial assets is welcome to collaborate.
Beforehand, please review our [guidelines for contributions](contribute.md).
Beforehand, please review our [guidelines for contributions](CONTRIBUTING.md).



Expand Down
18 changes: 7 additions & 11 deletions api-spec/README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,8 @@

# STAC API Specification

## Overview

A STAC API is the dynamic version of a SpatioTemporal Asset Catalog. It returns [STAC Items](../item-spec/item-spec.md)
(GeoJSON objects with required links time stamp and links to assets) from search queries on a RESTful endpoint, and also
offers an endpoint to return STAC Items as a compliant [Catalog](../catalog-spec/) for discovery.
A STAC API is the dynamic version of a SpatioTemporal Asset Catalog. It returns [STAC Items](../item-spec/README.md)
(GeoJSON objects with required links, time stamp and links to assets) from search queries on a RESTful endpoint, and also
offers an endpoint to return STAC Items as a compliant [STAC Catalog](../catalog-spec/README.md) for discovery.

The core of the spec is a single endpoint:

Expand All @@ -16,7 +13,6 @@ The core of the spec is a single endpoint:
It takes a JSON object that can filter on date and time:

```

{
"bbox": [ 5.5, 46, 8, 47.4 ],
"time": "2018-02-12T00:00:00Z/2018-03-18T12:31:12Z"
Expand All @@ -31,18 +27,18 @@ that fall within this area:
*map © OpenStreetMap contributors*

The return format is a [GeoJSON](http://geojson.org) feature collection with features compliant with the
[Item spec]((../item-spec/item-spec.md) for STAC. It returns to a limit optionally requested by the client, and includes
[Item spec]((../item-spec/README.md) for STAC. It returns to a limit optionally requested by the client, and includes
pageable links to iterate through any results past that limit.

### Dynamic Catalog API
## Dynamic Catalog API

The other endpoint that is included as an option in STAC API is `/stac/`, which implements the [Catalog Spec](../catalog-spec/).
The other endpoint that is included as an option in STAC API is `/stac/`, which implements the [STAC Catalog Spec](../catalog-spec/README.md).
See the [Dynamic Catalog](https://github.com/radiantearth/stac-spec/blob/structure_and_cleanup/catalog-spec/catalog-spec.md#dynamic-catalogs)
section of that spec for more information. Implementing this enables tools like
[STAC Browser](https://medium.com/@mojodna/a-stac-browser-348a60674061) to use the dynamic catalog, to enable better
discovery through people browsing and search engines crawling.

The OpenAPI spec in this directory documents the endpoint, and refer to the Catalog and [Dataset](../dataset-spec) for more information about the full content and link structure.
The OpenAPI spec in this directory documents the endpoint, and refer to the STAC Catalog and [STAC Collection](../collection-spec/README.md) for more information about the full content and link structure.

## API Fragments

Expand Down
18 changes: 9 additions & 9 deletions api-spec/STAC-query.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -512,15 +512,15 @@ components:
additionalProperties:
description: Any additional properties added in via extensions.
itemCollectionLinks:
type: object
properties:
next:
type: string
format: url
description: >-
A URL to obtain the next paginated data set. If not present, the
client should utilize the `nextPageToken`.
example: >-
type: array
description: >-
An array of links. Can be used for pagination, e.g. by providing a link
with the `next` relation type.
items:
$ref: '#/components/schemas/link'
example:
- rel: next
href: >-
http://api.cool-sat.com/query/gasd312fsaeg/ANsXtp9mrqN0yrKWhf-y2PUpHRLQb1GT-mtxNcXou8TwkXhi1Jbk
queryFilter:
type: object
Expand Down
18 changes: 9 additions & 9 deletions api-spec/STAC-standalone.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -481,15 +481,15 @@ components:
additionalProperties:
description: Any additional properties added in via extensions.
itemCollectionLinks:
type: object
properties:
next:
type: string
format: url
description: >-
A URL to obtain the next paginated data set. If not present, the
client should utilize the `nextPageToken`.
example: >-
type: array
description: >-
An array of links. Can be used for pagination, e.g. by providing a link
with the `next` relation type.
items:
$ref: '#/components/schemas/link'
example:
- rel: next
href: >-
http://api.cool-sat.com/query/gasd312fsaeg/ANsXtp9mrqN0yrKWhf-y2PUpHRLQb1GT-mtxNcXou8TwkXhi1Jbk
tags:
- name: STAC
Expand Down
22 changes: 11 additions & 11 deletions api-spec/WFS3core+STAC.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -640,15 +640,15 @@ components:
additionalProperties:
description: Any additional properties added in via extensions.
itemCollectionLinks:
type: object
properties:
next:
type: string
format: url
description: >-
A URL to obtain the next paginated data set. If not present, the
client should utilize the `nextPageToken`.
example: >-
type: array
description: >-
An array of links. Can be used for pagination, e.g. by providing a link
with the `next` relation type.
items:
$ref: '#/components/schemas/link'
example:
- rel: next
href: >-
http://api.cool-sat.com/query/gasd312fsaeg/ANsXtp9mrqN0yrKWhf-y2PUpHRLQb1GT-mtxNcXou8TwkXhi1Jbk
root:
type: object
Expand Down Expand Up @@ -785,11 +785,11 @@ components:
- properties
- constructions
version:
title: Dataset Version
title: Collection Version
type: string
example: 1
license:
title: Dataset License Name
title: Collection License Name
type: string
example: Apache-2.0
provider:
Expand Down
4 changes: 2 additions & 2 deletions api-spec/definitions/STAC-collections.fragment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,11 @@ components:
- properties
- constructions
version:
title: Dataset Version
title: Collection Version
type: string
example: 1
license:
title: Dataset License Name
title: Collection License Name
type: string
example: Apache-2.0
provider:
Expand Down
18 changes: 8 additions & 10 deletions api-spec/definitions/STAC-standalone.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -474,16 +474,14 @@ components:
additionalProperties:
description: Any additional properties added in via extensions.
itemCollectionLinks:
type: object
properties:
next:
type: string
format: url
description: >-
A URL to obtain the next paginated data set. If not present, the
client should utilize the `nextPageToken`.
example: >-
http://api.cool-sat.com/query/gasd312fsaeg/ANsXtp9mrqN0yrKWhf-y2PUpHRLQb1GT-mtxNcXou8TwkXhi1Jbk
type: array
description: >-
An array of links. Can be used for pagination, e.g. by providing a link with the `next` relation type.
items:
$ref: '#/components/schemas/link'
example:
- rel: next
href: http://api.cool-sat.com/query/gasd312fsaeg/ANsXtp9mrqN0yrKWhf-y2PUpHRLQb1GT-mtxNcXou8TwkXhi1Jbk
tags:
- name: STAC
description: Extension to WFS3 Core to support STAC metadata model and search API
14 changes: 8 additions & 6 deletions api-spec/wfs-stac.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,16 +26,17 @@ that capability. So any WFS that is providing data that can be represented as ST
STAC endpoint.

In addition to the `/stac/search` endpoint, STAC API's provide the ability to 'browse' all the [Items](../item-spec/) that
are available from a service. This is done using the `/stac/` endpoint, returning a full [STAC Catalog](../catalog-spec/). Ideally that Catalog makes extensive use of [STAC Datasets](../dataset-spec/) to further describe the holdings.
are available from a service. This is done using the `/stac/` endpoint, returning a full [STAC Catalog](../catalog-spec/).
Ideally that Catalog makes extensive use of [STAC Collections](../collection-spec/) to further describe the holdings.

The STAC Dataset spec is designed to be compatible with the WFS /collections/{collectionId} endpoint's response. This enables
WFS + STAC implementations to just extend their collections with a bit more information to be STAC Compliant dataset
The STAC Collection spec is designed to be compatible with the WFS /collections/{collectionId} endpoint's response. This enables
WFS + STAC implementations to just extend the WFS collections with a bit more information to be STAC compliant Collection
definitions.

An additional best practice is to use the WFS items available in /collections/{collectionId}/items as the 'canonical' web
location. Then the STAC Catalogs returned from `/stac/` can either link directly to those (from the appropriate sub-catalog -
for example /stac/landsat8/42/31/2017/ would be a catalog consisting of links to /collections/). Or it can return json
in the link structure (like /stac/landsat8/42/31/2017/item203123.json), and have that returned json use a link with rel=canonical that goes back to the `Item` that is in the collection.
in the link structure (like /stac/landsat8/42/31/2017/item203123.json), and have that returned json use a link with rel=canonical that goes back to the Item that is in the collection.

### WFS Structure

Expand All @@ -47,8 +48,9 @@ GET /collections

Lists the collections of data on the server that can be queried ([7.11](https://rawgit.com/opengeospatial/WFS_FES/master/docs/17-069.html#_feature_collections_metadata)),
and each describes basic information about the geospatial data collection, like its name and description, as well as the
spatial and temporal extents of all the data contained. A STAC search extension would only query those collections which
have data that validates as STAC `Items` - with a datetime field and references to assets. But a STAC can live alongside
spatial and temporal extents of all the data contained. The collections returned are compliant to both WFS Collections and
[STAC collections](../collections-spec/README.md). A STAC search extension would only query those collections which
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would ideally like a bit more discussion here, reiterating the point of the STAC collections, and contrasting them against WFS collections, perhaps mentioning any additional alignment we'd see in the future, etc. Not required to merge this by any means, but definitely a nice to have.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be added at some point, but don't feel that I can really make a good proposal for this one now. Feel free to make a PR for it?!

have data that validates as STAC Items - with a datetime field and references to assets. But a STAC can live alongside
other WFS collections, like an organization might choose to have their building and road data in WFS collections, alongside
their STAC-compatible imagery data.

Expand Down
Loading