Dataset improvements

[m-mohr]

Based on the last meeting (see recap: https://groups.google.com/forum/#!topic/stac-spec/A-7_e-eo3oU) I made this PR to implement the changes (related issues: # #184):

• Removed provider and license from Items. Fixes Better define what should go in 'provider' field #78, Handle contact info and other collection level metadata with 'catalog' link? #36 and may make Single provider for datasets #225 obsolete.
• Make datasets required for Items. Somehow fixes Consider versioning STAC Items #136.
• Added a type to the provider object to specify whether the provider is a producer, processor or host. (Newly introduced by me, please discuss.)
• Removed the host, organization info is now in the provider and further info will be added at some point by the storage profiles (see Storage profiles #148).
• Removed the reference to SemVer from the version field (SemVer is pretty software related - was feedback from within openEO).
• Some minor fixes

Doesn't yet implement it to the API (needs to be solved with #257) and the decision on the name (#260) is still due.

[cholmes]

Overall looks great. Put in a few notes to add some more information.

I did raise one thing worth discussing, which is making the requirement of linking to a dataset not in the Item spec. Like put it in catalog/dataset - to make a complaint catalog/dataset all Items referenced from it should refer to a dataset.

[cholmes]

Perhaps we make a recommendation here that the homepage should ideally include a way to find a contact about the dataset? To help explain a bit why we dropped 'contact' info.

[m-mohr]

Okay, will add that next week.

[cholmes]

Maybe it's a bad idea, but I was thinking we could bring the requirement to link to a dataset in at the Catalog level (or the Dataset level). Like let the item spec stand alone, and just have a strong recommendation. And then for Catalog it could add that any Item referenced from the Catalog should have a Dataset.

As I've been explaining the set of specs I've been thinking about trying to make sure they could be used standalone. And requiring a dataset link in an Item does make it less able to stand on its own.

And I think we should put a bit more color here on why this is so strongly recommended. Explain that it provides the license and provider information, as well as additional information about the set of Items.

[m-mohr]

It doesn't really care where things are written down as required - they are still required then. So if we don't want close coupling of things, we can just say that they are strongly recommended and must avoid using requirements between the specs at all. Sure, we can add that items that are linked from in a dataset are required to link back to its dataset, but that's a different thing for me than what you wrote in your recap: "This also lead to requiring a dataset link in every Item". So let's make it that only items linked from a dataset requre a link back. I'll work on that next week.

[m-mohr]

Changed, please review it again.

[cholmes]

We should also probably explain a bit more on what 'producer', 'processor' and 'host' mean. Doesn't need to be in this table, could be just below or above. But just a bit of explanation to help people make a decision about which to use.

[m-mohr]

Okay, I'll add explanation about it next week.

[m-mohr]

Added the explanation. Please check whether it needs more explanation or not.

[matthewhanson]

I find it a bit annoying that Item has an ID, but for Datasets we have name rather than ID.

Can we change it so that both have a unique Identifier that are both called id?

[m-mohr]

@matthewhanson I think I discussed that earlier with @cholmes and the problem is that we are following different specifications here. GeoJSON requires using "id" and WFS3 requires using "name". I don't like it, too, but that's how it is. We have this trouble also in other parts of the spec as "name" is used sometimes synonymously for IDs or titles. Only id and title are pretty much self-explanatory.

[matthewhanson]

Ok, yeah that's too bad. It might be too late in the game to ask WFS folks to change to ID, which is clearly a better term as it's specific. name is not.

At any rate, for other parts of STAC where we can we should avoid name entirely and just use id and title.

[m-mohr]

@matthewhanson Just try it and open an issue? I thought the same for some changes I proposed, but it seems as if they are getting adopted in WFS. We can only win ;-)
The other place that immediately comes to mind in STAC is the asset name, see my (last) comment in #139.

[hgs-msmith]

Approved, with addenda covered in the discussion.
+1 on recommending the change from 'name' to 'id' for WFS.

[m-mohr]

I created an issue for changing name to id in WFS: opengeospatial/ogcapi-features#171
@matthewhanson @hgs-msmith @cholmes Feel free to support that issue. ;-)

[m-mohr]

Will be changed to id in WFS3, see the issue. :) I already changed the STAC catalog and dataset specification. As name was often used as title in catalogs, I basically removed name and introduced both title and id into catalogs. Is that fine with anybody?

[m-mohr]

All requested changes should be incroporated, please re-check whether everything is as expected.

[cholmes]

I just commented on the WFS issue - I'm not fully convinced Clements fully understood that we wanted to change the 'name' in the response document, not just in the url template names. I remember the discussion about the template names, which I think @hgs-msmith brought up, but don't remember them discussing the response context.

But I suppose we can just go ahead with the change and advocate on that issue, as there's pretty good arguments for it and WFS 3.0 isn't locked in yet. Let's just keep tracking the issue and sounding in, and I'm ok for us to go ahead with 'id' (it just may be that WFS+STAC implementations might have to put in both name and id until it works out).

[cholmes]

Looks good. I can't seem to find the discussion about the required dataset, but just wanted to say that your changes look great, and captured what I was thinking. And also wanted to acknowledge that yes, it's a bit less than what we talked about on the call, but on further reflection it seemed a further requirement than I was thinking, particularly since license and provider aren't required STAC fields, so moving them to dataset isn't enough justification to make dataset required.

[cholmes]

Oh, also note that no iterations of this PR actually fixed #136 - the 'version' in dataset is not the version of the STAC specification they are following, it's the version of the dataset.

We'd need to add in the stac_version parameter in to dataset, or catalog.

[m-mohr]

WFS id vs name: I actually thought changing the URL template would imply also changing the response, otherwise it would be even more inconsistent than before. I think it is fine to follow the route @cholmes suggested.

I can't seem to find the discussion about the required dataset

Its in the chapter about the relation types. It's not a big discussion, but more a note that using the item rel type makes it required for the linked item to refer back to its dataset. Do you see it now?

it's a bit less than what we talked about on the call

But you still think it is good as it is now or should we still change/improve something?

Regarding stac_version (#136): I was not sure whether I was excpected to add that or not, but I can make a separate PR for it.

[cholmes]

I can't seem to find the discussion about the required dataset

Its in the chapter about the relation types. It's not a big discussion, but more a note that using the item rel type makes it required for the linked item to refer back to its dataset. Do you see it now?

Oh I just meant I couldn't find our github discussion to respond in the thread. I found the edits in the spec - they look great. And yes, I think it's good as it is right now.

Regarding stac_version (#136): I was not sure whether I was excpected to add that or not, but I can make a separate PR for it.

No, I wasn't expecting you to add it. But a separate PR on it would be great.

[m-mohr]

Oh I just meant I couldn't find our github discussion to respond in the thread. I found the edits in the spec - they look great. And yes, I think it's good as it is right now.

@cholmes Were you looking for the discussions in #174 and #136 ?