Pagination (limit parameter) for collections?

[m-mohr]

OGC API - Features specifies a limit parameter for items. There's no limit / pagination for collections though. We have some cases with hundreds of collections so would it make sense to add an optional limit parameter for pagination of collections?

[bradh]

This seemed like a reasonable request on first pass, but maybe we don't need it. I don't know of anyone who paginated "GetCapabilities" responses, and this is (conceptually) a subset of what we would have returned in that kind of document.

Is there ever a case where you wouldn't want all of the collections? If you're always going to iterate to end, then pagination only makes sense if the response it too large to handle in one hit. I don't see that as the case for less than at least (rough order) thousands of collections, even on a web client.

@m-mohr Any feel for rough size of the JSON for your cases?

[cportele]

In general, such an option could be useful, but it has to be a separate conformance class and should be in a future part of Common (not part 1).

[m-mohr]

@bradh At the moment often between 1 to 10 megabytes.

@cportele Why?

[cportele]

@m-mohr - In my view, part 1 should only contain the requirements that most implementations will need and this is not one of them. This was also discussed in the Features/STAC sprint in March 2018. See opengeospatial/ogcapi-features#76.

[dblodgett-usgs]

I agree with @cportele that this would be in a separate conformance class and not in core.

I would also like to advocate for this sort of general utility. I have experienced getCapabilities documents that got completely out of hand -- particularly in SOS where the offering list was tens of thousands of sensors. Looking forward to features use cases concerning citizen science, or extensions that push things like monitoring stations up to the collections level, I could imagine collections being something that could proliferate rapidly.

That said, we may want to make an architectural decision about collections now that says use cases in which this would occur are best implemented as collections of collections such that the thing that would proliferate would be encapsulated in something that could be grokked at the top level by a human without a need to do any filtering?

[bradh]

As an alternative, perhaps there should be a simpler way to get the list, and a more detailed description if you need it.

I'm concerned that hundreds of collections turns into megabytes of JSON though. I don't think consumers should be seeing 10K text per collection. If we're going to show it to the user, we need to be realistic about what we're exporting.

[cmheazel]

The CollectionInfo (or Collection in Features) entity includes an extent. It would be logical to extend the scope of the limit, bbox, and datetime parameters to include CollectionInfo as well. So my suggestion is to bring the limit, bbox, and datetime parameters into Common as a new conformance class with the scope of collections, collectionInfo, and items.

[bradh]

@cmheazel Is this comment about extents related to pagination, or was it intended for another issue?

[cmheazel]

API-Common:Collections supports use of the limit parameter on the /collections path. This will limit the number of collections included in the /collections response. So an API with a large number of collections can return the information about those collections as a set of pages of size specified by the limit parameter and accessed by the "next" link.

[m-mohr]

Yes, it's in the spec now. Can be closed from my side.