Do something when STAC endpoint url ends with "/"

[remicres]

Hi,

When the STAC enpoint ends with "/", random things crashes in pystac.

from pystac_client import Client
api1 = Client("https://some-stac-api.org/")  # erratically works
api2 = Client("https://some-stac-api.org")  # works 

Maybe it could be more user/developer friendly to throw an exception when the enpoint is badly formed (I spent like 30 minutes figuring out why, I guess its a classic for newcomers).
Or, fix the url, or use urllib or something else to concatenate URL?

[gadomski]

Can you provide examples of the breakages? I just tried a couple of queries using with and without a trailing / and they seemed to work fine.

[remicres]

Yes it looks like it depends on the API. Our own API is still internal right now (we have used STAC-FastAPI out of the box) but here is another one on which I can reproduce the issue: https://data.geo.admin.ch/api/stac/v0.9

[gadomski]

That endpoint is pretty old -- it's stac_version is v0.9.0 and it doesn't have any listed conformance classes. Do you have a server example that's in the STAC-API v1.0.0-rc.* range?

[remicres]

Yes I have one, but not public. It consists of pgstac + stac-fastapi (built two weeks ago from the stac-fastapi repository, master branch) so I believe it is in the v1.0.0-rc.* range... how can I check that?
It's running in k8s behind an ingress that does basic url rewrite, I don't think that is the cause (the pystac Client just append api to the endpoint right?)
Thanks

[gadomski]

I tested against the planetary computer (https://planetarycomputer.microsoft.com/api/stac/v1), which is also pgstac + stac-fastapi, and haven't had any issues as of yet. Can you try some of your problematic queries against that endpoint, and post any results here?

[remicres]

Sure. Here is what I got from scratch.

Run docker image (fresh build from master branch of stac-utils/stacapi)

cresson@cin-mo-gpu:~/integration-tests$ docker run -e ... -ti -p 8000:8082 stac-fastapi:main

Run python code from another server:

from pystac_client import Client
url = ...
api = Client.open(url)
print([c for c in api.get_collections()])

With url = "http://10.134.xxx.yyy:8000"

Client:

[]

STAC FastAPI server logs:

INFO:     Will watch for changes in these directories: ['/stac-fastapi']
INFO:     Uvicorn running on http://0.0.0.0:8082 (Press CTRL+C to quit)
INFO:     Started reloader process [1] using StatReload
INFO:     Started server process [8]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     10.34.xxx.zzz:53328 - "GET / HTTP/1.1" 200 OK
INFO:     10.34.xxx.zzz:53328 - "GET /collections HTTP/1.1" 200 OK

With url = "http://10.134.xxx.yyy:8000/" (with a "/" at the end)

Output:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<string>", line 1, in <listcomp>
  File "/usr/local/lib/python3.8/dist-packages/pystac_client/client.py", line 258, in get_collections
    for page in self._stac_io.get_pages(url):  # type: ignore
  File "/usr/local/lib/python3.8/dist-packages/pystac_client/stac_api_io.py", line 220, in get_pages
    page = self.read_json(url, method=method, parameters=parameters)
  File "/usr/local/lib/python3.8/dist-packages/pystac/stac_io.py", line 198, in read_json
    txt = self.read_text(source, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/pystac_client/stac_api_io.py", line 97, in read_text
    return self.request(href, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/pystac_client/stac_api_io.py", line 144, in request
    raise APIError.from_response(resp)
pystac_client.exceptions.APIError: {"detail":"Not Found"}

STAC FastAPI server logs:

...
INFO:     10.34.xxx.zzz:53332 - "GET / HTTP/1.1" 200 OK
INFO:     10.34.xxx.zzz:53332 - "GET //collections HTTP/1.1" 404 Not Found

[gadomski]

Thanks for the example! I'm now able to reproduce. Fix in #373.

My tests weren't failing because https://planetarycomputer.microsoft.com/api/stac/v1/collections and https://planetarycomputer.microsoft.com/api/stac/v1//collections (notice the double slash) both work. But using a vanilla stac-fastapi I was able to reproduce the issue.