Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

xcube server to deliver data #97

Open
1 of 3 tasks
forman opened this issue Jun 18, 2019 · 3 comments
Open
1 of 3 tasks

xcube server to deliver data #97

forman opened this issue Jun 18, 2019 · 3 comments
Labels
DCFS enhancement New feature or request important This is very important for the project urgent High external pressure to address this ASAP xcube serve This is related to server component, CLI "xcube serve"

Comments

@forman
Copy link
Member

forman commented Jun 18, 2019

Is your feature request related to a problem? Please describe.

xcube serve does not yet allow for data access.

Describe the solution you'd like

The xcube web API should provide data access functions, e.g. the forms

  • POST /datasets/{dataset}/data
  • GET /datasets/{dataset}/data?<query-params>
  • GET /datasets/{dataset}/data.zarr

In the first two forms a cube subset is reformatted and then made available for download. Hence, the response returns a binary stream. The <query-params> and the POST body provides the following parameters:

  • bbox=<xmin>,<ymin>,<xmax>,<ymax> - optional bounding box for the output
  • timeRange=<start>,<end> - optional time range for the output
  • variables=<name1>,<name2>,...,<nameN> - optional output variables
  • format=<format> - optional output format, where <format> is one of netcdf4, geotiff, zarr.zip, zarr. The latter creates an object storage view on the output using ZARR format.
  • name=<name> - optional output name

The last form creates an object storage view on a dataset using ZARR format, so users can directly open entire data cubes, e.g.

from xcube.api import read_cube
cube = read_cube('https://myhost/api/datasets/s2a_10m/data.zarr') 
...

Describe alternatives you've considered

We may also implement an OGC CSW interface.

@forman forman added enhancement New feature or request urgent High external pressure to address this ASAP important This is very important for the project xcube serve This is related to server component, CLI "xcube serve" labels Jun 18, 2019
@forman forman self-assigned this Jun 18, 2019
@forman forman changed the title Allow xcube serve to deliver data xcube server to deliver data Jun 19, 2019
@forman
Copy link
Member Author

forman commented Jun 20, 2019

@dzelge and @AliceBalfanz, could you please review proposed API and provide feedback and / or new use cases.

@forman forman added the in progress The assignee is working on it label Jun 23, 2019
@forman
Copy link
Member Author

forman commented Jun 27, 2019

Added web API for data access, slightly different than proposed initially:

  • List bucket objects: /s3bucket, see AWS docs GET
  • Get bucket object: /s3bucket/{ds_id}/{path}, see AWS docs HEAD and GET

See #117

@forman forman removed the in progress The assignee is working on it label Aug 12, 2019
@forman forman removed their assignment Aug 12, 2019
@AliceBalfanz
Copy link
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
DCFS enhancement New feature or request important This is very important for the project urgent High external pressure to address this ASAP xcube serve This is related to server component, CLI "xcube serve"
Projects
None yet
Development

No branches or pull requests

2 participants