Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Directly stream some exports data #846

Open
grololo06 opened this issue Feb 4, 2023 · 4 comments
Open

Directly stream some exports data #846

grololo06 opened this issue Feb 4, 2023 · 4 comments
Assignees
Labels
ecopart Relates to EcoPart/particle data feature New functionality page-export Everything related to export functionality

Comments

@grololo06
Copy link
Member

grololo06 commented Feb 4, 2023

Present API https://ecotaxa.obs-vlfr.fr/api/redoc#tag/objects/operation/export_object_set only allows to start a background job and get the needed data. So, on client side, it's several steps to get the result: start the job, monitor the job, download the result, extract from the result.
Present issue would allow to get, for summary exports, the data directly as the API call response.
This would allow a simpler interaction, present a datasource for e.g. EcoPart, and allow neat GUI features such as e.g. export preview inside a browser.
Note: Under the condition of good performance. Users should not wait 10 minutes for their data to appear.

@grololo06 grololo06 self-assigned this Feb 4, 2023
@grololo06 grololo06 added the ecopart Relates to EcoPart/particle data label Feb 4, 2023
@grololo06 grololo06 added the page-export Everything related to export functionality label Feb 4, 2023
@grololo06
Copy link
Member Author

@juliecoust if you think about a specific output format feel free to tell.

@grololo06 grololo06 added the feature New functionality label Feb 4, 2023
@moi90
Copy link

moi90 commented Feb 6, 2023

Presumably, this would only include metadata, not images, right?

JSON itself is not really streamable. Maybe multipart with one JSON for each object (or a batch of objects)?
Then again: With multipart, one could also include images...

@grololo06
Copy link
Member Author

Hello, it would be for "summary exports" for the moment, i.e. computations around abundance, concentrations, biovolumes. Pls see recently solved issues on the topic. Indeed it's not really "streaming" as the data volume is quite low.

@moi90
Copy link

moi90 commented Feb 7, 2023

Ah, ok :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ecopart Relates to EcoPart/particle data feature New functionality page-export Everything related to export functionality
Projects
None yet
Development

No branches or pull requests

4 participants