Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Throttle Initialization-fetching #164

Open
Qqwy opened this issue Apr 23, 2018 · 4 comments
Open

Throttle Initialization-fetching #164

Qqwy opened this issue Apr 23, 2018 · 4 comments
Labels
back-end This issue is part of the back-end (Python part) of the application. enhancement New feature or request nice to have Lower priority, nice to have features
Milestone

Comments

@Qqwy
Copy link
Collaborator

Qqwy commented Apr 23, 2018

To make sure we don't open up our application for a DDoS, we should fetch data slowly, also during the initial initialization-fetch: Fetch in small batches of data, and then take a minute before fetching the next batch.

@InputUsername
Copy link
Contributor

In addition, we should not start fetching new data until historical data is retrieved, otherwise EndpointCalls will end up in the wrong order.

@Qqwy Qqwy added bug Something isn't working enhancement New feature or request back-end This issue is part of the back-end (Python part) of the application. labels Apr 24, 2018
@Qqwy Qqwy added this to the Sprint #5 milestone Apr 24, 2018
@Qqwy
Copy link
Collaborator Author

Qqwy commented Apr 25, 2018

@InputUsername That second notice is implemented in PR #176

@TheVeggydude TheVeggydude modified the milestones: Sprint #5, Sprint #6 May 14, 2018
@ArjanTilstra ArjanTilstra modified the milestones: Sprint #6, Sprint #7 May 24, 2018
@InputUsername InputUsername added nice to have Lower priority, nice to have features and removed bug Something isn't working labels Jun 4, 2018
@TheVeggydude TheVeggydude removed this from the Sprint #7 milestone Jun 18, 2018
@Qqwy
Copy link
Collaborator Author

Qqwy commented Jun 18, 2018

This is nice-to-have, but currently the one thing that blocks adding Zeeguu to the project.

@InputUsername
Copy link
Contributor

Implementing this would allow the Zeeguu dashboard to be added but would also break seeding; since the function to fetch historic data would instead fetch part of the historic data, and then be ran as a periodic task while fetching data. Seeding does not use periodic tasks and therefore cannot support the new function, and leaving in the old version of the function would be code duplication.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
back-end This issue is part of the back-end (Python part) of the application. enhancement New feature or request nice to have Lower priority, nice to have features
Projects
None yet
Development

No branches or pull requests

4 participants