Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Potential optimisations for requests that do not ask for a filter #2093

Open
ml-evs opened this issue Jun 12, 2024 · 2 comments
Open

Potential optimisations for requests that do not ask for a filter #2093

ml-evs opened this issue Jun 12, 2024 · 2 comments
Labels
enhancement New feature or request suggestions

Comments

@ml-evs
Copy link
Member

ml-evs commented Jun 12, 2024

e.g., on server startup, take a series of chunked blobs corresponding to the default sort order of the API, then if a user requests with no filter and no page limit, return this static chunked data directly.

cc @JonathanSchmidt1 @eimrek (maybe this is a useful optimisation for e.g., the Li-ion set too)

@ml-evs ml-evs added enhancement New feature or request suggestions labels Jun 12, 2024
@eimrek
Copy link
Member

eimrek commented Jun 12, 2024

thanks @ml-evs, with basic indexes in mongodb, the default load can be made fairly fast except for the total number of structures (data_returned). Might be worth adding this just for that number. Or alternatively use the mongodb estimated document count for the filterless query (but i guess this can be wrong is some very rare cases?)

@ml-evs ml-evs closed this as not planned Won't fix, can't repro, duplicate, stale Jun 13, 2024
@ml-evs ml-evs reopened this Jun 13, 2024
@ml-evs
Copy link
Member Author

ml-evs commented Jun 13, 2024

Maybe this is a good idea after all, if a web client always hits the same request, we could cache these as JSON files that are served statically rather than via the API, e.g., via the config file:

"static_index": ["?page_limit=100&response_fields=id&page_offset=0"]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request suggestions
Projects
None yet
Development

No branches or pull requests

2 participants