-
Notifications
You must be signed in to change notification settings - Fork 104
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add sorting and pagination to bingo-elastic #1826
Comments
In progress PR for Python: #1712 |
uladkaminski
added a commit
to uladkaminski/Indigo
that referenced
this issue
Mar 15, 2024
Java changes PR: #1827 |
uladkaminski
added a commit
to uladkaminski/Indigo
that referenced
this issue
Mar 15, 2024
khyurri
pushed a commit
that referenced
this issue
Mar 18, 2024
uladkaminski
added a commit
to uladkaminski/Indigo
that referenced
this issue
Mar 26, 2024
8 tasks
Additional Java changes PR: #1880 |
khyurri
pushed a commit
that referenced
this issue
Apr 8, 2024
uladkaminski
added a commit
to uladkaminski/Indigo
that referenced
this issue
May 10, 2024
uladkaminski
added a commit
to uladkaminski/Indigo
that referenced
this issue
Aug 7, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Background
Currently, users can only access a limited count of compounds (up to 10,000 by default) from ElasticSearch. We need to incorporate sorting and pagination functionality into the bingo-elastic driver. This will allow users to receive more than 10,000 compounds from the index.
Solution
ElasticSearch offers a pagination feature (PIT). Visit https://www.elastic.co/guide/en/elasticsearch/reference/current/paginate-search-results.html to learn more.
We need to incorporate both the Java and Python functionality to enable transparent use of PIT, thus eliminating the need for user-level management. Therefore, if a user sets a sorting parameter, the bingo-elastic driver will automatically continue fetching documents until the limit is reached.
Importantly, after implementing this change, the limit option should be optional. This will allow users to download the entire index if they choose to do so.
The text was updated successfully, but these errors were encountered: