-
Notifications
You must be signed in to change notification settings - Fork 2.3k
Approximation framework to support numeric search_after queries
#18896
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Approximation framework to support numeric search_after queries
#18896
Conversation
Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com>
Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com>
Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com>
Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com>
Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com>
|
❌ Gradle check result for 3c6596a: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
server/src/main/java/org/opensearch/index/mapper/DateFieldMapper.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com>
|
❌ Gradle check result for 9137fe5: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com>
|
For some reason @kkewwei 's approval didn't count towards merge. Approving the PR while we figure out the codeowners issue. |
6468464
into
opensearch-project:main
…ensearch-project#18896) * Initial commit for search_after queries Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Test with increment and decrement with search_after Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * search_after queries Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Support framework for search_after queries Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Fix gradle precommit issues Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Add comments Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Attempt to fix the test Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Attempt to fix the test Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Update CHANGELOG.md Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Update CHANGELOG.md and fetch upstream Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Update tests validating with lucene searchAfter Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Update tests validating with lucene searchAfter Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Update tests validating with lucene searchAfter Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Update code with comments Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Add encode tests Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Fix conflicts Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Fix spotless Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Code clean up Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Upstream fetch Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Upstream Fetch Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Add clamps Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Upstream Fetch to resolve conflicts Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> --------- Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com>
…ensearch-project#18896) * Initial commit for search_after queries Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Test with increment and decrement with search_after Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * search_after queries Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Support framework for search_after queries Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Fix gradle precommit issues Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Add comments Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Attempt to fix the test Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Attempt to fix the test Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Update CHANGELOG.md Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Update CHANGELOG.md and fetch upstream Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Update tests validating with lucene searchAfter Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Update tests validating with lucene searchAfter Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Update tests validating with lucene searchAfter Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Update code with comments Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Add encode tests Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Fix conflicts Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Fix spotless Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Code clean up Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Upstream fetch Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Upstream Fetch Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Add clamps Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Upstream Fetch to resolve conflicts Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> --------- Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com>
…ensearch-project#18896) * Initial commit for search_after queries Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Test with increment and decrement with search_after Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * search_after queries Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Support framework for search_after queries Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Fix gradle precommit issues Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Add comments Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Attempt to fix the test Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Attempt to fix the test Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Update CHANGELOG.md Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Update CHANGELOG.md and fetch upstream Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Update tests validating with lucene searchAfter Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Update tests validating with lucene searchAfter Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Update tests validating with lucene searchAfter Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Update code with comments Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Add encode tests Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Fix conflicts Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Fix spotless Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Code clean up Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Upstream fetch Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Upstream Fetch Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Add clamps Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * Upstream Fetch to resolve conflicts Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> --------- Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com>
This PR targets to support
search_afterqueries from the solution described in this issue #18546.ASCsort and upper bound forDESCsort. Example:Adds new
byte[] encodePoint(Object value, boolean roundUp);inNumericPointEncoderas todaysearch_aftervalue is returned as object and requires converting it into the appropriate field specific byte[].search_afterwithASCsort we want values greater than thesearch_aftervalue and forDESCsort we want values less than thesearch_after.Handles the search_after conversation for dates with date math rounding operations, here is the sample query. Note the existing
parseToLongis for date math rounding operations and not for creatingsearch_afterbounds.Coming from this PR https://github.com/opensearch-project/OpenSearch/pull/18763/files where approximation code path is disabled for multi sort, hence
search_aftershould work as expected when dealing with multiple tie breakersearch_aftervaluesRegarding benchmarks following are the queries we should see improvement from the existing workloads
desc_sort_with_after_timestamp,asc_sort_with_after_timestampdesc_sort_with_after_timestamp,asc_sort_with_after_timestampasc_sort_with_after_population,desc_sort_with_after_geonameid,asc_sort_with_after_geonameidDescription
[Describe what this change achieves]
Related Issues
Resolves #18546
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.