-
Notifications
You must be signed in to change notification settings - Fork 2.3k
Remove the setShardIndex parameter in CollapseTopFieldDocs.merge() #19053
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
❌ Gradle check result for 4d4d71f: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Binlong Gao <gbinlong@amazon.com>
Signed-off-by: Binlong Gao <gbinlong@amazon.com>
Signed-off-by: Binlong Gao <gbinlong@amazon.com>
Signed-off-by: Binlong Gao <gbinlong@amazon.com>
Signed-off-by: Binlong Gao <gbinlong@amazon.com>
|
❌ Gradle check result for fb3b44d: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Binlong Gao <gbinlong@amazon.com>
|
❌ Gradle check result for 65f8d3b: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #19053 +/- ##
============================================
- Coverage 72.98% 72.83% -0.16%
+ Complexity 69475 69436 -39
============================================
Files 5647 5647
Lines 319210 319206 -4
Branches 46171 46170 -1
============================================
- Hits 232977 232493 -484
- Misses 67385 67875 +490
+ Partials 18848 18838 -10 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Hi @reta , mind taking a look at this PR? Thanks! |
server/src/main/java/org/apache/lucene/search/grouping/CollapseTopFieldDocs.java
Show resolved
Hide resolved
Thanks @gaobinlong , LGTM, I just have one question / comment to understand what potential impact it could have, thank you. |
…pensearch-project#19053) * Do not set shardIndex of top docs in CollapsingTopDocsCollectorContext Signed-off-by: Binlong Gao <gbinlong@amazon.com> * Modify change log Signed-off-by: Binlong Gao <gbinlong@amazon.com> * Format code Signed-off-by: Binlong Gao <gbinlong@amazon.com> * Remove setShardIndex parameter in CollapseTopFieldDocs.merge() Signed-off-by: Binlong Gao <gbinlong@amazon.com> * Modify change log Signed-off-by: Binlong Gao <gbinlong@amazon.com> * tiny change Signed-off-by: Binlong Gao <gbinlong@amazon.com> --------- Signed-off-by: Binlong Gao <gbinlong@amazon.com>
…pensearch-project#19053) * Do not set shardIndex of top docs in CollapsingTopDocsCollectorContext Signed-off-by: Binlong Gao <gbinlong@amazon.com> * Modify change log Signed-off-by: Binlong Gao <gbinlong@amazon.com> * Format code Signed-off-by: Binlong Gao <gbinlong@amazon.com> * Remove setShardIndex parameter in CollapseTopFieldDocs.merge() Signed-off-by: Binlong Gao <gbinlong@amazon.com> * Modify change log Signed-off-by: Binlong Gao <gbinlong@amazon.com> * tiny change Signed-off-by: Binlong Gao <gbinlong@amazon.com> --------- Signed-off-by: Binlong Gao <gbinlong@amazon.com>
…pensearch-project#19053) * Do not set shardIndex of top docs in CollapsingTopDocsCollectorContext Signed-off-by: Binlong Gao <gbinlong@amazon.com> * Modify change log Signed-off-by: Binlong Gao <gbinlong@amazon.com> * Format code Signed-off-by: Binlong Gao <gbinlong@amazon.com> * Remove setShardIndex parameter in CollapseTopFieldDocs.merge() Signed-off-by: Binlong Gao <gbinlong@amazon.com> * Modify change log Signed-off-by: Binlong Gao <gbinlong@amazon.com> * tiny change Signed-off-by: Binlong Gao <gbinlong@amazon.com> --------- Signed-off-by: Binlong Gao <gbinlong@amazon.com>
…pensearch-project#19053) * Do not set shardIndex of top docs in CollapsingTopDocsCollectorContext Signed-off-by: Binlong Gao <gbinlong@amazon.com> * Modify change log Signed-off-by: Binlong Gao <gbinlong@amazon.com> * Format code Signed-off-by: Binlong Gao <gbinlong@amazon.com> * Remove setShardIndex parameter in CollapseTopFieldDocs.merge() Signed-off-by: Binlong Gao <gbinlong@amazon.com> * Modify change log Signed-off-by: Binlong Gao <gbinlong@amazon.com> * tiny change Signed-off-by: Binlong Gao <gbinlong@amazon.com> --------- Signed-off-by: Binlong Gao <gbinlong@amazon.com> Signed-off-by: Ankit Jain <jainankitk@apache.org>
…pensearch-project#19053) * Do not set shardIndex of top docs in CollapsingTopDocsCollectorContext Signed-off-by: Binlong Gao <gbinlong@amazon.com> * Modify change log Signed-off-by: Binlong Gao <gbinlong@amazon.com> * Format code Signed-off-by: Binlong Gao <gbinlong@amazon.com> * Remove setShardIndex parameter in CollapseTopFieldDocs.merge() Signed-off-by: Binlong Gao <gbinlong@amazon.com> * Modify change log Signed-off-by: Binlong Gao <gbinlong@amazon.com> * tiny change Signed-off-by: Binlong Gao <gbinlong@amazon.com> --------- Signed-off-by: Binlong Gao <gbinlong@amazon.com> Signed-off-by: Ankit Jain <jainankitk@apache.org>
…pensearch-project#19053) * Do not set shardIndex of top docs in CollapsingTopDocsCollectorContext Signed-off-by: Binlong Gao <gbinlong@amazon.com> * Modify change log Signed-off-by: Binlong Gao <gbinlong@amazon.com> * Format code Signed-off-by: Binlong Gao <gbinlong@amazon.com> * Remove setShardIndex parameter in CollapseTopFieldDocs.merge() Signed-off-by: Binlong Gao <gbinlong@amazon.com> * Modify change log Signed-off-by: Binlong Gao <gbinlong@amazon.com> * tiny change Signed-off-by: Binlong Gao <gbinlong@amazon.com> --------- Signed-off-by: Binlong Gao <gbinlong@amazon.com>
…pensearch-project#19053) * Do not set shardIndex of top docs in CollapsingTopDocsCollectorContext Signed-off-by: Binlong Gao <gbinlong@amazon.com> * Modify change log Signed-off-by: Binlong Gao <gbinlong@amazon.com> * Format code Signed-off-by: Binlong Gao <gbinlong@amazon.com> * Remove setShardIndex parameter in CollapseTopFieldDocs.merge() Signed-off-by: Binlong Gao <gbinlong@amazon.com> * Modify change log Signed-off-by: Binlong Gao <gbinlong@amazon.com> * tiny change Signed-off-by: Binlong Gao <gbinlong@amazon.com> --------- Signed-off-by: Binlong Gao <gbinlong@amazon.com>
Description
In Lucene 9.0.0, this change was introduced that removes the ability of TopDocs.merge to set shard indices, and we adopted the change in 2.0.0, but CollapseTopFieldDocs.merge() still supports setting shard indices, that should also be removed.
As a result, when collapsing search results with concurrent segment search enabled, the shardIndex of the top docs are set twice in both queryPhase and FetchSearchPhase, that causes incorrect result are collected in queryPhase and later triggers an assertion error.
Actually, setting shard indices in queryPhase when calling CollapseTopFieldDocs.merge() is not needed because we are merging segment level results, they are all from one shard, we can only compare the lucene doc id when tie breaking for same sort value, small doc id wins.
This PR removes the setShardIndex parameter in CollapseTopFieldDocs.merge() method, following the change of the Lucene PR: apache/lucene-solr#757.
Related Issues
Resolves #19051 and #19111.
Check List
- [ ] API changes companion pull request created, if applicable.
- [ ] Public documentation issue/PR created, if applicable.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.