docCountError must be 0 for shard level reduce

[jed326]

Description

After #8860, the shard_size parameters are not applied until the shard level reduce. This means that no bucket candidates will be eliminated until the reduce phase, which also means there should be no doc_count_error at the slice level reduce. This change fixes that and the test failures in #9209

Related Issues

Resolves #9209

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Commits are signed per the DCO using --signoff
• Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[jed326]

@reta @sohami could you please help review? I didn't add a changelog line since this isn't modifying any existing behavior, I see it instead as a bug fix for an unreleased experimental feature. Thanks!

[opensearch-trigger-bot]

Compatibility status:

> Task :checkCompatibility
Incompatible components: [https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/reporting.git]
Compatible components: [https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/performance-analyzer-rca.git]

BUILD SUCCESSFUL in 21m 51s

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: UNSTABLE ❕
• TEST FAILURES:

      1 org.opensearch.search.SearchWeightedRoutingIT.testSearchAggregationWithNetworkDisruption_FailOpenEnabled

• URL: https://build.ci.opensearch.org/job/gradle-check/22413/
• CommitID: 9feb36a
  Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[codecov]

Codecov Report

Merging #9246 (0a709f3) into main (7278f43) will decrease coverage by 0.08%.
Report is 19 commits behind head on main.
The diff coverage is 50.00%.

@@             Coverage Diff              @@
##               main    #9246      +/-   ##
============================================
- Coverage     71.09%   71.01%   -0.08%     
+ Complexity    57364    57265      -99     
============================================
  Files          4770     4770              
  Lines        270548   270548              
  Branches      39563    39563              
============================================
- Hits         192334   192135     -199     
- Misses        62036    62157     +121     
- Partials      16178    16256      +78     

Files Changed	Coverage Δ
...earch/aggregations/bucket/terms/InternalTerms.java	87.76% <50.00%> (+0.42%)	⬆️

... and 475 files with indirect coverage changes

[jed326]

@reta @sohami updated the code comment. Let me know what you think!

[reta]

@reta @sohami updated the code comment. Let me know what you think!

LGTM but would prefer @sohami to signoff here, thanks @jed326 !

[opensearch-trigger-bot]

Compatibility status:

> Task :checkCompatibility
Incompatible components: [https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/reporting.git]
Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/ml-commons.git]

BUILD SUCCESSFUL in 23m 23s

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS ✅
• URL: https://build.ci.opensearch.org/job/gradle-check/22536/
• CommitID: 0a709f3

[sohami]

LGTM. Thanks @jed326