Remove unnecessary refresh listeners from NRTReplicationReaderManager.

[mch2]

Description

This change removes RefreshListeners used by IndexShard & InternalEngine to provide waitFor functionality from NRT replicas. These listeners were previously registered onto NRT replicas only to be force released on the next scheduled refresh cycle without actually refreshing the reader.

This change also removes the unnecessary blocking refresh from NRTReaderManager because we no longer have conflicting refresh invocations from scheduledRefresh, resolving the deadlock in #8858.

Added test to ensure refreshListeners are not registered. This is also covered by existing tests testDropPrimaryDuringReplication and testPrimaryRelocation.

Related Issues

Resolves #8858
related #8859

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Commits are signed per the DCO using --signoff
• Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: UNSTABLE ❕
• TEST FAILURES:

      1 org.opensearch.indices.replication.SegmentReplicationRelocationIT.testPrimaryRelocation

• URL: https://build.ci.opensearch.org/job/gradle-check/20866/
• CommitID: 320723a
  Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[codecov]

Codecov Report

Merging #8859 (d5971fd) into main (5d78de6) will increase coverage by 0.06%.
The diff coverage is 74.19%.

@@             Coverage Diff              @@
##               main    #8859      +/-   ##
============================================
+ Coverage     70.98%   71.05%   +0.06%     
- Complexity    57166    57181      +15     
============================================
  Files          4757     4757              
  Lines        269823   269822       -1     
  Branches      39474    39477       +3     
============================================
+ Hits         191536   191709     +173     
+ Misses        62158    61964     -194     
- Partials      16129    16149      +20     

Files Changed	Coverage Δ
.../opensearch/index/engine/NRTReplicationEngine.java	79.47% <0.00%> (+1.69%)	⬆️
...org/opensearch/index/seqno/ReplicationTracker.java	68.36% <74.07%> (-0.91%)	⬇️
...arch/index/engine/NRTReplicationReaderManager.java	96.96% <100.00%> (ø)
...in/java/org/opensearch/index/shard/IndexShard.java	69.57% <100.00%> (-0.17%)	⬇️
.../replication/checkpoint/ReplicationCheckpoint.java	87.71% <100.00%> (+2.00%)	⬆️

... and 445 files with indirect coverage changes

[mch2]

Gradle Check (Jenkins) Run Completed with:

• RESULT: UNSTABLE ❕
• TEST FAILURES:

      1 org.opensearch.indices.replication.SegmentReplicationRelocationIT.testPrimaryRelocation

• URL: https://build.ci.opensearch.org/job/gradle-check/20866/
• CommitID: 320723a
  Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

This test is failing with a timeout over 1m for relocation to execute. I've made a change to reduce the amount of docs ingested with testPrimaryRelocation and testPrimaryRelocationWithSegRepFailure.

These tests were ingesting 100-1k docs and randomly selecting a refresh policy. With the IMMEDIATE refresh policy a blocking refresh is performed that acquires a primary operation permit and increases the time required for the primary to block operations for relocation. On my machine this change reduces the test time with max docs from 1m to 5-6s.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: UNSTABLE ❕
• TEST FAILURES:

      1 org.opensearch.indices.replication.SegmentReplicationIT.testScrollCreatedOnReplica
      1 org.opensearch.index.ShardIndexingPressureSettingsIT.testShardIndexingPressureEnforcedEnabledDisabledSetting

• URL: https://build.ci.opensearch.org/job/gradle-check/20875/
• CommitID: d5971fd
  Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[mch2]

#7556