[BUG] MinimumMasterNodesIT.testThreeNodesNoMasterBlock (random test failure)

[CEHENKLE]

Describe the bug
Random test failure :( Please dig in and figure out what went wrong :(

https://fork-jenkins.searchservices.aws.dev/job/OpenSearch_CI/job/PR_Checks/job/Gradle_Check/1079/artifact/gradle_check_1079.log/view/

[andrross]

Another failure (different test within same class) here: https://fork-jenkins.searchservices.aws.dev/job/OpenSearch_CI/job/PR_Checks/job/Gradle_Check/1429/artifact/gradle_check_1429.log/*view*/

./gradlew ':server:internalClusterTest' --tests "org.opensearch.cluster.MinimumMasterNodesIT.testTwoNodesNoMasterBlock" -Dtests.seed=8C423154E3EBD680 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=el-GR -Dtests.timezone=Asia/Aden -Druntime.java=17

[tlfeng]

The test can not be reproduced, and looks like the last failure occurred in CI was on 12/10/2021, it has been almost a year ago.
I ran the command to run the test for 1k times, and there were only 2 failure occurred due to 20-minute general test timeout (the timeout is mentioned in #1477 (comment)).
./gradlew ':server:internalClusterTest' --tests "org.opensearch.cluster.MinimumClusterManagerNodesIT.testTwoNodesNoClusterManagerBlock" -Dtests.iters=1000

REPRODUCE WITH: ./gradlew ':server:internalClusterTest' --tests "org.opensearch.cluster.MinimumClusterManagerNodesIT.testCannotCommitStateThreeNodes {seed=[894AC60C805AC961:A3A096FAFCD862A]}" -Dtests.seed=894AC60C805AC961 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=ar-EG -Dtests.timezone=America/Yellowknife -Druntime.java=17
  2> java.lang.Exception: Test abandoned because suite timeout was reached.
        at __randomizedtesting.SeedInfo.seed([894AC60C805AC961]:0)
...
BUILD FAILED in 40m 27s
43 actionable tasks: 1 executed, 42 up-to-date

[r1walz]

Hi, @andrross, @tlfeng. It looks same as #10006. Since, test run history isn't available on this issue. Shall we close it and track progress on #10006 instead?

[andrross]

Closing in favor of #10006