[CI] FileSettingsRoleMappingsRestartIT testFileSettingsReprocessedOnRestartWithoutVersionChange failing

[elasticsearchmachine]

Build Scans:

• elasticsearch-periodic-platform-support #6057 / oraclelinux-8_platform-support-unix
• elasticsearch-periodic-platform-support #6057 / almalinux-8-aarch64_checkpart4_platform-support-arm
• elasticsearch-pull-request #53671 / part-4
• elasticsearch-pull-request #53683 / part-4
• elasticsearch-pull-request #53584 / part-4
• elasticsearch-intake #16646 / part4
• elasticsearch-pull-request #53567 / part-4
• elasticsearch-intake #16613 / part4
• elasticsearch-pull-request #53491 / part-4
• elasticsearch-pull-request #53461 / part-4

Reproduction Line:

./gradlew ":x-pack:plugin:security:internalClusterTest" --tests "org.elasticsearch.xpack.security.FileSettingsRoleMappingsRestartIT.testFileSettingsReprocessedOnRestartWithoutVersionChange" -Dtests.seed=FEE684EC18129042 -Dtests.locale=ebu-KE -Dtests.timezone=AST -Druntime.java=23

Applicable branches:
main

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

java.lang.AssertionError: 
Expected: iterable with items [] in any order
     but: no match for: "everyone_kibana_together"

Issue Reasons:

• [main] 26 failures in test testFileSettingsReprocessedOnRestartWithoutVersionChange (4.9% fail rate in 526 executions)
• [main] 16 failures in step part-4 (6.3% fail rate in 252 executions)
• [main] 3 failures in step part4 (3.0% fail rate in 101 executions)
• [main] 2 failures in step encryption-at-rest (50.0% fail rate in 4 executions)
• [main] 3 failures in pipeline elasticsearch-periodic-platform-support (42.9% fail rate in 7 executions)
• [main] 17 failures in pipeline elasticsearch-pull-request (7.0% fail rate in 243 executions)
• [main] 3 failures in pipeline elasticsearch-intake (3.0% fail rate in 101 executions)
• [main] 2 failures in pipeline elasticsearch-periodic (28.6% fail rate in 7 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

[elasticsearchmachine]

This has been muted on branch 8.x

Mute Reasons:

• [8.x] 2 failures in test testFileSettingsReprocessedOnRestartWithoutVersionChange (1.4% fail rate in 145 executions)

Build Scans:

• elasticsearch-periodic #6046 / openjdk17_checkpart4_java-matrix
• elasticsearch-pull-request #52941 / part-4

[elasticsearchmachine]

Pinging @elastic/es-core-infra (Team:Core/Infra)

[ldematte]

Very similar to #120923: both started failing earlier (in this case, just 2 times, 13th of Dec and 20th of Jan), but had a "surge" in the last 2 days (even with no change to the related services, not that I'm aware of). Could be related to #118351 or #114295? (cc @n1v0lg and @prdoyle)

[elasticsearchmachine]

This has been muted on branch main

Mute Reasons:

• [main] 25 failures in test testFileSettingsReprocessedOnRestartWithoutVersionChange (4.9% fail rate in 512 executions)
• [main] 16 failures in step part-4 (6.5% fail rate in 247 executions)
• [main] 3 failures in step part4 (3.1% fail rate in 98 executions)
• [main] 2 failures in step encryption-at-rest (50.0% fail rate in 4 executions)
• [main] 3 failures in pipeline elasticsearch-periodic-platform-support (42.9% fail rate in 7 executions)
• [main] 17 failures in pipeline elasticsearch-pull-request (7.1% fail rate in 238 executions)
• [main] 3 failures in pipeline elasticsearch-intake (3.1% fail rate in 98 executions)
• [main] 2 failures in pipeline elasticsearch-periodic (33.3% fail rate in 6 executions)

Build Scans:

• elasticsearch-periodic-platform-support #6057 / almalinux-8-aarch64_checkpart4_platform-support-arm
• elasticsearch-pull-request #53671 / part-4
• elasticsearch-pull-request #53683 / part-4
• elasticsearch-pull-request #53584 / part-4
• elasticsearch-intake #16646 / part4
• elasticsearch-pull-request #53567 / part-4
• elasticsearch-intake #16613 / part4
• elasticsearch-pull-request #53491 / part-4
• elasticsearch-pull-request #53461 / part-4
• elasticsearch-intake #16574 / part4