Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] org.opensearch.indices.IndicesRequestCacheIT.testDynamicStalenessThresholdUpdate is flaky #13949

Closed
shiv0408 opened this issue Jun 3, 2024 · 3 comments
Assignees
Labels
bug Something isn't working flaky-test Random test failure that succeeds on second run Search:Performance

Comments

@shiv0408
Copy link
Member

shiv0408 commented Jun 3, 2024

Describe the bug

Test org.opensearch.indices.IndicesRequestCacheIT.testDynamicStalenessThresholdUpdate is flaky

Stacktrace:

java.lang.AssertionError: expected:<0> but was:<690>
	at __randomizedtesting.SeedInfo.seed([A7FC852C3DDAA254:C4910ED3575EF8C]:0)
	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.Assert.failNotEquals(Assert.java:835)
	at org.junit.Assert.assertEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:633)
	at org.opensearch.indices.IndicesRequestCacheIT.lambda$testDynamicStalenessThresholdUpdate$7(IndicesRequestCacheIT.java:1071)
	at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1128)
	at org.opensearch.indices.IndicesRequestCacheIT.testDynamicStalenessThresholdUpdate(IndicesRequestCacheIT.java:1069)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
	at org.opensearch.test.OpenSearchTestClusterRule$1.evaluate(OpenSearchTestClusterRule.java:369)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:1583)
	Suppressed: java.lang.AssertionError: expected:<0> but was:<690>
		at org.junit.Assert.fail(Assert.java:89)
		at org.junit.Assert.failNotEquals(Assert.java:835)
		at org.junit.Assert.assertEquals(Assert.java:647)
		at org.junit.Assert.assertEquals(Assert.java:633)
		at org.opensearch.indices.IndicesRequestCacheIT.lambda$testDynamicStalenessThresholdUpdate$7(IndicesRequestCacheIT.java:1071)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1116)
		... 40 more

Related component

Search:Performance

To Reproduce

REPRODUCE WITH: ./gradlew ':server:internalClusterTest' --tests "org.opensearch.indices.IndicesRequestCacheIT" -Dtests.method="testDynamicStalenessThresholdUpdate {p0={"opensearch.experimental.feature.pluggable.caching.enabled":"false"}}" -Dtests.seed=A7FC852C3DDAA254 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=nl -Dtests.timezone=America/Creston -Druntime.java=21 NOTE: leaving temporary files on disk at: /var/jenkins/workspace/gradle-check/search/server/build/testrun/internalClusterTest/temp/org.opensearch.indices.IndicesRequestCacheIT_A7FC852C3DDAA254-001 NOTE: test params are: codec=Asserting(Lucene99): {index_uuid=PostingsFormat(name=Asserting), type=PostingsFormat(name=Asserting)}, docValues:{}, maxPointsInLeafNode=560, maxMBSortInHeap=5.6989961692108295, sim=Asserting(RandomSimilarity(queryNorm=true): {}), locale=nl, timezone=America/Creston
NOTE: Linux 5.15.0-1056-aws amd64/Eclipse Adoptium 21.0.3 (64-bit)/cpus=32,threads=1,free=211071080,total=536870912
NOTE: All tests run in this JVM: [CancellableTasksIT, ShrinkIndexIT, IndicesShardStoreRequestIT, GetTermVectorsIT, SpecificClusterManagerNodesIT, TemplateUpgradeServiceIT, ClusterManagerTaskThrottlingIT, NodeRepurposeCommandIT, SegmentReplicationPressureIT, PeerRecoveryRetentionLeaseCreationIT, IndicesRequestCacheIT]

Expected behavior

Test should always pass

Additional Details

No response

@shiv0408
Copy link
Member Author

shiv0408 commented Jun 3, 2024

@kiranprakash154 can you check if this can be reproduced?

@peternied
Copy link
Member

[Triage - attendees 1 2 3 4 5 6 7]
@shiv0408 Thanks for creating this issue, could you create a pull request to address this issue?

@peternied peternied added the flaky-test Random test failure that succeeds on second run label Jun 5, 2024
andrross added a commit to andrross/OpenSearch that referenced this issue Jun 7, 2024
andrross added a commit to andrross/OpenSearch that referenced this issue Jun 7, 2024
These are very frequent offenders and are causing a lot of pain in CI.

Related issues:
- opensearch-project#11374
- opensearch-project#12308
- opensearch-project#13540
- opensearch-project#13503
- opensearch-project#13711
- opensearch-project#13949
- opensearch-project#13437
- opensearch-project#13600

Signed-off-by: Andrew Ross <andrross@amazon.com>
andrross added a commit to andrross/OpenSearch that referenced this issue Jun 7, 2024
These are very frequent offenders and are causing a lot of pain in CI.

Related issues:

- opensearch-project#11374
- opensearch-project#12308
- opensearch-project#13540
- opensearch-project#13503
- opensearch-project#13711
- opensearch-project#13949
- opensearch-project#13437
- opensearch-project#13600

Signed-off-by: Andrew Ross <andrross@amazon.com>
andrross added a commit that referenced this issue Jun 7, 2024
These are very frequent offenders and are causing a lot of pain in CI.

Related issues:

- #11374
- #12308
- #13540
- #13503
- #13711
- #13949
- #13437
- #13600

Signed-off-by: Andrew Ross <andrross@amazon.com>
opensearch-trigger-bot bot pushed a commit that referenced this issue Jun 7, 2024
These are very frequent offenders and are causing a lot of pain in CI.

Related issues:

- #11374
- #12308
- #13540
- #13503
- #13711
- #13949
- #13437
- #13600

Signed-off-by: Andrew Ross <andrross@amazon.com>
(cherry picked from commit fbe048f)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
andrross pushed a commit that referenced this issue Jun 10, 2024
These are very frequent offenders and are causing a lot of pain in CI.

Related issues:

- #11374
- #12308
- #13540
- #13503
- #13711
- #13949
- #13437
- #13600


(cherry picked from commit fbe048f)

Signed-off-by: Andrew Ross <andrross@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
parv0201 pushed a commit to parv0201/OpenSearch that referenced this issue Jun 10, 2024
@reta
Copy link
Collaborator

reta commented Jun 19, 2024

Closing in favour of #14288

@reta reta closed this as completed Jun 19, 2024
@github-project-automation github-project-automation bot moved this from 🆕 New to ✅ Done in Search Project Board Jun 19, 2024
kkewwei pushed a commit to kkewwei/OpenSearch that referenced this issue Jul 24, 2024
…opensearch-project#14082)

These are very frequent offenders and are causing a lot of pain in CI.

Related issues:

- opensearch-project#11374
- opensearch-project#12308
- opensearch-project#13540
- opensearch-project#13503
- opensearch-project#13711
- opensearch-project#13949
- opensearch-project#13437
- opensearch-project#13600

(cherry picked from commit fbe048f)

Signed-off-by: Andrew Ross <andrross@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Signed-off-by: kkewwei <kkewwei@163.com>
wdongyu pushed a commit to wdongyu/OpenSearch that referenced this issue Aug 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working flaky-test Random test failure that succeeds on second run Search:Performance
Projects
Status: Done
Development

No branches or pull requests

4 participants