[BUG] org.opensearch.http.SearchRestCancellationIT.testAutomaticCancellationMultiSearchDuringQueryPhase test is flaky #13674

shiv0408 · 2024-05-14T22:58:26Z

Describe the bug

Test org.opensearch.http.SearchRestCancellationIT.testAutomaticCancellationMultiSearchDuringQueryPhase is flaky

Stack trace:

java.lang.AssertionError
	at org.junit.Assert.fail(Assert.java:87)
	at org.junit.Assert.assertTrue(Assert.java:42)
	at org.junit.Assert.assertTrue(Assert.java:53)
	at org.opensearch.http.SearchRestCancellationIT.lambda$ensureSearchTaskIsCancelled$0(SearchRestCancellationIT.java:222)
	at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1128)
	at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1101)
	at org.opensearch.http.SearchRestCancellationIT.ensureSearchTaskIsCancelled(SearchRestCancellationIT.java:218)
	at org.opensearch.http.SearchRestCancellationIT.verifyCancellationDuringQueryPhase(SearchRestCancellationIT.java:140)
	at org.opensearch.http.SearchRestCancellationIT.testAutomaticCancellationMultiSearchDuringQueryPhase(SearchRestCancellationIT.java:114)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
	at org.opensearch.test.OpenSearchTestClusterRule$1.evaluate(OpenSearchTestClusterRule.java:369)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:1583)
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.http.SearchRestCancellationIT.lambda$ensureSearchTaskIsCancelled$0(SearchRestCancellationIT.java:222)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1116)
		... 43 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.http.SearchRestCancellationIT.lambda$ensureSearchTaskIsCancelled$0(SearchRestCancellationIT.java:222)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1116)
		... 43 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.http.SearchRestCancellationIT.lambda$ensureSearchTaskIsCancelled$0(SearchRestCancellationIT.java:222)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1116)
		... 43 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.http.SearchRestCancellationIT.lambda$ensureSearchTaskIsCancelled$0(SearchRestCancellationIT.java:222)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1116)
		... 43 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.http.SearchRestCancellationIT.lambda$ensureSearchTaskIsCancelled$0(SearchRestCancellationIT.java:222)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1116)
		... 43 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.http.SearchRestCancellationIT.lambda$ensureSearchTaskIsCancelled$0(SearchRestCancellationIT.java:222)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1116)
		... 43 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.http.SearchRestCancellationIT.lambda$ensureSearchTaskIsCancelled$0(SearchRestCancellationIT.java:222)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1116)
		... 43 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.http.SearchRestCancellationIT.lambda$ensureSearchTaskIsCancelled$0(SearchRestCancellationIT.java:222)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1116)
		... 43 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.http.SearchRestCancellationIT.lambda$ensureSearchTaskIsCancelled$0(SearchRestCancellationIT.java:222)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1116)
		... 43 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.http.SearchRestCancellationIT.lambda$ensureSearchTaskIsCancelled$0(SearchRestCancellationIT.java:222)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1116)
		... 43 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.http.SearchRestCancellationIT.lambda$ensureSearchTaskIsCancelled$0(SearchRestCancellationIT.java:222)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1116)
		... 43 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.http.SearchRestCancellationIT.lambda$ensureSearchTaskIsCancelled$0(SearchRestCancellationIT.java:222)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1116)
		... 43 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.http.SearchRestCancellationIT.lambda$ensureSearchTaskIsCancelled$0(SearchRestCancellationIT.java:222)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1116)
		... 43 more

Related component

Search

To Reproduce

./gradlew ':qa:smoke-test-http:integTest' --tests "org.opensearch.http.SearchRestCancellationIT.testAutomaticCancellationMultiSearchDuringQueryPhase" -Dtests.seed=8BE32D75FF21B14A -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=mk-MK -Dtests.timezone=Asia/Phnom_Penh -Druntime.java=21

Expected behavior

Test should always pass

Additional Details

CI failing in unrelated change: https://build.ci.opensearch.org/job/gradle-check/38710/testReport/org.opensearch.http/SearchRestCancellationIT/testAutomaticCancellationMultiSearchDuringQueryPhase_2/

The text was updated successfully, but these errors were encountered:

peternied · 2024-05-15T15:17:48Z

[Triage - attendees 1 2 3 4 5 6 7 8]
@shiv0408 Thanks for creating this issue to track this flaky test case

getsaurabh02 · 2024-05-15T17:06:50Z

@kaushalmahi12 can we try to reproduce this and see if its flakiness or any functional issue with the cancellation.

reta · 2024-06-19T15:54:01Z

Closing in favour of #14311

shiv0408 added bug Something isn't working untriaged labels May 14, 2024

github-actions bot added the Search Search query, autocomplete ...etc label May 14, 2024

shiv0408 mentioned this issue May 14, 2024

Optimize remote state stale file deletion #13131

Merged

7 tasks

peternied added flaky-test Random test failure that succeeds on second run and removed untriaged labels May 15, 2024

getsaurabh02 assigned kaushalmahi12 May 15, 2024

akolarkunnu mentioned this issue Jun 14, 2024

COMPAT locale provider will be removed in a future release #14345

Merged

9 tasks

kkewwei mentioned this issue Jun 18, 2024

fix FuzzyQuery in keyword field will use IndexOrDocValuesQuery when both of index and doc_value are true #14378

Merged

3 tasks

reta closed this as completed Jun 19, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG] org.opensearch.http.SearchRestCancellationIT.testAutomaticCancellationMultiSearchDuringQueryPhase test is flaky #13674

[BUG] org.opensearch.http.SearchRestCancellationIT.testAutomaticCancellationMultiSearchDuringQueryPhase test is flaky #13674

shiv0408 commented May 14, 2024

peternied commented May 15, 2024

getsaurabh02 commented May 15, 2024

reta commented Jun 19, 2024

[BUG] org.opensearch.http.SearchRestCancellationIT.testAutomaticCancellationMultiSearchDuringQueryPhase test is flaky #13674

[BUG] org.opensearch.http.SearchRestCancellationIT.testAutomaticCancellationMultiSearchDuringQueryPhase test is flaky #13674

Comments

shiv0408 commented May 14, 2024

Describe the bug

Related component

To Reproduce

Expected behavior

Additional Details

peternied commented May 15, 2024

getsaurabh02 commented May 15, 2024

reta commented Jun 19, 2024