Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] ReindexFailureTests testResponseOnSearchFailure fails on CI #49295

Closed
tlrx opened this issue Nov 19, 2019 · 3 comments · Fixed by #49301 or #49423
Closed

[CI] ReindexFailureTests testResponseOnSearchFailure fails on CI #49295

tlrx opened this issue Nov 19, 2019 · 3 comments · Fixed by #49301 or #49423
Assignees
@henningandersen
Labels
:Distributed Indexing/Reindex Issues relating to reindex that are not caused by issues further down >test-failure Triaged test failures from CI v7.6.0 v8.0.0-alpha1

Comments

@tlrx
Copy link
Member

tlrx commented Nov 19, 2019 •
edited
Loading

The test ReindexFailureTests testResponseOnSearchFailure failed multiple on CI today on master and 7.x branches:
https://gradle-enterprise.elastic.co/s/xjodptpkel2tq
https://gradle-enterprise.elastic.co/s/bgg6g2npo4epu
https://gradle-enterprise.elastic.co/s/kh4jcenda3mvi
https://gradle-enterprise.elastic.co/s/yxfqraa4vukpi

It reproduces locally on 7.x with :

./gradlew ':modules:reindex:test' --tests "org.elasticsearch.index.reindex.ReindexFailureTests.testResponseOnSearchFailure" -Dtests.seed=47226BB65839127C -Dtests.security.manager=true -Dtests.locale=de-DE -Dtests.timezone=Asia/Brunei -Dcompiler.java=12 -Druntime.java=8
@tlrx tlrx added >test-failure Triaged test failures from CI v8.0.0 :Distributed Indexing/Reindex Issues relating to reindex that are not caused by issues further down v7.6.0 labels Nov 19, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (:Distributed/Reindex)

@tlrx tlrx changed the title [CI] [CI] ReindexFailureTests testResponseOnSearchFailure fails on CI Nov 19, 2019
tlrx added a commit to tlrx/elasticsearch that referenced this issue Nov 19, 2019
@tlrx
Mute ReindexFailureTests.testResponseOnSearchFailure
712cbcc 
Relates elastic#49295
@tlrx tlrx mentioned this issue Nov 19, 2019
Merged
@henningandersen henningandersen self-assigned this Nov 19, 2019
tlrx added a commit that referenced this issue Nov 19, 2019
@tlrx
Mute ReindexFailureTests.testResponseOnSearchFailure (#49298)
c3a7369 
Relates #49295
tlrx added a commit that referenced this issue Nov 19, 2019
@tlrx
Mute ReindexFailureTests.testResponseOnSearchFailure (#49298)
abed869 
Relates #49295
henningandersen added a commit to henningandersen/elasticsearch that referenced this issue Nov 19, 2019
@henningandersen
Reindex search response fix
92ce11e 
Fixed test case to also accept another error message, now that reindex
does not allow searching against red shards.

Closes elastic#49295
@henningandersen henningandersen mentioned this issue Nov 19, 2019
Merged
henningandersen added a commit that referenced this issue Nov 19, 2019
@henningandersen
Reindex search response fix (#49301)
f2cba4e 
Fixed test case to also accept another error message, now that reindex
does not allow searching against red shards.

Closes #49295
henningandersen added a commit that referenced this issue Nov 19, 2019
@henningandersen
Reindex search response fix (#49301)
bc29c98 
Fixed test case to also accept another error message, now that reindex
does not allow searching against red shards.

Closes #49295
@jimczi
Copy link
Contributor

jimczi commented Nov 20, 2019

Rreopening after another failure in 7x with a different string:

org.elasticsearch.index.reindex.ReindexFailureTests > testResponseOnSearchFailure FAILED
--
java.lang.AssertionError:
Expected: (((a string containing "all shards failed" or a string containing "No search context found") or a string containing "no such index [source]") or a string containing "Failed to execute phase [query], Partial shards failure")
but: was "RemoteTransportException[[node_s0][127.0.0.1:37333][indices:data/write/reindex]]; nested: SearchPhaseExecutionException[Partial shards failure (1 shards unavailable)];"
at __randomizedtesting.SeedInfo.seed([E04AF90BC989047C:DBC02D3DAD32AFF6]:0)
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
at org.junit.Assert.assertThat(Assert.java:956)
at org.junit.Assert.assertThat(Assert.java:923)
at org.elasticsearch.index.reindex.ReindexFailureTests.testResponseOnSearchFailure(ReindexFailureTests.java:118)
REPRODUCE WITH: ./gradlew ':modules:reindex:test' --tests "org.elasticsearch.index.reindex.ReindexFailureTests.testResponseOnSearchFailure" -Dtests.seed=E04AF90BC989047C -Dtests.security.manager=true -Dtests.locale=mt-MT -Dtests.timezone=Europe/Busingen -Dcompiler.java=12 -Druntime.java=8

@jimczi jimczi reopened this Nov 20, 2019
@markharwood
Copy link
Contributor

Another failure today on 7.x which reproduces

./gradlew ':modules:reindex:test' --tests "org.elasticsearch.index.reindex.ReindexFailureTests.testResponseOnSearchFailure" -Dtests.seed=1CD9FF092A5CCFA4 -Dtests.security.manager=true -Dtests.locale=es-UY -Dtests.timezone=Mexico/BajaNorte -Dcompiler.java=12 -Druntime.java=8

henningandersen added a commit to henningandersen/elasticsearch that referenced this issue Nov 21, 2019
@henningandersen
Reindex search response fix again
c077305 
Fixed test case to more broadly accept all messages with "Partial
shards failure" in it, to hopefully catch all relevant search messages
now that reindex does not allow searching against red shards.

Closes elastic#49295
@henningandersen henningandersen mentioned this issue Nov 21, 2019
Merged
henningandersen added a commit that referenced this issue Nov 21, 2019
@henningandersen
Reindex search response fix again (#49423)
aa1dc0d 
Fixed test case to more broadly accept all messages with "Partial
shards failure" in it, to hopefully catch all relevant search messages
now that reindex does not allow searching against red shards.

Closes #49295
henningandersen added a commit that referenced this issue Nov 21, 2019
@henningandersen
Reindex search response fix again (#49423)
0164de8 
Fixed test case to more broadly accept all messages with "Partial
shards failure" in it, to hopefully catch all relevant search messages
now that reindex does not allow searching against red shards.

Closes #49295
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@henningandersen henningandersen

Labels
:Distributed Indexing/Reindex Issues relating to reindex that are not caused by issues further down >test-failure Triaged test failures from CI v7.6.0 v8.0.0-alpha1
Projects
None yet
Milestone
No milestone
6 participants
@markharwood @tlrx @jakelandis @elasticmachine @jimczi @henningandersen