Skip to content

[CI] NodeTests.testAwaitCloseTimeoutsOnNonInterruptibleTask failure #44435

Closed
@matriv

Description

@matriv

CI link: https://scans.gradle.com/s/ioo3aduuax56w/console-log?task=:server:test Failed for 7.x

Cannot reproduce locally:

./gradlew :server:test --tests "org.elasticsearch.node.NodeTests.testAwaitCloseTimeoutsOnNonInterruptibleTask" \
  -Dtests.seed=4DD0445A73A34598 \
  -Dtests.security.manager=true \
  -Dtests.locale=fi-FI \
  -Dtests.timezone=America/Los_Angeles \
  -Dcompiler.java=12 \
  -Druntime.java=8

Stacktrace:


2> heinäkuuta 16, 2019 7:16:00 AP. com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
--
2> WARNING: Uncaught exception in thread: Thread[elasticsearch[mock_ node][generic][T#4],5,TGRP-NodeTests]
2> java.lang.IllegalStateException: threadcontext is already closed
2> 	at __randomizedtesting.SeedInfo.seed([4DD0445A73A34598]:0)
2> 	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextThreadLocal.ensureOpen(ThreadContext.java:673)
2> 	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextThreadLocal.get(ThreadContext.java:666)
2> 	at org.elasticsearch.common.util.concurrent.ThreadContext.newStoredContext(ThreadContext.java:206)
2> 	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.<init>(ThreadContext.java:693)
2> 	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.<init>(ThreadContext.java:688)
2> 	at org.elasticsearch.common.util.concurrent.ThreadContext.preserveContext(ThreadContext.java:383)
2> 	at org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor.wrapRunnable(EsThreadPoolExecutor.java:161)
2> 	at org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor.execute(EsThreadPoolExecutor.java:82)
2> 	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:244)
2> 	at org.elasticsearch.transport.InboundHandler.access$000(InboundHandler.java:43)
2> 	at org.elasticsearch.transport.InboundHandler$1.onFailure(InboundHandler.java:219)
2> 	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.onFailure(ThreadContext.java:758)
2> 	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:39)
2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
2> 	at java.lang.Thread.run(Thread.java:748)

and many occurrences of the following:

...
1> [2019-07-16T07:15:58,829][WARN ][o.e.d.HandshakingTransportAddressConnector] [mock_ node] handshake failed for [connectToRemoteMasterNode[[::1]:9302]]
--
1> java.lang.IllegalStateException: handshake failed, mismatched cluster name [Cluster [single-node-cluster-TEST_WORKER_VM=[38]-CLUSTER_SEED=[-8120033806674142137]-HASH=[12800ACF49D]]] - {[::1]:9302}{otZFAwAAQACZ0eRAAAAAAA}{0:0:0:0:0:0:0:1}{[::1]:9302}
1> 	at org.elasticsearch.transport.TransportService$5.onResponse(TransportService.java:461) [main/:?]
1> 	at org.elasticsearch.transport.TransportService$5.onResponse(TransportService.java:456) [main/:?]
1> 	at org.elasticsearch.action.ActionListenerResponseHandler.handleResponse(ActionListenerResponseHandler.java:54) [main/:?]
1> 	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleResponse(TransportService.java:1146) [main/:?]
1> 	at org.elasticsearch.transport.InboundHandler$1.doRun(InboundHandler.java:224) [main/:?]
1> 	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:773) [main/:?]
1> 	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [main/:?]
1> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_212]
1> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_212]
1> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212]
1> [2019-07-16T07:15:58,830][WARN ][o.e.d.HandshakingTransportAddressConnector] [mock_ node] handshake failed for [connectToRemoteMasterNode[127.0.0.1:9302]]
1> java.lang.IllegalStateException: handshake failed, mismatched cluster name [Cluster [single-node-cluster-TEST_WORKER_VM=[38]-CLUSTER_SEED=[-8120033806674142137]-HASH=[12800ACF49D]]] - {127.0.0.1:9302}{otZFAwAAQACZ0eRAAAAAAA}{127.0.0.1}{127.0.0.1:9302}
1> 	at org.elasticsearch.transport.TransportService$5.onResponse(TransportService.java:461) [main/:?]
1> 	at org.elasticsearch.transport.TransportService$5.onResponse(TransportService.java:456) [main/:?]
1> 	at org.elasticsearch.action.ActionListenerResponseHandler.handleResponse(ActionListenerResponseHandler.java:54) [main/:?]
1> 	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleResponse(TransportService.java:1146) [main/:?]
1> 	at org.elasticsearch.transport.InboundHandler$1.doRun(InboundHandler.java:224) [main/:?]
1> 	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:773) [main/:?]
1> 	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [main/:?]
1> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_212]
1> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_212]
1> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212]
...

Not muting, 1st occurrence ever.

Full logs can be found here

Metadata

Metadata

Assignees

Labels

:Core/Infra/CoreCore issues without another label>test-failureTriaged test failures from CI

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions