Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flaky test: ResponseCancelTest #952

Closed
idelpivnitskiy opened this issue Feb 28, 2020 · 7 comments
Closed

Flaky test: ResponseCancelTest #952

idelpivnitskiy opened this issue Feb 28, 2020 · 7 comments
Labels
flaky tests Unit tests are flaky

Comments

@idelpivnitskiy
Copy link
Member

idelpivnitskiy commented Feb 28, 2020 •
edited
Loading

https://ci.servicetalk.io/job/servicetalk-java11-prb/1028/testReport/junit/io.servicetalk.http.netty/ResponseCancelTest/cancel/

io.servicetalk.http.netty.ResponseCancelTest.cancel

Error Message

java.lang.AssertionError: Unexpected response success.
Expected: an instance of java.nio.channels.ClosedChannelException
     but: null
Stacktrace

java.lang.AssertionError: Unexpected response success.
Expected: an instance of java.nio.channels.ClosedChannelException
     but: null
	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
	at io.servicetalk.http.netty.ResponseCancelTest$ClientTerminationSignal.resumeExpectFailure(ResponseCancelTest.java:236)
	at io.servicetalk.http.netty.ResponseCancelTest.cancel(ResponseCancelTest.java:122)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at io.servicetalk.concurrent.internal.ServiceTalkTestTimeout$TimeoutStatement$CallableStatement.call(ServiceTalkTestTimeout.java:171)
	at io.servicetalk.concurrent.internal.ServiceTalkTestTimeout$TimeoutStatement$CallableStatement.call(ServiceTalkTestTimeout.java:163)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.lang.Thread.run(Thread.java:834)
@idelpivnitskiy idelpivnitskiy added the flaky tests Unit tests are flaky label Feb 28, 2020
@idelpivnitskiy idelpivnitskiy mentioned this issue Feb 28, 2020
Merged
@NiteshKant
Copy link
Collaborator

The test passes locally pretty consistently (run until failure in IDE). Lets keep an eye on this and see how flaky it is.

@idelpivnitskiy idelpivnitskiy changed the title Flaky test: ResponseCancelTest.cancel Flaky test: ResponseCancelTest Feb 29, 2020
@idelpivnitskiy
Copy link
Member Author

@NiteshKant it happens frequently on CI and cancelAfterSuccessOnTransport test also fails with assertion error:

  1. https://ci.servicetalk.io/job/servicetalk-java8-prb/1030/testReport/junit/io.servicetalk.http.netty/ResponseCancelTest/cancelAfterSuccessOnTransport/
io.servicetalk.http.netty.ResponseCancelTest.cancelAfterSuccessOnTransport

Error Message

java.lang.AssertionError: Unexpected response success.
Expected: an instance of java.nio.channels.ClosedChannelException
     but: null
Stacktrace

java.lang.AssertionError: Unexpected response success.
Expected: an instance of java.nio.channels.ClosedChannelException
     but: null
	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
	at io.servicetalk.http.netty.ResponseCancelTest$ClientTerminationSignal.resumeExpectFailure(ResponseCancelTest.java:236)
	at io.servicetalk.http.netty.ResponseCancelTest.cancelAfterSuccessOnTransport(ResponseCancelTest.java:139)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at io.servicetalk.concurrent.internal.ServiceTalkTestTimeout$TimeoutStatement$CallableStatement.call(ServiceTalkTestTimeout.java:171)
	at io.servicetalk.concurrent.internal.ServiceTalkTestTimeout$TimeoutStatement$CallableStatement.call(ServiceTalkTestTimeout.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.lang.Thread.run(Thread.java:748)
  1. https://ci.servicetalk.io/job/servicetalk-java11-prb/1031/testReport/junit/io.servicetalk.http.netty/ResponseCancelTest/cancel/
io.servicetalk.http.netty.ResponseCancelTest.cancel

Error Message

java.lang.AssertionError: Unexpected response success.
Expected: an instance of java.nio.channels.ClosedChannelException
     but: null
Stacktrace

java.lang.AssertionError: Unexpected response success.
Expected: an instance of java.nio.channels.ClosedChannelException
     but: null
	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
	at io.servicetalk.http.netty.ResponseCancelTest$ClientTerminationSignal.resumeExpectFailure(ResponseCancelTest.java:236)
	at io.servicetalk.http.netty.ResponseCancelTest.cancel(ResponseCancelTest.java:122)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at io.servicetalk.concurrent.internal.ServiceTalkTestTimeout$TimeoutStatement$CallableStatement.call(ServiceTalkTestTimeout.java:171)
	at io.servicetalk.concurrent.internal.ServiceTalkTestTimeout$TimeoutStatement$CallableStatement.call(ServiceTalkTestTimeout.java:163)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.lang.Thread.run(Thread.java:834)

@NiteshKant
Copy link
Collaborator

I see, thanks. Let me look at the test

@idelpivnitskiy
Copy link
Member Author

https://ci.servicetalk.io/job/servicetalk-java8-prb/1033/testReport/io.servicetalk.http.netty/ResponseCancelTest/cancelAfterSuccessOnTransport/

io.servicetalk.http.netty.ResponseCancelTest.cancelAfterSuccessOnTransport

Error Message
java.lang.AssertionError: Unexpected response success.
Expected: an instance of java.nio.channels.ClosedChannelException
     but: null
Stacktrace
java.lang.AssertionError: Unexpected response success.
Expected: an instance of java.nio.channels.ClosedChannelException
     but: null
	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
	at io.servicetalk.http.netty.ResponseCancelTest$ClientTerminationSignal.resumeExpectFailure(ResponseCancelTest.java:236)
	at io.servicetalk.http.netty.ResponseCancelTest.cancelAfterSuccessOnTransport(ResponseCancelTest.java:139)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at io.servicetalk.concurrent.internal.ServiceTalkTestTimeout$TimeoutStatement$CallableStatement.call(ServiceTalkTestTimeout.java:171)
	at io.servicetalk.concurrent.internal.ServiceTalkTestTimeout$TimeoutStatement$CallableStatement.call(ServiceTalkTestTimeout.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.lang.Thread.run(Thread.java:748)

@idelpivnitskiy
Copy link
Member Author

@NiteshKant do you mind if I will submit a PR to ignore ResponseCancelTest for now? Otherwise, all PRBs are failing.

@NiteshKant
Copy link
Collaborator

#954 should fix the failures

@idelpivnitskiy
Copy link
Member Author

Confirm, #954 resolved this issue.

@Scottmitch Scottmitch mentioned this issue Jul 29, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
flaky tests Unit tests are flaky
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@NiteshKant @idelpivnitskiy