Pact consumer - when running a JUnit 4 spec with multiple tests every second pact test fails

[andrewspinks]

We are using pact-jvm-provider-gradle_2.11::3.2.1 to test the downstream dependency in a springboot app. We have found that in one of our specs that has multiple pact tests every second test fails saying that the expected requests were not received. If we run them individually each test works fine, and also when we put a 4 second delay into a @before, then they all pass. We have also tried running them in a different order and it is always every second test that fails, regardless.

We have tried using versions 3.3.2, 3.3.1, 3.2.13 and 3.1.3 and all have the same problem.

[uglyog]

I'm assuming that your test is not specifying a random port for the mock server?

[uglyog]

Or are you talking about verifying the pacts. You mentioned pact-jvm-provider-gradle and 'one of our specs that has multiple pact tests'?

[uglyog]

You have JUnit 4 spec in the title, so I'm assuming the issue is with the Junit library, not the groovy one?

[uglyog]

I was not able to reproduce it (of course ;-) ) so, could you provide an example test and logs?

Here is the test I created: Defect342MultiTest.groovy

[andrewspinks]

@uglyog Apologies, I copied the wrong dependency out of our gradle file. It should have been: au.com.dius:pact-jvm-consumer-groovy_2.11:3.2.1

I was trying to reproduce the issue in a simple environment when I noticed the problem has disappeared! Looking through our changes it seems that changing the configuration of our RestTemplate's requestFactory has magically made the problem go away. I'll dig a bit more to see if I can track down what was the underlying cause was...

[andrewspinks]

@uglyog Here is a failing test. https://github.com/andrewspinks/pact-error/blob/master/src/test/groovy/com/example/PactSpec.groovy I will try and simplify it further to take out the springboot stuff and see if it still fails.

[uglyog]

Awesome, thanks for the test project. Hopefully it is something as simple as requiring a wait for the mock server to shutdown at the end of the test.

[uglyog]

Using a random port for each test does not have this issue, so it prob needs a wait for the mock server to be ready before running the test.

[uglyog]

Ok, getting closer to the problem. It looks like an issue with the HTTP client, not the pact library.

From what I can gather, I think it is to do with persistent connections (as controlled by the Keep-Alive header). The HTTP client is sending a Keep-Alive header, and creating a persistent connection in a pool. Then at the end of the test, the mock server is shut down and the persistent connection is no longer valid.

The next test will try reuse the previous connection, which is stale as a new mock server has now started on that port, and results in a org.apache.http.NoHttpResponseException.

By putting the timeout in the startup, when the second test runs, the HTTP client will not reuse the old connection because it has probably exceeded the max stale check time in the pool, and will get a new one.

[uglyog]

This fixes your test:

  @After
  void teardown() {
    Executor.closeIdleConnections()
  }

[andrewspinks]

Wow, awesome Ron, thanks! Is there some troubleshooting doco that could be updated? Otherwise I'll close this issue.

[uglyog]

We could add a note to the two project readmes.

[ricwal-richa]

I am using au.com.dius:pact-jvm-provider-gradle_2.11:3.2.4 and getting 'Connection refused' for every single test (attaching stacktrace and debug mode files). Also is there a way to increase timeout or put a delay directly through a property defined in gradle build file ?
I am getting same error even for single individual test when using provider-gradle library for localhost.
Multitest_failure_debug.txt
Multitest_failure_stacktrace.txt
I am trying to run the sample test given under Defect342MultiTest.groovy

[uglyog]

@ricwal-richa are you verifying an actual provider? It looks like there is nothing running on port 8080, hence the connection refused errors.

[peternelissen]

Adding the exception for SEO reasons. Searched a long time for this bug, maybe this helps people find it:

For me it only happened for two PUT requests after each other.

Caused by: java.net.SocketException: Software caused connection abort: recv failed
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)

[asocaciu]

I want to point out another workaround for connections remaining open, if using the standard JDK http library
@BeforeAll public static void setup() { System.setProperty("http.keepAlive", "false"); }
Reference: https://docs.oracle.com/javase/6/docs/technotes/guides/net/http-keepalive.html

[uglyog]

Added a note to readme about persistent HTTP/1.1 connections: https://github.com/pact-foundation/pact-jvm/tree/master/consumer#dealing-with-persistent-http11-connections-keep-alive