Tests run at incorrect phase in case of integration tests name not ending in "IT" #25070

peterhuba · 2022-04-21T14:03:21Z

Describe the bug

If an integration test class (@QuarkusIntegrationTest) name doesn't end in "IT", the test is run at a wrong phase, and the build fails.
Would expect to work correctly with any naming.

Expected behavior

Build should pass irrespective of the naming used for the integration test class. The annotation @QuarkusIntegrationTest could already indicate that it is an integration test.

Actual behavior

Complete output:

user@laptop:~/work/lambda-rest-example$ quarkus build --native
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------< com.example:lambda-rest-example >-------------------
[INFO] Building lambda-rest-example 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ lambda-rest-example ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ lambda-rest-example ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ lambda-rest-example ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/hpeter/work/lambda-rest-example/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ lambda-rest-example ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 2 source files to /home/hpeter/work/lambda-rest-example/target/test-classes
[INFO]
[INFO] --- maven-surefire-plugin:3.0.0-M5:test (default-test) @ lambda-rest-example ---
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.example.GreetingIntegrationTest
2022-04-21 16:43:43,851 INFO  [org.jbo.threads] (main) JBoss Threads version 3.4.2.Final
2022-04-21 16:43:44,339 INFO  [io.qua.ama.lam.run.MockEventServer] (build-1) Mock Lambda Event Server Started
======================================================================================
  native.image.path was not set, making a guess for the correct path of native image
  guessed path: /home/hpeter/work/lambda-rest-example/target/lambda-rest-example-1.0-SNAPSHOT-runner
======================================================================================
Executing "/home/hpeter/work/lambda-rest-example/target/lambda-rest-example-1.0-SNAPSHOT-runner -Dquarkus.http.port=8081 -Dquarkus.http.ssl-port=8444 -Dtest.url=http://localhost:8081 -Dquarkus.log.file.path=/home/hpeter/work/lambda-rest-example/target/quarkus.log -Dquarkus.log.file.enable=true -Dquarkus-internal.aws-lambda.test-api=localhost:8081/_lambda_"
__  ____  __  _____   ___  __ ____  ______
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/
2022-04-21 16:43:44,534 INFO  [io.quarkus] (main) lambda-rest-example 1.0-SNAPSHOT native (powered by Quarkus 2.8.1.Final) started in 0.070s.
2022-04-21 16:43:44,536 INFO  [io.quarkus] (main) Profile prod activated.
2022-04-21 16:43:44,536 INFO  [io.quarkus] (main) Installed features: [amazon-lambda, cdi, resteasy, security, smallrye-context-propagation, vertx]
2022-04-21 16:43:44,537 INFO  [io.qua.ama.lam.run.AbstractLambdaPollLoop] (Lambda Thread (NORMAL)) Listening on: http://localhost:8081/_lambda_/2018-06-01/runtime/invocation/next
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.62 s - in com.example.GreetingIntegrationTest
[INFO] Running com.example.GreetingTest
2022-04-21 16:43:46,900 INFO  [org.jbo.threads] (main) JBoss Threads version 3.4.2.Final
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.258 s <<< FAILURE! - in com.example.GreetingTest
[ERROR] com.example.GreetingTest.testJaxrs  Time elapsed: 0.001 s  <<< ERROR!
java.lang.RuntimeException:
java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
        [error]: Build step io.quarkus.amazon.lambda.deployment.DevServicesLambdaProcessor#startEventServer threw an exception: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.net.BindException: Address already in use
        at io.quarkus.amazon.lambda.runtime.MockEventServer.start(MockEventServer.java:82)
        at io.quarkus.amazon.lambda.deployment.DevServicesLambdaProcessor.startEventServer(DevServicesLambdaProcessor.java:81)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:925)
        at io.quarkus.builder.BuildContext.run(BuildContext.java:277)
        at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
        at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2449)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1478)
        at java.base/java.lang.Thread.run(Thread.java:829)
        at org.jboss.threads.JBossThread.run(JBossThread.java:501)
Caused by: java.util.concurrent.ExecutionException: java.net.BindException: Address already in use
        at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395)
        at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999)
        at io.quarkus.amazon.lambda.runtime.MockEventServer.start(MockEventServer.java:79)
        ... 12 more
Caused by: java.net.BindException: Address already in use
        at java.base/sun.nio.ch.Net.bind0(Native Method)
        at java.base/sun.nio.ch.Net.bind(Net.java:459)
        at java.base/sun.nio.ch.Net.bind(Net.java:448)
        at java.base/sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:227)
        at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:134)
        at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:562)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1334)
        at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:506)
        at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:491)
        at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:973)
        at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:260)
        at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:356)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasksFrom(SingleThreadEventExecutor.java:425)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:374)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:491)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:829)

Caused by: java.lang.RuntimeException:
io.quarkus.builder.BuildException: Build failure: Build failed due to errors
        [error]: Build step io.quarkus.amazon.lambda.deployment.DevServicesLambdaProcessor#startEventServer threw an exception: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.net.BindException: Address already in use
        at io.quarkus.amazon.lambda.runtime.MockEventServer.start(MockEventServer.java:82)
        at io.quarkus.amazon.lambda.deployment.DevServicesLambdaProcessor.startEventServer(DevServicesLambdaProcessor.java:81)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:925)
        at io.quarkus.builder.BuildContext.run(BuildContext.java:277)
        at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
        at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2449)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1478)
        at java.base/java.lang.Thread.run(Thread.java:829)
        at org.jboss.threads.JBossThread.run(JBossThread.java:501)
Caused by: java.util.concurrent.ExecutionException: java.net.BindException: Address already in use
        at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395)
        at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999)
        at io.quarkus.amazon.lambda.runtime.MockEventServer.start(MockEventServer.java:79)
        ... 12 more
Caused by: java.net.BindException: Address already in use
        at java.base/sun.nio.ch.Net.bind0(Native Method)
        at java.base/sun.nio.ch.Net.bind(Net.java:459)
        at java.base/sun.nio.ch.Net.bind(Net.java:448)
        at java.base/sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:227)
        at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:134)
        at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:562)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1334)
        at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:506)
        at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:491)
        at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:973)
        at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:260)
        at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:356)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasksFrom(SingleThreadEventExecutor.java:425)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:374)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:491)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:829)

Caused by: io.quarkus.builder.BuildException:
Build failure: Build failed due to errors
        [error]: Build step io.quarkus.amazon.lambda.deployment.DevServicesLambdaProcessor#startEventServer threw an exception: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.net.BindException: Address already in use
        at io.quarkus.amazon.lambda.runtime.MockEventServer.start(MockEventServer.java:82)
        at io.quarkus.amazon.lambda.deployment.DevServicesLambdaProcessor.startEventServer(DevServicesLambdaProcessor.java:81)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:925)
        at io.quarkus.builder.BuildContext.run(BuildContext.java:277)
        at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
        at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2449)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1478)
        at java.base/java.lang.Thread.run(Thread.java:829)
        at org.jboss.threads.JBossThread.run(JBossThread.java:501)
Caused by: java.util.concurrent.ExecutionException: java.net.BindException: Address already in use
        at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395)
        at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999)
        at io.quarkus.amazon.lambda.runtime.MockEventServer.start(MockEventServer.java:79)
        ... 12 more
Caused by: java.net.BindException: Address already in use
        at java.base/sun.nio.ch.Net.bind0(Native Method)
        at java.base/sun.nio.ch.Net.bind(Net.java:459)
        at java.base/sun.nio.ch.Net.bind(Net.java:448)
        at java.base/sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:227)
        at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:134)
        at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:562)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1334)
        at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:506)
        at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:491)
        at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:973)
        at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:260)
        at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:356)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasksFrom(SingleThreadEventExecutor.java:425)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:374)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:491)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:829)

Caused by: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.net.BindException: Address already in use
Caused by: java.util.concurrent.ExecutionException: java.net.BindException: Address already in use
Caused by: java.net.BindException: Address already in use

[INFO]
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR]   GreetingTest.testJaxrs » Runtime java.lang.RuntimeException: io.quarkus.builde...
[INFO]
[ERROR] Tests run: 2, Failures: 0, Errors: 1, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  9.538 s
[INFO] Finished at: 2022-04-21T16:43:47+03:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M5:test (default-test) on project lambda-rest-example: There are test failures.
[ERROR]
[ERROR] Please refer to /home/hpeter/work/lambda-rest-example/target/surefire-reports for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

How to Reproduce?

generate a project using mvn archetype:generate \ -DarchetypeGroupId=io.quarkus \ -DarchetypeArtifactId=quarkus-amazon-lambda-rest-archetype \ -DarchetypeVersion=2.8.1.Final
rename GreetingIT to GreetingIntegrationTest
run quarkus build --native, and for some reason the integration test gets run before the plain test, by the maven-surefire-plugin:3.0.0-M5:test at test phase, and not integration-test phase and fails

Output of `uname -a` or `ver`

Linux XS15-DQJ9ND3 5.10.16.3-microsoft-standard-WSL2 #1 SMP Fri Apr 2 22:23:49 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Output of `java -version`

openjdk 11.0.14 2022-01-18 OpenJDK Runtime Environment (build 11.0.14+9-Ubuntu-0ubuntu2.20.04) OpenJDK 64-Bit Server VM (build 11.0.14+9-Ubuntu-0ubuntu2.20.04, mixed mode, sharing)

GraalVM version (if different from Java)

(using docker to build native)

Quarkus version or git rev

2.8.1.Final

Build tool (ie. output of `mvnw --version` or `gradlew --version`)

Apache Maven 3.8.4 (9b656c72d54e5bacbed989b64718c159fe39b537) Maven home: /opt/maven Java version: 11.0.14, vendor: Ubuntu, runtime: /usr/lib/jvm/java-11-openjdk-amd64 Default locale: en, platform encoding: UTF-8 OS name: "linux", version: "5.10.16.3-microsoft-standard-wsl2", arch: "amd64", family: "unix"

Additional information

No response

The text was updated successfully, but these errors were encountered:

famod · 2022-04-21T21:12:20Z

In general, it's up to maven-surefire-pugin and maven-failsafe-plugin (not Quarkus!) to decide when/whether to run a specific test.
Includes for surefire: https://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#includes
Includes for failsafe: https://maven.apache.org/surefire/maven-failsafe-plugin/integration-test-mojo.html#includes

So you should probably tell surefire to exclude (or not include) *IntegrationTest and failsafe to do include *IntegrationTest.

Having said that, it might make sense to pre-configure generated Quarkus projects like that.

WDYT @geoand @aloubyansky?

aloubyansky · 2022-04-21T21:23:52Z

Generated projects are supposed to be templates/starting points that are meant to be further customized. IMO, they should not contain config that isn't common enough. Would most users find this addition useful or redundant?

geoand · 2022-04-22T11:39:15Z

Having said that, it might make sense to pre-configure generated Quarkus projects like that.

That is what @edeandrea's work proposes to do

geoand · 2022-04-22T11:40:06Z

I am going to close this, as our documentation clearly states that these tests can only be run by Maven Surefire (or the equivalent Gradle task)

edeandrea · 2022-04-22T14:12:23Z

Having said that, it might make sense to pre-configure generated Quarkus projects like that.

That is what @edeandrea's work proposes to do

@geoand are you talking about #21997?

peterhuba added the kind/bug Something isn't working label Apr 21, 2022

quarkus-bot bot added the triage/needs-triage label Apr 21, 2022

peterhuba mentioned this issue Apr 21, 2022

Tests run at incorrect phase in case of integration tests name not ending in "IT" #25069

Closed

geoand closed this as completed Apr 22, 2022

quarkus-bot bot removed the triage/needs-triage label Apr 22, 2022

geoand added triage/invalid This doesn't seem right area/testing labels Apr 22, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tests run at incorrect phase in case of integration tests name not ending in "IT" #25070

Tests run at incorrect phase in case of integration tests name not ending in "IT" #25070

peterhuba commented Apr 21, 2022

famod commented Apr 21, 2022 •

edited

Loading

aloubyansky commented Apr 21, 2022

geoand commented Apr 22, 2022

geoand commented Apr 22, 2022

edeandrea commented Apr 22, 2022

Tests run at incorrect phase in case of integration tests name not ending in "IT" #25070

Tests run at incorrect phase in case of integration tests name not ending in "IT" #25070

Comments

peterhuba commented Apr 21, 2022

Describe the bug

Expected behavior

Actual behavior

How to Reproduce?

Output of uname -a or ver

Output of java -version

GraalVM version (if different from Java)

Quarkus version or git rev

Build tool (ie. output of mvnw --version or gradlew --version)

Additional information

famod commented Apr 21, 2022 • edited Loading

aloubyansky commented Apr 21, 2022

geoand commented Apr 22, 2022

geoand commented Apr 22, 2022

edeandrea commented Apr 22, 2022

Output of `uname -a` or `ver`

Output of `java -version`

Build tool (ie. output of `mvnw --version` or `gradlew --version`)

famod commented Apr 21, 2022 •

edited

Loading