Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: Running multiple instances of static Kraft container will fail on listeners config #8619

Closed
loicgreffier opened this issue May 13, 2024 · 1 comment · Fixed by #8850
Closed

[Bug]: Running multiple instances of static Kraft container will fail on listeners config #8619

loicgreffier opened this issue May 13, 2024 · 1 comment · Fixed by #8850
Labels
modules/kafka type/bug

Comments

@loicgreffier
Copy link

loicgreffier commented May 13, 2024 •
edited
Loading

Module

Kafka

Testcontainers version

1.19.8

Using the latest Testcontainers version?

Yes

Host OS

Windows (WSL2)

Host Arch

x64

Docker version

Client: Docker Engine - Community
 Version:           24.0.6
 API version:       1.43
 Go version:        go1.20.7
 Git commit:        ed223bc
 Built:             Mon Sep  4 12:31:44 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          24.0.6
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.7
  Git commit:       1a79695
  Built:            Mon Sep  4 12:31:44 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.24
  GitCommit:        61f9fd88f79f081d64d6fa3bb1a0dc71ec870523
 runc:
  Version:          1.1.9
  GitCommit:        v1.1.9-0-gccaecfc
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

What happened?

Declare a Kraft Kafka container in an abstract class

abstract class KraftIntegrationTest {
    @Container
    static KafkaContainer broker = new KafkaContainer(DockerImageName
            .parse("confluentinc/cp-kafka:7.6.1"))
            .withKraft();
}

Declare at least two classes implementing KraftIntegrationTest so the KafkaContainer runs at least twice

public class FirstKraftChildIntegrationTest extends KraftIntegrationTest { 
    @Test
    void test() {
        // Do something
    }
}

public class SecondKraftChildIntegrationTest extends KraftIntegrationTest { 
    @Test
    void test() {
        // Do something
    }
}

The second container will not start and fail on KAFKA_LISTENERS containing multiple listeners with the same name.

Relevant log output

8:31:53.624 [main] INFO org.testcontainers.images.PullPolicy -- Image pull policy will be performed by: DefaultPullPolicy()
18:31:53.627 [main] INFO org.testcontainers.utility.ImageNameSubstitutor -- Image name substitution will be performed by: DefaultImageNameSubstitutor (composite of 'ConfigurationFileImageNameSubstitutor' and 'PrefixingImageNameSubstitutor')
18:31:54.098 [main] INFO org.testcontainers.dockerclient.DockerClientProviderStrategy -- Found Docker environment with local Unix socket (unix:///var/run/docker.sock)
18:31:54.107 [main] INFO org.testcontainers.DockerClientFactory -- Docker host IP address is localhost
18:31:54.124 [main] INFO org.testcontainers.DockerClientFactory -- Connected to docker: 
  Server Version: 24.0.9
  API Version: 1.43
  Operating System: Ubuntu 22.04.4 LTS
  Total Memory: 15981 MB
18:31:54.155 [main] INFO tc.testcontainers/ryuk:0.7.0 -- Pulling docker image: testcontainers/ryuk:0.7.0. Please be patient; this may take some time but only needs to be done once.
18:31:54.433 [docker-java-stream-548174466] INFO tc.testcontainers/ryuk:0.7.0 -- Starting to pull image
18:31:54.447 [docker-java-stream-548174466] INFO tc.testcontainers/ryuk:0.7.0 -- Pulling image layers:  0 pending,  0 downloaded,  0 extracted, (0 bytes/0 bytes)
18:31:54.604 [docker-java-stream-548174466] INFO tc.testcontainers/ryuk:0.7.0 -- Pulling image layers:  2 pending,  1 downloaded,  0 extracted, (33 KB/? MB)
18:31:54.628 [docker-java-stream-548174466] INFO tc.testcontainers/ryuk:0.7.0 -- Pulling image layers:  1 pending,  2 downloaded,  0 extracted, (62 KB/? MB)
18:31:54.732 [docker-java-stream-548174466] INFO tc.testcontainers/ryuk:0.7.0 -- Pulling image layers:  1 pending,  2 downloaded,  1 extracted, (308 KB/? MB)
18:31:54.808 [docker-java-stream-548174466] INFO tc.testcontainers/ryuk:0.7.0 -- Pulling image layers:  1 pending,  2 downloaded,  2 extracted, (3 MB/? MB)
18:31:54.815 [main] INFO tc.testcontainers/ryuk:0.7.0 -- Image testcontainers/ryuk:0.7.0 pull took PT0.66058858S
18:31:54.834 [main] INFO tc.testcontainers/ryuk:0.7.0 -- Creating container for image: testcontainers/ryuk:0.7.0
18:31:54.894 [main] INFO tc.testcontainers/ryuk:0.7.0 -- Container testcontainers/ryuk:0.7.0 is starting: 4592c932e7d73620684c82237568494a2cb991b64da2d955a6c8f1194378ab0f
18:31:55.210 [main] INFO tc.testcontainers/ryuk:0.7.0 -- Container testcontainers/ryuk:0.7.0 started in PT0.376065228S
18:31:55.214 [main] INFO org.testcontainers.utility.RyukResourceReaper -- Ryuk started - will monitor and terminate Testcontainers containers on JVM exit
18:31:55.214 [main] INFO org.testcontainers.DockerClientFactory -- Checking the system...
18:31:55.215 [main] INFO org.testcontainers.DockerClientFactory -- ✔︎ Docker server version should be at least 1.6.0
18:31:55.217 [main] INFO tc.confluentinc/cp-kafka:7.6.1 -- Pulling docker image: confluentinc/cp-kafka:7.6.1. Please be patient; this may take some time but only needs to be done once.
18:31:55.414 [docker-java-stream-1940910550] INFO tc.confluentinc/cp-kafka:7.6.1 -- Starting to pull image
18:31:55.415 [docker-java-stream-1940910550] INFO tc.confluentinc/cp-kafka:7.6.1 -- Pulling image layers:  0 pending,  0 downloaded,  0 extracted, (0 bytes/0 bytes)
18:31:55.617 [docker-java-stream-1940910550] INFO tc.confluentinc/cp-kafka:7.6.1 -- Pulling image layers: 10 pending,  1 downloaded,  0 extracted, (961 KB/? MB)
18:31:55.705 [docker-java-stream-1940910550] INFO tc.confluentinc/cp-kafka:7.6.1 -- Pulling image layers:  9 pending,  2 downloaded,  0 extracted, (15 MB/? MB)
18:31:56.044 [docker-java-stream-1940910550] INFO tc.confluentinc/cp-kafka:7.6.1 -- Pulling image layers:  8 pending,  3 downloaded,  0 extracted, (53 MB/? MB)
18:31:56.141 [docker-java-stream-1940910550] INFO tc.confluentinc/cp-kafka:7.6.1 -- Pulling image layers:  7 pending,  4 downloaded,  0 extracted, (61 MB/? MB)
18:31:56.229 [docker-java-stream-1940910550] INFO tc.confluentinc/cp-kafka:7.6.1 -- Pulling image layers:  6 pending,  5 downloaded,  0 extracted, (67 MB/? MB)
18:31:56.338 [docker-java-stream-1940910550] INFO tc.confluentinc/cp-kafka:7.6.1 -- Pulling image layers:  5 pending,  6 downloaded,  0 extracted, (79 MB/? MB)
18:31:56.470 [docker-java-stream-1940910550] INFO tc.confluentinc/cp-kafka:7.6.1 -- Pulling image layers:  4 pending,  7 downloaded,  0 extracted, (91 MB/? MB)
18:31:56.644 [docker-java-stream-1940910550] INFO tc.confluentinc/cp-kafka:7.6.1 -- Pulling image layers:  3 pending,  8 downloaded,  0 extracted, (116 MB/? MB)
18:31:56.739 [docker-java-stream-1940910550] INFO tc.confluentinc/cp-kafka:7.6.1 -- Pulling image layers:  2 pending,  9 downloaded,  0 extracted, (126 MB/? MB)
18:31:57.089 [docker-java-stream-1940910550] INFO tc.confluentinc/cp-kafka:7.6.1 -- Pulling image layers:  2 pending,  9 downloaded,  1 extracted, (175 MB/? MB)
18:31:59.314 [docker-java-stream-1940910550] INFO tc.confluentinc/cp-kafka:7.6.1 -- Pulling image layers:  1 pending, 10 downloaded,  1 extracted, (376 MB/? MB)
18:32:00.046 [docker-java-stream-1940910550] INFO tc.confluentinc/cp-kafka:7.6.1 -- Pulling image layers:  0 pending, 11 downloaded,  1 extracted, (413 MB/431 MB)
18:32:08.027 [docker-java-stream-1940910550] INFO tc.confluentinc/cp-kafka:7.6.1 -- Pulling image layers:  0 pending, 11 downloaded,  2 extracted, (418 MB/431 MB)
18:32:08.121 [docker-java-stream-1940910550] INFO tc.confluentinc/cp-kafka:7.6.1 -- Pulling image layers:  0 pending, 11 downloaded,  3 extracted, (425 MB/431 MB)
18:32:08.149 [docker-java-stream-1940910550] INFO tc.confluentinc/cp-kafka:7.6.1 -- Pulling image layers:  0 pending, 11 downloaded,  4 extracted, (425 MB/431 MB)
18:32:08.457 [docker-java-stream-1940910550] INFO tc.confluentinc/cp-kafka:7.6.1 -- Pulling image layers:  0 pending, 11 downloaded,  5 extracted, (430 MB/431 MB)
18:32:08.469 [docker-java-stream-1940910550] INFO tc.confluentinc/cp-kafka:7.6.1 -- Pulling image layers:  0 pending, 11 downloaded,  6 extracted, (430 MB/431 MB)
18:32:08.484 [docker-java-stream-1940910550] INFO tc.confluentinc/cp-kafka:7.6.1 -- Pulling image layers:  0 pending, 11 downloaded,  7 extracted, (430 MB/431 MB)
18:32:08.494 [docker-java-stream-1940910550] INFO tc.confluentinc/cp-kafka:7.6.1 -- Pulling image layers:  0 pending, 11 downloaded,  8 extracted, (430 MB/431 MB)
18:32:08.506 [docker-java-stream-1940910550] INFO tc.confluentinc/cp-kafka:7.6.1 -- Pulling image layers:  0 pending, 11 downloaded,  9 extracted, (430 MB/431 MB)
18:32:09.452 [docker-java-stream-1940910550] INFO tc.confluentinc/cp-kafka:7.6.1 -- Pulling image layers:  0 pending, 11 downloaded, 10 extracted, (431 MB/431 MB)
18:32:09.465 [docker-java-stream-1940910550] INFO tc.confluentinc/cp-kafka:7.6.1 -- Pulling image layers:  0 pending, 11 downloaded, 11 extracted, (431 MB/431 MB)
18:32:09.473 [docker-java-stream-1940910550] INFO tc.confluentinc/cp-kafka:7.6.1 -- Pull complete. 11 layers, pulled in 14s (downloaded 431 MB at 30 MB/s)
18:32:09.473 [main] INFO tc.confluentinc/cp-kafka:7.6.1 -- Image confluentinc/cp-kafka:7.6.1 pull took PT14.255653839S
18:32:09.479 [main] INFO tc.confluentinc/cp-kafka:7.6.1 -- Creating container for image: confluentinc/cp-kafka:7.6.1
18:32:09.497 [main] INFO tc.confluentinc/cp-kafka:7.6.1 -- Container confluentinc/cp-kafka:7.6.1 is starting: 8fdc1b62f7bd3d8d10a0c8f2d7fcf73cf14486802f6767761b9ced3ed17c64e9
18:32:16.961 [main] INFO tc.confluentinc/cp-kafka:7.6.1 -- Container confluentinc/cp-kafka:7.6.1 started in PT7.482221571S
18:32:17.021 [main] INFO org.springframework.test.context.support.AnnotationConfigContextLoaderUtils -- Could not detect default configuration classes for test class [io.loicgreffier.testcontainers.kraft.SecondKraftChildIntegrationTest]: SecondKraftChildIntegrationTest does not declare any static, non-private, non-final, nested classes annotated with @Configuration.
18:32:17.106 [main] INFO org.springframework.boot.test.context.SpringBootTestContextBootstrapper -- Found @SpringBootConfiguration io.loicgreffier.testcontainers.TestcontainersApplication for test class io.loicgreffier.testcontainers.kraft.SecondKraftChildIntegrationTest

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::                (v3.2.5)

2024-05-13T18:32:17.393Z  INFO 1830 --- [testcontainers] [           main] i.l.t.k.SecondKraftChildIntegrationTest  : Starting SecondKraftChildIntegrationTest using Java 21.0.2 with PID 1830 (started by runner in /home/runner/work/testcontainers-kraft-bug/testcontainers-kraft-bug)
2024-05-13T18:32:17.394Z  INFO 1830 --- [testcontainers] [           main] i.l.t.k.SecondKraftChildIntegrationTest  : No active profile set, falling back to 1 default profile: "default"
2024-05-13T18:32:18.283Z  INFO 1830 --- [testcontainers] [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port 8080 (http)
2024-05-13T18:32:18.294Z  INFO 1830 --- [testcontainers] [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2024-05-13T18:32:18.294Z  INFO 1830 --- [testcontainers] [           main] o.apache.catalina.core.StandardEngine    : Starting Servlet engine: [Apache Tomcat/10.1.20]
2024-05-13T18:32:18.331Z  INFO 1830 --- [testcontainers] [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2024-05-13T18:32:18.332Z  INFO 1830 --- [testcontainers] [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 924 ms
2024-05-13T18:32:18.647Z  INFO 1830 --- [testcontainers] [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port 8080 (http) with context path ''
2024-05-13T18:32:18.656Z  INFO 1830 --- [testcontainers] [           main] i.l.t.k.SecondKraftChildIntegrationTest  : Started SecondKraftChildIntegrationTest in 1.457 seconds (process running for 25.577)
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
WARNING: A Java agent has been loaded dynamically (/home/runner/.m2/repository/net/bytebuddy/byte-buddy-agent/1.14.13/byte-buddy-agent-1.14.13.jar)
WARNING: If a serviceability tool is in use, please run with -XX:+EnableDynamicAgentLoading to hide this warning
WARNING: If a serviceability tool is not in use, please run with -Djdk.instrument.traceUsage for more information
WARNING: Dynamic loading of agents will be disallowed by default in a future release
2024-05-13T18:32:19.333Z  INFO 1830 --- [testcontainers] [           main] i.l.t.k.SecondKraftChildIntegrationTest  : testB
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 26.07 s -- in io.loicgreffier.testcontainers.kraft.SecondKraftChildIntegrationTest
[INFO] Running io.loicgreffier.testcontainers.kraft.FirstKraftChildIntegrationTest
2024-05-13T18:32:19.590Z  INFO 1830 --- [testcontainers] [           main] tc.confluentinc/cp-kafka:7.6.1           : Creating container for image: confluentinc/cp-kafka:7.6.1
2024-05-13T18:32:19.627Z  INFO 1830 --- [testcontainers] [           main] tc.confluentinc/cp-kafka:7.6.1           : Container confluentinc/cp-kafka:7.6.1 is starting: 4b3a0ba2ba7d8cd4aa2252f6ff54eca47c3d109f6171b52578cf172ec0eb208d
2024-05-13T18:33:19.934Z ERROR 1830 --- [testcontainers] [           main] tc.confluentinc/cp-kafka:7.6.1           : Could not start container

java.lang.IllegalStateException: Wait strategy failed. Container exited with code 1
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:533) ~[testcontainers-1.19.8.jar:na]
	at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:354) ~[testcontainers-1.19.8.jar:na]
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81) ~[duct-tape-1.0.8.jar:na]
	at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:344) ~[testcontainers-1.19.8.jar:na]
	at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:330) ~[testcontainers-1.19.8.jar:na]
	at org.testcontainers.junit.jupiter.TestcontainersExtension$StoreAdapter.start(TestcontainersExtension.java:280) ~[junit-jupiter-1.19.8.jar:na]
	at org.testcontainers.junit.jupiter.TestcontainersExtension$StoreAdapter.access$200(TestcontainersExtension.java:267) ~[junit-jupiter-1.19.8.jar:na]
	at org.testcontainers.junit.jupiter.TestcontainersExtension.lambda$null$4(TestcontainersExtension.java:82) ~[junit-jupiter-1.19.8.jar:na]
	at org.junit.platform.engine.support.store.NamespacedHierarchicalStore.lambda$getOrComputeIfAbsent$5(NamespacedHierarchicalStore.java:147) ~[junit-platform-engine-1.10.2.jar:1.10.2]
	at org.junit.platform.engine.support.store.NamespacedHierarchicalStore$MemoizingSupplier.computeValue(NamespacedHierarchicalStore.java:372) ~[junit-platform-engine-1.10.2.jar:1.10.2]
	at org.junit.platform.engine.support.store.NamespacedHierarchicalStore$MemoizingSupplier.get(NamespacedHierarchicalStore.java:361) ~[junit-platform-engine-1.10.2.jar:1.10.2]
	at org.junit.platform.engine.support.store.NamespacedHierarchicalStore$StoredValue.evaluate(NamespacedHierarchicalStore.java:308) ~[junit-platform-engine-1.10.2.jar:1.10.2]
	at org.junit.platform.engine.support.store.NamespacedHierarchicalStore$StoredValue.access$200(NamespacedHierarchicalStore.java:287) ~[junit-platform-engine-1.10.2.jar:1.10.2]
	at org.junit.platform.engine.support.store.NamespacedHierarchicalStore.getOrComputeIfAbsent(NamespacedHierarchicalStore.java:149) ~[junit-platform-engine-1.10.2.jar:1.10.2]
	at org.junit.jupiter.engine.execution.NamespaceAwareStore.lambda$getOrComputeIfAbsent$2(NamespaceAwareStore.java:57) ~[junit-jupiter-engine-5.10.2.jar:5.10.2]
	at org.junit.jupiter.engine.execution.NamespaceAwareStore.accessStore(NamespaceAwareStore.java:90) ~[junit-jupiter-engine-5.10.2.jar:5.10.2]
	at org.junit.jupiter.engine.execution.NamespaceAwareStore.getOrComputeIfAbsent(NamespaceAwareStore.java:57) ~[junit-jupiter-engine-5.10.2.jar:5.10.2]
	at org.testcontainers.junit.jupiter.TestcontainersExtension.lambda$startContainers$5(TestcontainersExtension.java:82) ~[junit-jupiter-1.19.8.jar:na]
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596) ~[na:na]
	at org.testcontainers.junit.jupiter.TestcontainersExtension.startContainers(TestcontainersExtension.java:82) ~[junit-jupiter-1.19.8.jar:na]
	at org.testcontainers.junit.jupiter.TestcontainersExtension.beforeAll(TestcontainersExtension.java:56) ~[junit-jupiter-1.19.8.jar:na]
	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$invokeBeforeAllCallbacks$12(ClassBasedTestDescriptor.java:396) ~[junit-jupiter-engine-5.10.2.jar:5.10.2]
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.10.2.jar:1.10.2]
	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.invokeBeforeAllCallbacks(ClassBasedTestDescriptor.java:396) ~[junit-jupiter-engine-5.10.2.jar:5.10.2]
	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.before(ClassBasedTestDescriptor.java:212) ~[junit-jupiter-engine-5.10.2.jar:5.10.2]
	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.before(ClassBasedTestDescriptor.java:85) ~[junit-jupiter-engine-5.10.2.jar:5.10.2]
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:148) ~[junit-platform-engine-1.10.2.jar:1.10.2]
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.10.2.jar:1.10.2]
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) ~[junit-platform-engine-1.10.2.jar:1.10.2]
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) ~[junit-platform-engine-1.10.2.jar:1.10.2]
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) ~[junit-platform-engine-1.10.2.jar:1.10.2]
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.10.2.jar:1.10.2]
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) ~[junit-platform-engine-1.10.2.jar:1.10.2]
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) ~[junit-platform-engine-1.10.2.jar:1.10.2]
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596) ~[na:na]
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41) ~[junit-platform-engine-1.10.2.jar:1.10.2]
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155) ~[junit-platform-engine-1.10.2.jar:1.10.2]
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.10.2.jar:1.10.2]
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) ~[junit-platform-engine-1.10.2.jar:1.10.2]
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) ~[junit-platform-engine-1.10.2.jar:1.10.2]
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) ~[junit-platform-engine-1.10.2.jar:1.10.2]
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.10.2.jar:1.10.2]
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) ~[junit-platform-engine-1.10.2.jar:1.10.2]
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) ~[junit-platform-engine-1.10.2.jar:1.10.2]
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35) ~[junit-platform-engine-1.10.2.jar:1.10.2]
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57) ~[junit-platform-engine-1.10.2.jar:1.10.2]
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54) ~[junit-platform-engine-1.10.2.jar:1.10.2]
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) ~[junit-platform-launcher-1.10.2.jar:1.10.2]
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) ~[junit-platform-launcher-1.10.2.jar:1.10.2]
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) ~[junit-platform-launcher-1.10.2.jar:1.10.2]
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) ~[junit-platform-launcher-1.10.2.jar:1.10.2]
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) ~[junit-platform-launcher-1.10.2.jar:1.10.2]
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) ~[junit-platform-launcher-1.10.2.jar:1.10.2]
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) ~[junit-platform-launcher-1.10.2.jar:1.10.2]
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) ~[junit-platform-launcher-1.10.2.jar:1.10.2]
	at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) ~[junit-platform-launcher-1.10.2.jar:1.10.2]
	at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) ~[surefire-junit-platform-3.1.2.jar:3.1.2]
	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) ~[surefire-junit-platform-3.1.2.jar:3.1.2]
	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) ~[surefire-junit-platform-3.1.2.jar:3.1.2]
	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) ~[surefire-junit-platform-3.1.2.jar:3.1.2]
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) ~[surefire-booter-3.1.2.jar:3.1.2]
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) ~[surefire-booter-3.1.2.jar:3.1.2]
	at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) ~[surefire-booter-3.1.2.jar:3.1.2]
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) ~[surefire-booter-3.1.2.jar:3.1.2]
Caused by: org.testcontainers.containers.ContainerLaunchException: Timed out waiting for log output matching '.*Transitioning from RECOVERY to RUNNING.*'
	at org.testcontainers.containers.wait.strategy.LogMessageWaitStrategy.waitUntilReady(LogMessageWaitStrategy.java:47) ~[testcontainers-1.19.8.jar:1.19.8]
	at org.testcontainers.containers.wait.strategy.AbstractWaitStrategy.waitUntilReady(AbstractWaitStrategy.java:52) ~[testcontainers-1.19.8.jar:1.19.8]
	at org.testcontainers.containers.GenericContainer.waitUntilContainerStarted(GenericContainer.java:909) ~[testcontainers-1.19.8.jar:na]
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:500) ~[testcontainers-1.19.8.jar:na]
	... 63 common frames omitted

2024-05-13T18:33:19.962Z ERROR 1830 --- [testcontainers] [           main] tc.confluentinc/cp-kafka:7.6.1           : Log output from the failed container:
===> User
uid=1000(appuser) gid=1000(appuser) groups=1000(appuser)
===> Configuring ...
Running in KRaft mode...
Error: Exception in thread "main" java.lang.IllegalArgumentException: requirement failed: Each listener must have a different name, listeners: CONTROLLER://0.0.0.0:9094,PLAINTEXT://0.0.0.0:9093,BROKER://0.0.0.0:9092,CONTROLLER://0.0.0.0:9094
	at scala.Predef$.require(Predef.scala:337)
	at kafka.utils.CoreUtils$.validate$1(CoreUtils.scala:214)
	at kafka.utils.CoreUtils$.listenerListToEndPoints(CoreUtils.scala:268)
	at kafka.server.KafkaConfig.listeners(KafkaConfig.scala:2125)
	at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1812)
	at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1607)
	at kafka.tools.StorageTool$.$anonfun$main$1(StorageTool.scala:50)
	at scala.Option.flatMap(Option.scala:283)
	at kafka.tools.StorageTool$.main(StorageTool.scala:50)
	at kafka.tools.StorageTool.main(StorageTool.scala)

2024-05-13T18:33:19.965Z  INFO 1830 --- [testcontainers] [           main] t.c.s.AnnotationConfigContextLoaderUtils : Could not detect default configuration classes for test class [io.loicgreffier.testcontainers.kraft.FirstKraftChildIntegrationTest]: FirstKraftChildIntegrationTest does not declare any static, non-private, non-final, nested classes annotated with @Configuration.
2024-05-13T18:33:19.967Z  INFO 1830 --- [testcontainers] [           main] .b.t.c.SpringBootTestContextBootstrapper : Found @SpringBootConfiguration io.loicgreffier.testcontainers.TestcontainersApplication for test class io.loicgreffier.testcontainers.kraft.FirstKraftChildIntegrationTest
Error:  Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 60.38 s <<< FAILURE! -- in io.loicgreffier.testcontainers.kraft.FirstKraftChildIntegrationTest
Error:  io.loicgreffier.testcontainers.kraft.FirstKraftChildIntegrationTest -- Time elapsed: 60.38 s <<< ERROR!
org.testcontainers.containers.ContainerLaunchException: Container startup failed for image confluentinc/cp-kafka:7.6.1
	at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:359)
	at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:330)
	at org.testcontainers.junit.jupiter.TestcontainersExtension$StoreAdapter.start(TestcontainersExtension.java:280)
	at org.testcontainers.junit.jupiter.TestcontainersExtension$StoreAdapter.access$200(TestcontainersExtension.java:267)
	at org.testcontainers.junit.jupiter.TestcontainersExtension.lambda$null$4(TestcontainersExtension.java:82)
	at org.testcontainers.junit.jupiter.TestcontainersExtension.lambda$startContainers$5(TestcontainersExtension.java:82)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
	at org.testcontainers.junit.jupiter.TestcontainersExtension.startContainers(TestcontainersExtension.java:82)
	at org.testcontainers.junit.jupiter.TestcontainersExtension.beforeAll(TestcontainersExtension.java:56)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
Caused by: org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:88)
	at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:344)
	... 9 more
Caused by: org.testcontainers.containers.ContainerLaunchException: Could not create/start container
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:563)
	at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:354)
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81)
	... 10 more
Caused by: java.lang.IllegalStateException: Wait strategy failed. Container exited with code 1
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:533)
	... 12 more
Caused by: org.testcontainers.containers.ContainerLaunchException: Timed out waiting for log output matching '.*Transitioning from RECOVERY to RUNNING.*'
	at org.testcontainers.containers.wait.strategy.LogMessageWaitStrategy.waitUntilReady(LogMessageWaitStrategy.java:47)
	at org.testcontainers.containers.wait.strategy.AbstractWaitStrategy.waitUntilReady(AbstractWaitStrategy.java:52)
	at org.testcontainers.containers.GenericContainer.waitUntilContainerStarted(GenericContainer.java:909)
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:500)
	... 12 more

Additional Information

The second container fails to start because:

java.lang.IllegalArgumentException: requirement failed: Each listener must have a different name, listeners: CONTROLLER://0.0.0.0:9094,PLAINTEXT://0.0.0.0:9093,BROKER://0.0.0.0:9092,CONTROLLER://0.0.0.0:9094

Not sure this can be described as an actual bug as it happens when the container is explicitly declared static in a parent class. It does not happen when the container is not static, or static but not running in Kraft mode.

However, this is disrupting when the container has to be static, for example, when using it with Spring Boot Dynamic Property Sources which is a common use-case.

I think the issue is coming from there:

testcontainers-java/modules/kafka/src/main/java/org/testcontainers/containers/KafkaContainer.java

Line 332 in 1f291eb

String.format("%s,CONTROLLER://0.0.0.0:9094", getEnvVars().get("KAFKA_LISTENERS"))
where current env vars are concatenated with CONTROLLER://0.0.0.0:9094 each time a container starts.

Maybe this could be improved by checking the content of the KAFKA_LISTENERS var env before concatenating.

A code sample is available at https://github.com/loicgreffier/testcontainers-kraft-bug.
@dogruis
Copy link

dogruis commented May 15, 2024 •
edited
Loading

I think an easier way to reproduce it is to simply do this in a few lines.

var container = new KafkaContainer(DockerImageName
            .parse("confluentinc/cp-kafka:7.6.1"))
            .withKraft();
container.start();
container.stop();
container.start();

This is enough to reproduce the issue. I fixed it temporarily like this by removing CONTROLLER://0.0.0.0:9094 before start. But that's flimsy

var container = new KafkaContainer(DockerImageName
            .parse("confluentinc/cp-kafka:7.6.1"))
            .withKraft();
container.start();
container.stop();
container.withEnv("KAFKA_LISTENERS", "PLAINTEXT://0.0.0.0:9093,BROKER://0.0.0.0:9092");
container.start();

eddumelendez added a commit that referenced this issue Jul 5, 2024
@eddumelendez
Avoid duplicate values for listeners and listener_security_protocol_m…
b8c6ed0 
…ap in KafkaContainer

Currently, when KafkaContainer is started more than one time then
`KAFKA_LISTENERS` and `KAFKA_LISTENER_SECURITY_PROTOCOL_MAP` registers
an additional entry, which is duplicated.

Fixes #8619
@eddumelendez eddumelendez mentioned this issue Jul 5, 2024
Merged
@loicgreffier loicgreffier mentioned this issue Aug 26, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
modules/kafka type/bug
Projects
None yet
Milestone
No milestone
3 participants
@eddumelendez @dogruis @loicgreffier