Project contributes conflicting versions for kafka-clients to micronaut-platform

[mpkorstanje]

Expected Behavior

No response

Actual Behaviour

The micronaut-platform-4.6.1.pom as published to Maven Central contains a duplicate dependency for kafka-clients.

...
    <kafka.compat.version>3.8.0</kafka.compat.version>
    <kafka.version>3.7.0</kafka.version>
...
      <dependency>
        <groupId>org.apache.kafka</groupId>
        <artifactId>kafka-clients</artifactId>
        <version>${kafka.compat.version}</version>
      </dependency>
      <dependency>
        <groupId>org.apache.kafka</groupId>
        <artifactId>kafka-clients</artifactId>
        <version>${kafka.version}</version>
      </dependency>

As a result the 3.8.0 version is used. However when using io.micronaut.kafka:micronaut-kafka, the 3.7.0 version is expected. When using the 3.8.0 version this exception will be thrown:

  stack_trace: org.apache.kafka.common.KafkaException: Failed to create new NetworkClient
   at io.micronaut.configuration.kafka.health.DefaultNetworkClientCreator.create(DefaultNetworkClientCreator.java:99)
   at io.micronaut.configuration.kafka.health.KafkaHealthIndicator.lambda$new$1(KafkaHealthIndicator.java:94)
   at io.micronaut.core.util.SupplierUtil$1.get(SupplierUtil.java:47)
   at io.micronaut.configuration.kafka.health.KafkaHealthIndicator.getResult(KafkaHealthIndicator.java:148)
   at io.micronaut.configuration.kafka.health.KafkaHealthIndicator.getResult(KafkaHealthIndicator.java:59)
   at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
   at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:1024)
   at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
   at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
   at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
   at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
   at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
   at io.micronaut.management.health.aggregator.DefaultHealthAggregator.aggregateResults(DefaultHealthAggregator.java:111)
   at io.micronaut.management.health.aggregator.DefaultHealthAggregator.aggregate(DefaultHealthAggregator.java:72)
   at io.micronaut.management.endpoint.health.HealthEndpoint.getHealth(HealthEndpoint.java:131)
   at io.micronaut.management.endpoint.health.$HealthEndpoint$Definition$Exec.dispatch(Unknown Source)
   at io.micronaut.context.AbstractExecutableMethodsDefinition$DispatchedExecutableMethod.invoke(AbstractExecutableMethodsDefinition.java:456)
   at io.micronaut.context.DefaultBeanContext$BeanExecutionHandle.invoke(DefaultBeanContext.java:3940)
   at io.micronaut.web.router.AbstractRouteMatch.execute(AbstractRouteMatch.java:274)
   at io.micronaut.web.router.DefaultUriRouteMatch.execute(DefaultUriRouteMatch.java:38)
   at io.micronaut.http.server.RouteExecutor.executeRouteAndConvertBody(RouteExecutor.java:498)
   at io.micronaut.http.server.RouteExecutor.callRoute(RouteExecutor.java:486)
   at io.micronaut.http.server.RequestLifecycle.callRoute(RequestLifecycle.java:182)
   at io.micronaut.http.server.RequestLifecycle.executeRoute(RequestLifecycle.java:170)
   at io.micronaut.http.server.RequestLifecycle$2.provideResponse(RequestLifecycle.java:390)
   at io.micronaut.http.filter.FilterRunner.provideResponseAndHandleErrors(FilterRunner.java:372)
   at io.micronaut.http.filter.FilterRunner.filterRequest(FilterRunner.java:307)
   at io.micronaut.http.filter.FilterRunner.lambda$filterRequest$2(FilterRunner.java:280)
   at io.micronaut.http.filter.AroundLegacyFilter$FilterChainImpl.proceed(AroundLegacyFilter.java:122)
   at io.micronaut.configuration.metrics.binder.web.ServerRequestMeterRegistryFilter.doFilter(ServerRequestMeterRegistryFilter.java:78)
   at io.micronaut.http.filter.HttpServerFilter.doFilter(HttpServerFilter.java:48)
   at io.micronaut.http.filter.AroundLegacyFilter.processRequestFilter(AroundLegacyFilter.java:75)
   at io.micronaut.http.filter.FilterRunner.filterRequest(FilterRunner.java:276)
   at io.micronaut.http.filter.FilterRunner.run(FilterRunner.java:247)
   at io.micronaut.http.server.RequestLifecycle.runServerFilters(RequestLifecycle.java:422)
   at io.micronaut.http.server.RequestLifecycle.normalFlow(RequestLifecycle.java:160)
   at io.micronaut.http.server.netty.NettyRequestLifecycle.handleNormal(NettyRequestLifecycle.java:89)
   at io.micronaut.http.server.netty.RoutingInBoundHandler.accept(RoutingInBoundHandler.java:235)
   at io.micronaut.http.server.netty.websocket.NettyServerWebSocketUpgradeHandler.accept(NettyServerWebSocketUpgradeHandler.java:156)
   at io.micronaut.http.server.netty.handler.PipeliningServerHandler$MessageInboundHandler.read(PipeliningServerHandler.java:415)
   at io.micronaut.http.server.netty.handler.PipeliningServerHandler.channelRead(PipeliningServerHandler.java:221)
   at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
   at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
   at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
   at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
   at io.netty.handler.codec.http.websocketx.extensions.WebSocketServerExtensionHandler.onHttpRequestChannelRead(WebSocketServerExtensionHandler.java:158)
   at io.netty.handler.codec.http.websocketx.extensions.WebSocketServerExtensionHandler.channelRead(WebSocketServerExtensionHandler.java:82)
   at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
   at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
   at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
   at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
   at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
   at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
   at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
   at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
   at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
   at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
   at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:289)
   at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
   at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
   at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
   at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1407)
   at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
   at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
   at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:918)
   at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
   at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
   at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
   at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
   at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
   at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:994)
   at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
   at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
   at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.lang.NoSuchMethodError: 'void org.apache.kafka.clients.NetworkClient.<init>(org.apache.kafka.common.network.Selectable, org.apache.kafka.clients.Metadata, java.lang.String, int, long, long, int, int, int, long, long, org.apache.kafka.common.utils.Time, boolean, org.apache.kafka.clients.ApiVersions, org.apache.kafka.common.utils.LogContext)'
   at io.micronaut.configuration.kafka.health.DefaultNetworkClientCreator.create(DefaultNetworkClientCreator.java:93)
   ... 73 common frames omitted

It looks like this duplicate is caused by

micronaut-kafka/gradle/libs.versions.toml

Lines 7 to 8 in 1bad4bb

	# Required to keep catalog compatibility with 3.4.x. Can be removed for 4.0.0
	managed-kafka-compat = "3.8.0"

.

I reckon a quick fix would be to keep these two versions in sync. But the comment also says that the work around can be removed for v4.

Steps To Reproduce

No response

Environment Information

No response

Example Application

No response

Version

4.6.1

[mpkorstanje]

As a work around it is possible to use:

    <properties>
        <kafka.compat.version>${kafka.version}</kafka.compat.version>
    </properties>

[graemerocher]

Fixed in #1093