Disabling kafka streams health checks creates invalid topology with Kafka 2.5.0

[jgray1206]

The current property for disabling streams health checks is "kafka.streams.health.enabled". Being under the kafka.streams path, a ConfiguredStreamBuilder named "health" is being created, and a KafkaStreams instance is being built from it.

Kafka 2.5.0+ requires all topologies have either global tables or input streams. Since the "health" builder is not used to build a topology anywhere, it fits this category and triggers this error.

I believe the easiest solution would be changing the prefix for the streams HealthIndicator. That or filtering as to not create a "health" ConfiguredStreamBuilder.

[graemerocher]

Seems the solution is to upgrade to 2.5.1

[jgray1206]

Overwrote the Kafka version to a release candidate for 2.6.0. Looks like they made it more of an explicit error, actually. Instead of getting a “consumer is not subscribed to any topics or assigned any partitions” error when I disable the health check I get an “Invalid topology: Topology has no stream threads and no global threads, must subscribe to at least one source topic or global table” error.

[graemerocher]

Ok, would you like to submit a PR for your proposed fix?

[kascakm]

Similar error happens for me when I have only named streams. Still ConfiguredStreamBuilder is created for DefaultKafkaStreamsConfiguration, but it's not assigned any topic. Hence the error (Kafka 2.5.1):
Error instantiating bean of type [org.apache.kafka.streams.KafkaStreams]: Invalid topology: Topology has no stream threads and no global threads, must subscribe to at least one source topic or global table.

[graemerocher]

can one of you provide an example that reproduces the error

[kascakm]

Here it is
https://github.com/kascakm/micronaut-kafka-issue

[skobow]

I can confirm the behavior @kascakm described. So this does not seem to be solely related to the health endpoint. Is there an extra issue for that? That issue is really severe and took me quite some time to figure out why my streams app does not properly run in my k8s cluster. As a result of this issue the global health check fails and therefore the pods get constantly re-scheduled!