KafkaChannel doesn't respect the scope annotation when creating the topic

[aliok]

Describe the bug
When eventing.knative.dev/scope: namespace annotation is available on a channel, KafkaChannel should create the topic in the Kafka cluster that's specified in the config-kafka configmap within the same namespace.
But the topic is created in the Kafka cluster that's specified in the Knative system's knative-eventing/config-kafka configmap.

Expected behavior
Topic should be created in the correct Kafka instance.

To Reproduce

• Create a Kafka cluster in kafka namespace
• Use that cluster in Knative eventing system's knative-eventing/config-kafka configmap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: config-kafka
  namespace: knative-eventing
data:
  bootstrapServers: my-cluster-kafka-bootstrap.kafka:9092

• Create a Kafka cluster in foo namespace
• Use that cluster in foo's knative-eventing/config-kafka configmap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: config-kafka
  namespace: foo
data:
  bootstrapServers: my-cluster-kafka-bootstrap.foo:9092

• Create a channel in foo, with the namespace scope annotation:

apiVersion: messaging.knative.dev/v1alpha1
kind: KafkaChannel
metadata:
  name: my-kafka-channel
  namespace: foo
  annotations:
    eventing.knative.dev/scope: namespace
spec:
  numPartitions: 1
  replicationFactor: 1

• Topic is created in the first Kafka cluster

Knative release version
I reproduced the issue on 0.13, but it is valid in 0.14 too

Additional context
Creating namespace-scoped dispatchers : https://github.com/knative/eventing-contrib/blob/release-0.14/kafka/channel/README.md

Code refs:

• This is the place where the system Kafka config is read:

  eventing-contrib/kafka/channel/pkg/reconciler/controller/controller.go

  Line 90 in 25ab82d

  if _, err := kubeclient.Get(ctx).CoreV1().ConfigMaps(system.Namespace()).Get("config-kafka", metav1.GetOptions{}); err == nil {

• It is passed from here to there, until it is used in the topic creation:

  eventing-contrib/kafka/channel/pkg/reconciler/controller/kafkachannel.go

  Line 176 in 25ab82d

  if err := r.createTopic(ctx, kc, kafkaClusterAdmin); err != nil {

• scope reading happens after the topic creation using the system namespace

[lberk]

/label channel/kafka

[aliok]

/assign

[aliok]

/unassign @aliok

[aliok]

I am unassigning this from myself. There's a decision needs to be done first around if we would like to have this eventing.knative.dev/scope annotation or if we would like to find another way to support the use case.
Fixing this problem is not very hard but not trivial either. Thus I would like to park this.

Threads:

• https://knative.slack.com/archives/C9JP909F0/p1588682851136800
• https://knative.slack.com/archives/CR2141CGY/p1588686195058700

[aliok]

cc @scholzj

[scholzj]

Kafka doesn't have any native support for namespaces. So when the topic operator is enabled and a topic in Kafka is created, it will created a new KafkaTopic custom resource corresponding to the topic in the namespace it is watching. By default, that is the namespace where the Kafka cluster runs (but if a user provided this cluster it can be configured to a different namespace). So this is expected and should not cause any problems. Users who do not want this can also disable the Topic Operator completely and manage the Kafka topics only directly in Kafka. Hope this helps.

[AceHack]

we have multiple kafka clusters and need the ability to connect to different clusters

[AceHack]

Can you please fix this as soon as possible since "Fixing this problem is not very hard". It is causing us bad issues currently.

[AceHack]

@scholzj what do you man disable the Topic Operator? how can you do that? Also, knative does not have a CRD named KafkaTopic. I'm so confused by your comment.

[scholzj]

@scholzj what do you man disable the Topic Operator? how can you do that? Also, knative does not have a CRD named KafkaTopic. I'm so confused by your comment.

The Kafka contrib part is using Kafka from a project called Strimzi. The KafkaTopic resources are created by the Strimzi Topic Operator which can be disabled by removing this line from the Kafka custom resource which defines the Kafka cluster. for the record, I do not know much about Knative so I have no clue whether disabling it is an option or whether Knative uses it in any way.

[AceHack]

@scholzj in no way does knative kafka channel contrib use Strimzi. It creates topics all by itself without any external kafka operator such as Strimzi. For instance, we don't even have Strimzi installed and use knative kafka channel contrib and it works fine. Whenever you create KafkaChannel CRD that IS part of knative kafka channel contrib, it's the knative kafka channel controller creates the kafka topic, Strimizi is not involved at all. This fix needs to happen inside the knative kafka channel controller, not Strimizi.

[scholzj]

I didn't said it is using the KafkaTopic resources to create topics. I the operator would create them when you create a topic in Kafka. Anyway ... that is my understanding why I was cced here. If it is not the case, you can just ignore anything I said.

[anerathil]

Any news on this?

[aliok]

I know that this is valid for 0.17, not sure about the next versions.

Reproducer:

• Install Kafka cluster
• Apply YAML at https://gist.github.com/aliok/4563e9b6ff6539c9f3c10e0d4bdacca5
• Check kubectl get kafkachannels -n ns02 --> You will see that it tries to use the bootstrap server from global config (config-kafka in knative-eventing namespace), which is by default not pointing anywhere

[github-actions]

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Reopen the issue with /reopen. Mark the issue as
fresh by adding the comment /remove-lifecycle stale.

[AceHack]

Bump we still need something like this

[github-actions]

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Reopen the issue with /reopen. Mark the issue as
fresh by adding the comment /remove-lifecycle stale.

[AceHack]

Bump any plans on supporting more than one kafka cluster?

[lionelvillard]

/remove-lifecycle stale

[lionelvillard]

@aliok Does KafkaChannel support this now or is there more to do?

[aliok]

sorry, we still couldn't fix this issue :(

[vaikas]

@aliok would you mind recreating this issue in the eventing-kafka?
This repo is getting archived and I need to close all the issues here.

[aliok]

/close

Moved to eventing-kafka repo: knative-extensions/eventing-kafka#656

[knative-prow-robot]

@aliok: Closing this issue.

In response to this:

/close

Moved to eventing-kafka repo: knative-extensions/eventing-kafka#656

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.