Kafka Connect: augment network policy to allow (externally managed) ingress traffic

[nbarrientos]

Hi,

We'd like to do SSL termination in our own ingress controller and send in-cluster traffic from it to the Kafka Connect API service created by the operator. For this purpose, as the operator only supports ingress with TLS pass-through, we were thinking about declaring our own Ingress resource (unknown to the operator) with the configuration of our liking.

However, for this to work we'd need the NetworkPolicy created by the operator that regulates the traffic to the KC API to allow also ingress traffic from the ingress controller pod running in the kube-system namespace.

Is there any way to add extra rules to the operator-controlled NetworkPolicy or our only option would be to create (and manage) another NP with the extra configuration? I've been looking if the KafkaConnect CRD supports the .listeners bit to add extra networkPolicyPeers there but no dice.

Thanks.

[scholzj]

There is currently no plan for anything like that as we do not support (or recommend given the lack of security) exposing Connect to the outside. However, you can simply create your own additional network policy with your own name and your own rules to extend the default network policy created by the operator. So I don't think you do not really need it.

(Also, please keep in mind that the network policy is created by the operator only when the connector operator is enabled -> in that case, you should also use the Connect REST API only for monitoring tasks as the operator will revert any changes you might make through the REST API.)

[nbarrientos]

Hi @scholzj,

Thanks for the answer! we'll then manage our own NetworkPolicy. Just wanted to confirm that I was not missing something in the docs 😄

Regarding your suggestion about how to use the REST API, yes, indeed we're aware that having two components controlling the connectors might not good idea. We'd like to talk to the Connect API externally mainly for monitoring purposes as you're hinting and for instance to be able to restart failed tasks from an UI (Kafbat).

Thanks for your time and help.

[scholzj]

Thanks for the answer! we'll then manage our own NetworkPolicy. Just wanted to confirm that I was not missing something in the docs 😄

No, you did not miss anything in the docs. I suspect that once we manage to find time to provide some better security for the REST API we will likely offer some improved management for the network policy as it is with the Kafka broker listeners. But until then, your own network policy should work fine.