Unable to start Kafka server on Azure VMs using Terraform

[odidev]

Is there an existing issue for this?

• I have searched the existing issues

Where would you like this feature added?

Other/Unknown

Is this feature ask due to a problem that you are encountering?

While working to deploy Kafka cluster on Azure VMs using Terraform and Ansible, I am facing issues while creating a topic, I have checked and found that all three Zookeeper servers are running in cluster but one of the Kafka servers is not getting started in cluster.

VMs used: Dpsv5 series

OS: Ubuntu 20.04

Architecture: amd64 and arm64

The error which I am getting is:

[2023-05-30 08:35:45,623] INFO [Controller id=1, targetBrokerId=1] Node 1 disconnected. (org.apache.kafka.clients.NetworkClient) 

[2023-05-30 08:35:45,623] WARN [Controller id=1, targetBrokerId=1] Connection to node 1 (myvm.internal.cloudapp.net/10.1.0.9:9093) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient) 

[2023-05-30 08:35:45,623] INFO [Controller id=1, targetBrokerId=1] Client requested connection close from node 1 (org.apache.kafka.clients.NetworkClient) 

The code written to create the resources through terraform in main.tf file:

main.txt
providers.txt
variables.txt
zookeeper_cluster.txt
kafka_cluster.txt

Steps followed to deploy the kafka cluster.

Run the following commands to create resources through terraform:

terraform init 

  

terraform apply 

To know the ip of kafka servers:

cat /tmp/inventery 

To install the Zookeeper and the required dependencies and start zookeeper server:

ansible-playbook zookeeper_cluster.yaml -i /tmp/inventory 

To install the Kafka and the required dependencies:

ansible-playbook kafka_cluster.yaml -i /tmp/inventory 

To start kafka on all three server first do ssh on all three machines on which kafka is installed and run below command:

bin/kafka-server-start.sh config/server.properties 

However, When I have created virtual machines manually(not through Terraform) and installed zookeeper and kafka in them with the same configuration and tried running Kafka cluster and it is running fine.

It will be really helpful if you share pointers on the above mentioned issue.

Describe potential solutions.

When I have created virtual machines manually(not through Terraform) and installed zookeeper and kafka in them with the same configuration and tried running Kafka cluster and it is running fine.

Anything else?

No response

[zubair3134]

Did you try connecting to the broker 1?