Go Kafka ES client

A go CLI that,

• listens to an event through a HTTP server at endpoint '/event'

• produces messages to a Kafka topic

• consumes messages from a Kafka topic

• sends the messages to a Elasticsearch server

Requirements

• Running instance of Kafka on usual ports (for simplicity, you can run the docker container)

    sudo docker run --rm --net=host landoop/fast-data-dev

• Running instance of Elasticsearch on usual ports

To Run the application

• Make a new file run.env & copy the contents of run.env.sample to run.env. Make sure, you supply the required environment varibles in run.env.

  You need to change the run.sh to run two separate servers for Kafka producers and Kafka consumers

• Run a HTTP server and produce event to Kafka

    ./kafka produce

• Consume message from Kafka server and send it to Elasticsearch

    ./kafka consume

Build & run the program

    ./dev.sh

You can run test_messages to send dummy message in running Kafka producer server.