Kafka Streams Demo

This project is meant to explain how Kafka Streams and KsqlDB works

Open Kafka streams tutorial in the docs folder.

For more information see Kafka Streams docs and Ksql DB reference

How to run

start containers

cd docker
docker-compose up -d

create input and output topics

create input-topics

docker exec -it broker kafka-topics \
    --bootstrap-server localhost:9092 \
    --create \
    --partitions 3 \
    --topic 'input-topic-A'

docker exec -it broker kafka-topics \
    --bootstrap-server localhost:9092 \
    --create \
    --partitions 3 \
    --topic 'input-topic-B'

docker exec -it broker kafka-topics \
    --bootstrap-server localhost:9092 \
    --create \
    --partitions 3 \
    --topic 'input-topic-message'

docker exec -it broker kafka-topics \
    --bootstrap-server localhost:9092 \
    --config "cleanup.policy=compact" \
    --create \
    --partitions 3 \
    --topic 'input-topic-lookup'

create output-topic

docker exec -it broker kafka-topics \
    --bootstrap-server localhost:9092 \
    --create \
    --partitions 3 \
    --topic 'output-topic'

create dead-end-topic

docker exec -it broker kafka-topics \
    --bootstrap-server localhost:9092 \
    --create \
    --partitions 3 \
    --topic 'dead-end-topic'

start application

Optional step: restore gradle wrapper

gradle wrapper

start solution

./gradlew run

Once you finish, clean up all containers

stop and remove containers

cd docker
docker-compose down