📚 Learning and experimenting with Apache Kafka https://kafka.apache.org/.
This repository illustrates different concepts, patterns and examples via standalone subprojects. Each subproject is completely independent of the others and do not depend on the root project. This standalone subproject constraint forces the subprojects to be complete and maximizes the reader's chances of successfully running, understanding, and re-using the code.
The subprojects include:
An interactive program to consume from Kafka using the plain-ole'/regular/vanilla Java KafkaConsumer.
TIP: This is a good project to start with you if you are just learning about Kafka, or more specifically you are learning how to interface with Kafka via a Java program.
See the README in interactive/.
A comparison of algorithms for consuming messages from Kafka: sequential, partial parallelism, and asynchronous.
See the README in kafka-consumer-algorithms/.
Use the Java Kafka client to check for a connection to a Kafka cluster. Sometimes, this is called a health check.
See the README in connection-check/.
A basic Kafka Streams application.
See the README in streams/.
An intermediate Kafka Streams project that aggregates ZIP code data.
See the README in streams-zip-codes/.
A basic Spring Kafka application with a "seekable" Kafka listener.
See the README in spring-seekable/.
A basic Spring Kafka application that showcases the Spring framework behavior around Kafka message headers.
See the README in spring-headers/.
A basic Spring Kafka application that showcases the Spring framework features and behavior around Kafka error handling.
See the README in spring-errors/.
A simple Java program to process messages from a Kafka topic using abstractions from Spring for Apache Kafka.
See the README in spring-barebones/.
A Spring Kafka application that consumes from multiple Kafka brokers.
See the README in spring-multi-broker/.
Utility Bash scripts for starting and stopping Kafka.
See the README in utility-scripts/.
General clean-ups, TODOs and things I wish to implement for this project:
- Consider a subproject that shows metrics. Kafka I think already has JMX metrics or something, but I'm not totally up to speed on that.