Data Engineering examples for Airflow, Prefect, and Mage.ai; dbt for BigQuery, Redshift, ClickHouse, PostgreSQL; Spark/PySpark for Batch processing; and Kafka for Stream processing

A simple demo showing how to use Ably and fastAPI to route messages into Kafka for stream processing

Current 2022 Confluent Keynote Demo covering Stream Designer, Stream Catalog, and Stream Sharing.

Stream smartphone sensor data with FastAPI, Kafka, ksqlDB, and Docker.

Analyze Zeek IDS data with ksqlDB running on Confluent Platform via Docker on your laptop. Or spin up an arbitrary number of AWS hosts, each running Confluent Platform and ksqlDB for use in an instructor-led workshop.

For recreational use. Just a playground of Kafka+Spark+MQTT+KSQLDB+others

Interactive ksqlDB command line client with autocompletion and syntax highlighting written in Python

A Python wrapper for the KSQLdb REST API

Pythonic KSQL REST API - Next Gen.

Real-time Coinbase market data streaming pipeline with visualizations. Much appreciation to DataTalks.Club Data Engineering Zoom Camp: https://github.com/DataTalksClub/data-engineering-zoomcamp

Kafka Connect and kSQLDB with Oracle

An app to keep track of Youtube videos and sends the notification to a Telegram bot to inform you if anyone comments on those

Data streaming with Python and Apache Kafka

Streaming event pipeline around Apache Kafka and its ecosystem, simulating Real-time Data Streaming

Free and simple way to interact with ksqlDB using UI

This project demonstrates a modern ETL (Extract, Transform, Load) streaming pipeline using various open-source technologies.

Kubernetes demo

Building a streaming Kafka application to push live notifications for updates to views, likes, favorites and comments on Youtube videos.

Real time fraud analysis using Kafka Streams

This repository contains a KSQLDB setup connection that streams the average ETH Gas Estimate using the Ethereum Gas Estimate API as source data.