Skip to content

Latest commit

 

History

History
86 lines (55 loc) · 1.47 KB

README.md

File metadata and controls

86 lines (55 loc) · 1.47 KB

A collection of examples for Strimzi/Kafka, Spark, and Volcano

Goals

Kafka

  • Install Strimzi

  • Deploy cluster

  • Produce / consume messages

  • Adjust number of brokers

  • Benchmark perfomace

  • Deploy Kafka UI

  • Topic compression

  • Backup / restore

  • OIDC

  • Rack awareness

  • Monitoring

  • Cruise Control

  • Quotas

  • Retention policies

  • Scale up/down

  • Mirror maker

  • autoscaling brokers

  • Karapace deployment

  • Karapace security

Minio

  • single node single disk deploy to k8s
  • bucket notifications into kafka

VAP emulator

  • Object input
  • Object output
  • Auto scaling
  • ffmpeg process
  • Get args from kafka

S3 <--> POSIX

Smart guy

Smart guys video

S3 == eventual consistency

S3Proxy

S3Proxy - access a file system using the S3 API

Native Object Access

S3FS

s3fs

Goofys

Goofys

Faster but less POSIX-y

rclone

rclone

like rsync for cloud, but offers a fuse mount option

Weka S3

based on MinIO - allows s3 access to SMB/NFS/WekaFS data

CunoFS

Paid solution

Block S3 access

s3ql

treats object store like block device. Lacks native object access.

JuiceFS

treats like block storage.