Skip to content

hopoffbaby/k8s-strimzi-spark

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

80 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

A collection of examples for Strimzi/Kafka, Spark, and Volcano

Goals

Kafka

  • Install Strimzi

  • Deploy cluster

  • Produce / consume messages

  • Adjust number of brokers

  • Benchmark perfomace

  • Deploy Kafka UI

  • Topic compression

  • Backup / restore

  • OIDC

  • Rack awareness

  • Monitoring

  • Cruise Control

  • Quotas

  • Retention policies

  • Scale up/down

  • Mirror maker

  • autoscaling brokers

  • Karapace deployment

  • Karapace security

Minio

  • single node single disk deploy to k8s
  • bucket notifications into kafka

VAP emulator

  • Object input
  • Object output
  • Auto scaling
  • ffmpeg process
  • Get args from kafka

S3 <--> POSIX

Smart guy

Smart guys video

S3 == eventual consistency

S3Proxy

S3Proxy - access a file system using the S3 API

Native Object Access

S3FS

s3fs

Goofys

Goofys

Faster but less POSIX-y

rclone

rclone

like rsync for cloud, but offers a fuse mount option

Weka S3

based on MinIO - allows s3 access to SMB/NFS/WekaFS data

CunoFS

Paid solution

Block S3 access

s3ql

treats object store like block device. Lacks native object access.

JuiceFS

treats like block storage.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published