Skip to content

Latest commit

 

History

History
46 lines (33 loc) · 2.83 KB

README.md

File metadata and controls

46 lines (33 loc) · 2.83 KB

Churro - ETL for Kubernetes

churro is a cloud-native Extract-Transform-Load (ETL) application designed to build, scale, and manage data pipeline applications.

What is churro?

churro is an application that processes input files and streams by extracting content and loading that content into a database of your choice, all running on Kubernetes. Today, churro supports processing of JSON, XML, XLSX, CSV files along with JSON API streams. End users create churro pipelines to process data, those pipelines can be created using the churro web app or via a churro pipeline custom resource (yaml) directly.

Design

Some key aspects of the churro design:

  • churro is designed from the start to run within a Kubernetes cluster.
  • churro uses a micro-service architecture to scale ETL processing
  • churro has extension points defined to allow for customized processing to be performed per customer requirements.
  • churro is written in golang
  • churro currently supports persisting ingested data into cockroachdb, singlestore, and mysql databases
  • churro implements a kubernetes operator to handle git-ops style provisioning of churro pipeline resources including the pipeline database

For more details on the churro design, checkout out the documentation at the churro github pages.

Docs

Detailed documentation is found at the churro github pages, additional content such as blogs can be found at the churrodata.com web site.

Images

Today, churro container images are found on DockerHub here. These images are muulti-arch manifest images that include builds for both amd64 and arm64 architectures.

churro Cloud

Inquires about churro Cloud can be directed to info@churrodata.com. In the near future, users will be able to provision a churro instance on the cloud of their choice, with billing and management handled by churrodata.com

Starting with churro

People generally start with churro by creating a kubernetes cluster, then deploying churro to your running cluster. Installation documentation is found on the churro github pages.

Contributing

Since churro is open source, you can view the source code and make contributions such as pull requests on our github.com site. We encourage users to log any issues they find on our github issues site.

Support

churro enterprise support and services are provided by churrodata.com.