Skip to content

jmks/kconnectex

Folders and files

NameName
Last commit message
Last commit date

Latest commit

330df62 · Oct 28, 2024
Oct 24, 2024
Oct 24, 2024
Nov 27, 2020
Jan 25, 2022
Jul 5, 2021
Jun 4, 2021
Sep 14, 2022
Oct 24, 2024
Sep 8, 2021
Aug 30, 2021
Jan 25, 2022
Feb 1, 2022
Sep 8, 2021
Aug 27, 2021

Repository files navigation

kconnectex

Elixir API wrapper for the Kafka Connect API

Currently under development.

How To Use

Use as an Elixir library

client = Kconnectex.client "https://domain-and-port-to-kafka-connect-cluster"

{:ok, connectors} = Kconnectex.Connectors.list(client)

As a CLI

This app includes escript configuration. The CLI is inspired by kaf for Kafka.

Build it and display the help:

$ mix escript.build
$ ./kconnectex --help

Run CLI via Docker

If you don't have Elixir on your system, you can run the CLI via Docker.

A docker image is provided and is hosted at quay. A wrapper is also provided (kconnectex_docker_wrapper) to execute it in Docker.

$ cp ~/path/to/repo/kconnectex_docker_wrapper /somewhere/on/system/PATH/kconnectex
$ kconnectex cluster
{
  "commit": "6b2021cd52659cef",
  "kafka_cluster_id": "dK2QBCSU",
  "version": "2.6.1"
}

TODO

General

  • Having a "client" and "request" seems redundant?
  • Add more integration tests (with FileStreamSource)
  • Validating a config where the connector name does not match config => 500 from Connect (bug?)
  • docs for Connectors
  • specs

Connectors

  • Support "expand" queries
    • Where are docs on these?

Topics

  • get
  • reset

CLI

  • restore iex -S mix functionality
  • fix exit statuses
  • Make errors uniform - error type?
  • completion command to generate script for bash, zsh
  • strictly output JSON (possible?)

Development

To run the unit tests:

$ mix test

To run the integration tests:

$ docker-compose up
$ mix test --include integration

Release

You permission to push to the Docker repository (currently, just the author).

A script will release the CLI with a tag of the current version: ./release

Installation

It's not in hex yet, but the adventurous can grab it from Github:

def deps do
  [
    {:kconnectex, git: "https://github.com/jmks/kconnectex.git"}
  ]
end

If available in Hex, the package can be installed by adding kconnectex to your list of dependencies in mix.exs:

def deps do
  [
    {:kconnectex, "~> 0.2.0"}
  ]
end

Documentation can be generated with ExDoc and published on HexDocs. Once published, the docs can be found at https://hexdocs.pm/kconnectex.

About

Elixir API wrapper for the Kafka Connect API

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages