Skip to content
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 61 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,34 +1,26 @@
Confluent's Javascript Client for Apache Kafka<sup>TM</sup>
=====================================================

**confluent-kafka-js** is Confluent's Javascript client for [Apache Kafka](http://kafka.apache.org/) and the
[Confluent Platform](https://www.confluent.io/product/compare/).
**confluent-kafka-js** is Confluent's JavaScript client for [Apache Kafka](http://kafka.apache.org/) and the
[Confluent Platform](https://www.confluent.io/product/compare/). This is an **early access** library. The goal is to provide an highly performant, reliable and easy to use JavaScript client that is API compatible with [KafkaJS](https://github.com/tulios/kafkajs) to provide flexibility to users and streamline migrations from other clients.

This library leverages the work and concepts from two popular Apache Kafka JavaScript clients: [node-rdkafka](https://github.com/Blizzard/node-rdkafka) and [KafkaJS](https://github.com/tulios/kafkajs). The core is heavily based on the node-rdkafka library, which uses our own [librdkafka](https://github.com/confluentinc/librdkafka/tree/v2.3.0) library for core client functionality. However, we leverage a promisified API and a more idiomatic interface, similar to the one in KafkaJS, making it easy for developers to migrate and adopt this client depending on the patterns and interface they prefer.
__This library currently uses `librdkafka` based off of the master branch.__

Features:

- **High performance** - confluent-kafka-js is a lightweight wrapper around
[librdkafka](https://github.com/confluentinc/librdkafka), a finely tuned C
client.

- **Reliability** - There are a lot of details to get right when writing an Apache Kafka
client. We get them right in one place (librdkafka) and leverage this work
across all of our clients (also [confluent-kafka-python](https://github.com/confluentinc/confluent-kafka-python),
[confluent-kafka-go](https://github.com/confluentinc/confluent-kafka-go) and
and [confluent-kafka-dotnet](https://github.com/confluentinc/confluent-kafka-dotnet)).
## This library is currently in early access and not meant for production use

- **Future proof** - Confluent, founded by the
creators of Kafka, is building a [streaming platform](https://www.confluent.io/product/compare/)
with Apache Kafka at its core. It's high priority for us that client features keep
pace with core Apache Kafka and components of the [Confluent Platform](https://www.confluent.io/product/compare/).
**This library is in active development, pre-1.0.0, and it is likely to have many breaking changes.**

## This library is currently not ready for production use. It's an early-access preview in active development, pre-1.0.0, and there might be breaking changes.
For this early-access release, we aim to get feedback from JavaScript developers within the Apache Kafka community to help meet your needs. Some areas of feedback we are looking for include:
- Usability of the API compared to other clients
- Migration experience from the node-rdkafka and KafkaJs
- Overall quality and reliability

This library is based heavily on [node-rdkafka](https://github.com/Blizzard/node-rdkafka).
We invite you to raise issues to highlight any feedback you may have.

This library contains a promisified API, very similar to the one in [kafkajs](https://github.com/tulios/kafkajs). Some of the tests are also based on the ones in kafkajs.
Within the early-access, only **basic produce and consume functionality** as well as the ability to **create and delete topics** are supported. All other admin client functionality is coming in future releases. See [INTRODUCTION.md](INTRODUCTION.md) for more details on what is supported.

__This library currently uses `librdkafka` based off of the master branch.__
To use **Schema Registry**, use the existing [kafkajs/confluent-schema-registry](https://github.com/kafkajs/confluent-schema-registry) library that is compatible with this library. For a simple schema registry example, see [sr.js](https://github.com/confluentinc/confluent-kafka-js/blob/dev_early_access_development_branch/examples/kafkajs/sr.js).


## Requirements
Expand All @@ -39,8 +31,7 @@ The following configurations are supported for this early access preview:
* Linux (x64 and arm64) - both glibc and musl/alpine.
* macOS - arm64/m1.

Installation on any of these platforms is meant to be seamless, without any C/C++ compilation required. It can be installed
from GitHub:
Installation on any of these platforms is meant to be seamless, without any C/C++ compilation required. It can be installed from GitHub:

```bash
$ npm install "git+ssh://git@github.com/confluentinc/confluent-kafka-js.git#v0.1.5-devel"
Expand All @@ -50,6 +41,53 @@ Yarn and pnpm support is experimental.

# Getting Started

Below is a simple produce example for users migrating from KafkaJS.

```
// require('kafkajs') is replaced with require('confluent-kafka-js').KafkaJS.
// Since this example is within the package itself, we use '../..', but code
// will typically use 'confluent-kafka-js'.
const { Kafka } = require("confluent-kafka-js").KafkaJS;

async function producerStart() {
const kafka = new Kafka({
kafkaJS: {
brokers: ['<fill>'],
ssl: true,
sasl: {
mechanism: 'plain',
username: '<fill>',
password: '<fill>',
},
}
});

const producer = kafka.producer();

await producer.connect();

console.log("Connected successfully");

const res = []
for (let i = 0; i < 50; i++) {
res.push(producer.send({
topic: 'test-topic',
messages: [
{ value: 'v222', partition: 0 },
{ value: 'v11', partition: 0, key: 'x' },
]
}));
}
await Promise.all(res);

await producer.disconnect();

console.log("Disconnected successfully");
}

producerStart();
```

1. If you're migrating from `kafkajs`, you can use the [migration guide](MIGRATION.md#kafkajs).
2. If you're migrating from `node-rdkafka`, you can use the [migration guide](MIGRATION.md#node-rdkafka).
3. If you're starting afresh, you can use the [quickstart guide](QUICKSTART.md).
Expand Down