Why is binding better than native implementation?

[sashahohloma]

In our project, we plan to use Kafka for cross-service data exchange, but we are put off by the available choice of clients. Many clients look abandoned, including previously popular ones (for example, kafkajs or node-kafka). The only solutions are node-rdkafka or confluent-kafka-javascript based on librdkafka. There is currently no popular client for Kafka in the NodeJS community with active support without third-party dependencies, unlike RabbitMQ or NATS. Using bindings creates additional complexity when using applications on different platforms, and also requires studying the librdkafka documentation, which can be difficult for NodeJS developers without knowledge of other programming languages.

Is there a chance that an official client based on NodeJS will be introduced in the future?

[nhaq-confluent]

Hey @sashahohloma that is a great question.

As you may know at Confluent, we officially support multiple libraries, including confluent-kafka-go, confluent-kafka-python, confluent-kafka-dotnet, librdkafka and when this library goes GA, it will become officially supported as well.

We have noticed that our strategy of building up in librdkafka and then provide language bindings has shown lots of benefits, including our ability to release all libraries in tandem, be at the forefront of bringing new KIPs from Apache Kafka into the non-Java ecosystem and in many cases, noticeable performance gains. These benefits are why we currently don't foresee the release of any native libraries. Of course, your points about the drawbacks of the binding approach are valid, and we understand in some cases it can introduce additional complexities.

Regarding the documentation point of yours, I would like to hear more about which parts you think can be streamlined or simplified, so we make sure that Node developers can find the available resources effective as they look to use this library.

Thank you.

[sashahohloma]

@nhaq-confluent, thanks for the detailed answer! I understand your argument, but I don't agree with it completely. I haven't done any deep research into the effectiveness of your approach, but I've seen a reddit thread about Go clients where users prefer the popular native solution. An excellent example of the development of official clients can be found in the ClickHouse repository: ClickHouse developers support native clients in popular programming languages without bindings.

Typically, NodeJS technology is chosen due to its wide support for various platforms and standard libraries out of the box, which allows you to focus more attention on developing custom features. In my opinion, the use of bindings creates additional difficulties in the development process and delivery of code to the production environment: the speed of a clean installation increases, additional settings for building into a Docker image are required, there are difficulties with installing bindings for developers with different platforms, additional complexity when studying the features of the library on another programming language

From my experience over the last 3 years, only the official client for Kafka uses bindings and requires Python to install

[nhaq-confluent]

@sashahohloma I completely agree that there are many benefits to native clients.

As you mentioned, the two popular clients today for nodejs are kafkajs (native) and node-rdkafka (binded). With our client, we aim for users to have a way to migrate from both if they so choose.

We aim that despite the typical challenges brought from a binded client, we can still deliver a strong developer experience, and welcome and suggestions, feedback and issues here on GitHub in pursuit of that goal.