-
Notifications
You must be signed in to change notification settings - Fork 377
Globally enhance Kafka documentation #8291
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
91b1730
Globally enhance Kafka documentation
thild42 aa466a0
Update Getting Started
thild42 1a04f6a
Remove Paris 3AZ from Cassandra
thild42 36cb11e
EN proof
d9310c6
Resize Kafka ACL image
thild42 06b13a7
Date update + Links update
Y0Coss File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| - [de-de](https://www.ovhcloud.com/de/public-cloud/apache-kafka/) | ||
| - [en-asia](https://www.ovhcloud.com/asia/public-cloud/apache-kafka/) | ||
| - [en-au](https://www.ovhcloud.com/en-au/public-cloud/apache-kafka/) | ||
| - [en-ca](https://www.ovhcloud.com/en-ca/public-cloud/apache-kafka/) | ||
| - [en-gb](https://www.ovhcloud.com/en-gb/public-cloud/apache-kafka/) | ||
| - [en-ie](https://www.ovhcloud.com/en-ie/public-cloud/apache-kafka/) | ||
| - [en-in](https://www.ovhcloud.com/en-in/public-cloud/apache-kafka/) | ||
| - [en-sg](https://www.ovhcloud.com/en-sg/public-cloud/apache-kafka/) | ||
| - [en-us](https://www.ovhcloud.com/en/public-cloud/apache-kafka/) | ||
| - [es-es](https://www.ovhcloud.com/es-es/public-cloud/apache-kafka/) | ||
| - [es-us](https://www.ovhcloud.com/es/public-cloud/apache-kafka/) | ||
| - [fr-ca](https://www.ovhcloud.com/fr-ca/public-cloud/apache-kafka/) | ||
| - [fr-fr](https://www.ovhcloud.com/fr/public-cloud/apache-kafka/) | ||
| - [it-it](https://www.ovhcloud.com/it/public-cloud/apache-kafka/) | ||
| - [pl-pl](https://www.ovhcloud.com/pl/public-cloud/apache-kafka/) | ||
| - [pt-pt](https://www.ovhcloud.com/pt/public-cloud/apache-kafka/) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| - [de-de](https://www.ovhcloud.com/de/public-cloud/prices/#7211) | ||
| - [en-asia](https://www.ovhcloud.com/asia/public-cloud/prices/#7211) | ||
| - [en-au](https://www.ovhcloud.com/en-au/public-cloud/prices/#7211) | ||
| - [en-ca](https://www.ovhcloud.com/en-ca/public-cloud/prices/#7211) | ||
| - [en-gb](https://www.ovhcloud.com/en-gb/public-cloud/prices/#7211) | ||
| - [en-ie](https://www.ovhcloud.com/en-ie/public-cloud/prices/#7211) | ||
| - [en-in](https://www.ovhcloud.com/en-in/public-cloud/prices/#7211) | ||
| - [en-sg](https://www.ovhcloud.com/en-sg/public-cloud/prices/#7211) | ||
| - [en-us](https://www.ovhcloud.com/en/public-cloud/prices/#7211) | ||
| - [es-es](https://www.ovhcloud.com/es-es/public-cloud/prices/#7211) | ||
| - [es-us](https://www.ovhcloud.com/es/public-cloud/prices/#7211) | ||
| - [fr-ca](https://www.ovhcloud.com/fr-ca/public-cloud/prices/#7211) | ||
| - [fr-fr](https://www.ovhcloud.com/fr/public-cloud/prices/#7211) | ||
| - [it-it](https://www.ovhcloud.com/it/public-cloud/prices/#7211) | ||
| - [pl-pl](https://www.ovhcloud.com/pl/public-cloud/prices/#7211) | ||
| - [pt-pt](https://www.ovhcloud.com/pt/public-cloud/prices/#7211) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
50 changes: 50 additions & 0 deletions
50
pages/public_cloud/data_analytics/analytics/kafka_configure_acl/guide.en-gb.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| --- | ||
| title: Kafka - How to use Access Control Lists (ACLs) | ||
| excerpt: Learn how to configure Access Control Lists (ACLs) | ||
| updated: 2025-08-25 | ||
| --- | ||
|
|
||
| ## Objective | ||
|
|
||
| Apache Kafka is an open-source, distributed event streaming platform designed for real-time, large-scale data processing with high scalability, durability, and low latency. | ||
|
|
||
| This guide explains how to configure Access Control Lists (ACLs) via the OVHcloud Control Panel. | ||
|
|
||
| ## Requirements | ||
|
|
||
| - Access to the [OVHcloud Control Panel](/links/manager) | ||
| - A [Public Cloud project](/links/public-cloud/public-cloud) in your OVHcloud account | ||
| - A [Kafka cluster running](/pages/public_cloud/data_analytics/analytics/kafka_create_cluster) on OVHcloud Public Cloud [accepting incoming connections](/pages/public_cloud/data_analytics/analytics/kafka_incoming_connections) with at least one [topic](/pages/public_cloud/data_analytics/analytics/kafka_create_topics) | ||
|
|
||
| ## Instructions | ||
|
|
||
| ### Configure ACLs on topics | ||
|
|
||
| Kafka supports access control lists (ACLs) to manage permissions on topics. This approach allows you to limit the operations that are available to specific connections and to restrict access to certain data sets, which improves the security of your data. | ||
|
|
||
| By default the admin user has access to all topics with admin privileges. You can define some additional ACLs for all users / topics, by clicking on the `Add an ACL`{.action} button from the `ACL`{.action} tab: | ||
|
|
||
| {.thumbnail} | ||
|
|
||
| For a particular user, and one topic (or all with '*'), define the ACL with the following permissions: | ||
|
|
||
| - **admin**: full access to APIs and topic | ||
| - **read**: allow only searching and retrieving data from a topic | ||
| - **write**: allow updating, adding, and deleting data from a topic | ||
| - **readwrite**: full access to the topic | ||
|
|
||
| {.thumbnail} | ||
|
|
||
| *Note*: Write permission allows the service user to create new indexes that match the pattern, but it does not allow deletion of those indexes. | ||
|
|
||
| When multiple rules match, they are applied in the order listed above. If no rules match, access is denied. | ||
|
|
||
| ## We want your feedback! | ||
|
|
||
| We would love to help answer questions and appreciate any feedback you may have. | ||
|
|
||
| If you need training or technical assistance to implement our solutions, contact your sales representative or click on [this link](/links/professional-services) to get a quote and ask our Professional Services experts for a custom analysis of your project. | ||
|
|
||
| Are you on Discord? Connect to our channel at <https://discord.gg/ovhcloud> and interact directly with the team that builds our Analytics service! | ||
|
|
||
| Join our [community of users](/links/community). |
Binary file added
BIN
+26.3 KB
...blic_cloud/data_analytics/analytics/kafka_configure_acl/images/kafka_acl.v2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
File renamed without changes
4 changes: 4 additions & 0 deletions
4
pages/public_cloud/data_analytics/analytics/kafka_configure_acl/meta.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| id: 68436db5-0ad3-400c-b1b9-eb137c18f6f7 | ||
| full_slug: data-analytics-kafka-configure-acl | ||
| engine: kafka | ||
| reference_category: public-cloud-data-analytics-kafka-guides |
101 changes: 101 additions & 0 deletions
101
.../public_cloud/data_analytics/analytics/kafka_connect_cluster_cli/guide.en-gb.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,101 @@ | ||
| --- | ||
| title: Kafka - How to connect to a Kafka cluster with CLI | ||
| excerpt: Learn how to connect to a Kafka cluster using the CLI | ||
| updated: 2025-08-25 | ||
| --- | ||
|
|
||
| ## Objective | ||
|
|
||
| Apache Kafka is an open-source, distributed event streaming platform designed for real-time, large-scale data processing with high scalability, durability, and low latency. | ||
|
|
||
| This guide explains how to connect to a Kafka cluster using the CLI. | ||
|
|
||
| ## Requirements | ||
|
|
||
| - Access to the [OVHcloud Control Panel](/links/manager) | ||
| - A [Public Cloud project](/links/public-cloud/public-cloud) in your OVHcloud account | ||
| - A [Kafka cluster running](/pages/public_cloud/data_analytics/analytics/kafka_create_cluster) on OVHcloud Public Cloud [accepting incoming connections](/pages/public_cloud/data_analytics/analytics/kafka_incoming_connections) | ||
|
|
||
| ## Instructions | ||
|
|
||
| ### First CLI connection | ||
|
|
||
| > [!warning] | ||
| > Verify that the IP address visible from your browser application is part of the "Authorised IPs" defined for this Kafka service. | ||
| > | ||
| > Check also that the user has granted ACLs for the target topics. | ||
|
|
||
| #### Download server and user certificates | ||
|
|
||
| In order to connect to the Apache Kafka service, it is required to use server and user certificates. | ||
|
|
||
| ##### Server certificate | ||
|
|
||
| The server CA (*Certificate Authority*) certificate can be downloaded from the `Dashboard`{.action} tab: | ||
|
|
||
| {.thumbnail} | ||
|
|
||
| ##### User certificate and access key | ||
|
|
||
| The user certificate and the user access key can be downloaded from the `Users`{.action} tab: | ||
|
|
||
| {.thumbnail} | ||
|
|
||
| #### Install an Apache Kafka CLI | ||
|
|
||
| As part of the Apache Kafka official installation, you will get different scripts that will also allow you to connect to Kafka in a Java 8+ environment: [Apache Kafka Official Quickstart](https://kafka.apache.org/quickstart). | ||
|
|
||
| We propose to use a generic and more lightweight (does not require a JVM) producer and consumer client instead: `Kcat` (formerly known as `kafkacat`). | ||
|
|
||
| ##### **Install Kcat** | ||
|
|
||
| For this client installation, please follow the instructions available at: [Kafkacat Official Github](https://github.com/edenhill/kcat). | ||
|
|
||
| ##### **Kcat configuration file** | ||
|
|
||
| Let's create a configuration file to simplify the CLI commands to act as Kafka Producer and Consumer: | ||
|
|
||
| kafkacat.conf: | ||
|
|
||
| ```text | ||
| bootstrap.servers=kafka-f411d2ae-f411d2ae.database.cloud.ovh.net:20186 | ||
| enable.ssl.certificate.verification=false | ||
| ssl.ca.location=/home/user/kafkacat/ca.pem | ||
| security.protocol=ssl | ||
| ssl.key.location=/home/user/kafkacat/service.key | ||
| ssl.certificate.location=/home/user/kafkacat/service.cert | ||
| ``` | ||
|
|
||
| In our example, the cluster address and port are **kafka-f411d2ae-f411d2ae.database.cloud.ovh.net:20186** and the previously downloaded CA certificates are in the **/home/user/kafkacat/** folder. | ||
|
|
||
| Change theses values according to your own configuration. | ||
|
|
||
| ##### **Kafka producer** | ||
|
|
||
| For this first example let's push the "test-message-key" and its "test-message-content" to the "my-topic" topic. | ||
|
|
||
| ```bash | ||
| echo test-message-content | kcat -F kafkacat.conf -P -t my-topic -k test-message-key | ||
| ``` | ||
|
|
||
| *Note*: depending on the installed binary, the CLI command can be either **kcat** or **kafkacat**. | ||
|
|
||
| ##### **Kafka consumer** | ||
|
|
||
| The data can be retrieved from "my-topic". | ||
|
|
||
| ```bash | ||
| kcat -F kafkacat.conf -C -t my-topic -o -1 -e | ||
| ``` | ||
|
|
||
| *Note*: depending on the installed binary, the CLI command can be either **kcat** or **kafkacat**. | ||
|
|
||
| ## We want your feedback! | ||
|
|
||
| We would love to help answer questions and appreciate any feedback you may have. | ||
|
|
||
| If you need training or technical assistance to implement our solutions, contact your sales representative or click on [this link](/links/professional-services) to get a quote and ask our Professional Services experts for a custom analysis of your project. | ||
|
|
||
| Are you on Discord? Connect to our channel at <https://discord.gg/ovhcloud> and interact directly with the team that builds our Analytics service! | ||
|
|
||
| Join our [community of users](/links/community). |
File renamed without changes
File renamed without changes
4 changes: 4 additions & 0 deletions
4
pages/public_cloud/data_analytics/analytics/kafka_connect_cluster_cli/meta.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| id: 39f9dc6c-0985-4ab5-b379-1f8053f073c0 | ||
| full_slug: data-analytics-kafka-connect-cli | ||
| engine: kafka | ||
| reference_category: public-cloud-data-analytics-kafka-guides |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.