Skip to content

Commit dc7f7c6

Browse files
momo-junnicklixinyang
authored and
nicklixinyang
committed
[Doc] Add information for RocksDB and ETCD (apache#13830)
* [Doc] Initial commit for RocksDB and ETCD 1. Add RocksDB (standalone only) and ETCD as alternative backend metadata service. 2. Add a placeholder for RocksDB parameter list. * Add reference information for RocksDB configurations.
1 parent 007f290 commit dc7f7c6

3 files changed

+9
-4
lines changed

site2/docs/concepts-architecture-overview.md

+3
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,9 @@ Clusters can replicate amongst themselves using [geo-replication](concepts-repli
4747

4848
The Pulsar metadata store maintains all the metadata of a Pulsar cluster, such as topic metadata, schema, broker load data, and so on. Pulsar uses [Apache ZooKeeper](https://zookeeper.apache.org/) for metadata storage, cluster configuration, and coordination. The Pulsar metadata store can be deployed on a separate ZooKeeper cluster or deployed on an existing ZooKeeper cluster. You can use one ZooKeeper cluster for both Pulsar metadata store and [BookKeeper metadata store](https://bookkeeper.apache.org/docs/latest/getting-started/concepts/#metadata-storage). If you want to deploy Pulsar brokers connected to an existing BookKeeper cluster, you need to deploy separate ZooKeeper clusters for Pulsar metadata store and BookKeeper metadata store respectively.
4949

50+
> Pulsar also supports more metadata backend services, including [ETCD](https://etcd.io/) and [RocksDB](http://rocksdb.org/) (for standalone Pulsar only).
51+
52+
5053
In a Pulsar instance:
5154

5255
* A configuration store quorum stores configuration for tenants, namespaces, and other entities that need to be globally consistent.

site2/docs/getting-started-standalone.md

+3-3
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ title: Set up a standalone Pulsar locally
44
sidebar_label: Run Pulsar locally
55
---
66

7-
For local development and testing, you can run Pulsar in standalone mode on your machine. The standalone mode includes a Pulsar broker, the necessary ZooKeeper and BookKeeper components running inside of a single Java Virtual Machine (JVM) process.
7+
For local development and testing, you can run Pulsar in standalone mode on your machine. The standalone mode includes a Pulsar broker, the necessary [RocksDB](http://rocksdb.org/) and BookKeeper components running inside of a single Java Virtual Machine (JVM) process.
88

99
> **Pulsar in production?**
1010
> If you're looking to run a full production Pulsar installation, see the [Deploying a Pulsar instance](deploy-bare-metal.md) guide.
@@ -53,7 +53,7 @@ The Pulsar binary package initially contains the following directories:
5353
Directory | Contains
5454
:---------|:--------
5555
`bin` | Pulsar's command-line tools, such as [`pulsar`](reference-cli-tools.md#pulsar) and [`pulsar-admin`](https://pulsar.apache.org/tools/pulsar-admin/).
56-
`conf` | Configuration files for Pulsar, including [broker configuration](reference-configuration.md#broker), [ZooKeeper configuration](reference-configuration.md#zookeeper), and more.
56+
`conf` | Configuration files for Pulsar, including [broker configuration](reference-configuration.md#broker) and more.<br />**Note:** Pulsar standalone uses RocksDB as the local metadata store and its configuration file path [`metadataStoreConfigPath`](reference-configuration.md) is configurable in the `standalone.conf` file. For more information about the configurations of RocksDB, see [here](https://github.com/facebook/rocksdb/blob/main/examples/rocksdb_option_file_example.ini) and related [documentation](https://github.com/facebook/rocksdb/wiki/RocksDB-Tuning-Guide).
5757
`examples` | A Java JAR file containing [Pulsar Functions](functions-overview.md) example.
5858
`instances` | Artifacts created for [Pulsar Functions](functions-overview.md).
5959
`lib` | The [JAR](https://en.wikipedia.org/wiki/JAR_(file_format)) files used by Pulsar.
@@ -63,7 +63,7 @@ These directories are created once you begin running Pulsar.
6363

6464
Directory | Contains
6565
:---------|:--------
66-
`data` | The data storage directory used by ZooKeeper and BookKeeper.
66+
`data` | The data storage directory used by RocksDB and BookKeeper.
6767
`logs` | Logs created by the installation.
6868

6969
> **Tip**

site2/docs/reference-configuration.md

+3-1
Original file line numberDiff line numberDiff line change
@@ -672,6 +672,8 @@ You can set the log level and configuration in the [log4j2.yaml](https://github
672672
|haProxyProtocolEnabled | Enable or disable the [HAProxy](http://www.haproxy.org/) protocol. |false|
673673
|bookieId | If you want to custom a bookie ID or use a dynamic network address for a bookie, you can set the `bookieId`. <br><br>Bookie advertises itself using the `bookieId` rather than the `BookieSocketAddress` (`hostname:port` or `IP:port`).<br><br> The `bookieId` is a non-empty string that can contain ASCII digits and letters ([a-zA-Z9-0]), colons, dashes, and dots. <br><br>For more information about `bookieId`, see [here](http://bookkeeper.apache.org/bps/BP-41-bookieid/).|/|
674674
| maxTopicsPerNamespace | The maximum number of persistent topics that can be created in the namespace. When the number of topics reaches this threshold, the broker rejects the request of creating a new topic, including the auto-created topics by the producer or consumer, until the number of connected consumers decreases. The default value 0 disables the check. | 0 |
675+
| metadataStoreConfigPath | The configuration file path of the local metadata store. Standalone Pulsar uses [RocksDB](http://rocksdb.org/) as the local metadata store. The format is `/xxx/xx/rocksdb.ini`. |N/A|
676+
675677

676678
## WebSocket
677679

@@ -787,4 +789,4 @@ server.2=zk2.us-west.example.com:2888:3888
787789
server.3=zk3.us-west.example.com:2888:3888
788790
```
789791

790-
> We strongly recommend consulting the [ZooKeeper Administrator's Guide](https://zookeeper.apache.org/doc/current/zookeeperAdmin.html) for a more thorough and comprehensive introduction to ZooKeeper configuration
792+
> We strongly recommend consulting the [ZooKeeper Administrator's Guide](https://zookeeper.apache.org/doc/current/zookeeperAdmin.html) for a more thorough and comprehensive introduction to ZooKeeper configuration

0 commit comments

Comments
 (0)