[pulsar-broker] Introduce tenant based bookie isolation #3933
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
rerun java8 tests |
|
rerun java8 tests |
1 similar comment
|
rerun java8 tests |
|
rerun java8 tests |
|
rerun cpp tests |
rdhabalia
force-pushed
the
bk_iso
branch
2 times, most recently
from
2e274eb to
04b68e9
Compare
|
@merlimat can you please review this as we want to use it for our bookie isolation usecase. |
rdhabalia
force-pushed
the
bk_iso
branch
2 times, most recently
from
39f7afa to
79f56db
Compare
|
rerun integration tests |
|
@merlimat can you please review this PR, we want to use it for 2.4 release. |
fix tests add admin cli fix test
|
rerun integration tests |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Right now, broker has capability to set bookie isolation at broker level. Using which we can make broker to write into specific set of bookies by creating isolation group.
However, broker doesn't have capability to do bookie isolation based on tenant and We need tenant based bookie isolation because of multiple reasons : eg: performance and dedicated resource allocation.
Modification
Right now, user can setup rack-aware policy-group info for available bookies and user can also setup isolation group and take leverage of ZkIsolatedBookieEnsemblePlacementPolicy.
With this change:
broker can manage multiple
EnsemblePlacementPolicybased on managed-ledger configurationuser can setup
bookie-isolation groupat namespace level../pulsar-admin namespaces set-bookie-affinity-group <namespace> --group isolated-groupif bookie-isolation group is defined for a namespace then broker will use
ZkIsolatedBookieEnsemblePlacementPolicywith givenbookie-isolation groupand all writes for this namespace's topics will be written to isolated bookiesResult