Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improve][cli]Pulsar Dynamic Configuration Feature for Broker-level Configuration #20052

Closed
wants to merge 4 commits into from

Conversation

gaozhangmin
Copy link
Contributor

@gaozhangmin gaozhangmin commented Apr 10, 2023

Motivation

Currently, Pulsar's Dynamic Configuration Feature only supports cluster-level configuration, meaning that any changes made to the dynamic configuration apply to all brokers in the cluster.

This pull request proposes a new feature that allows for broker-level configuration, meaning that changes made to the dynamic configuration will only affect specific machines in the cluster, rather than all of them.

Modifications

Add new parameters scope to admin commands update-dynamic-config, delete-dynamic-config, get-all-dynamic-config , scope value is cluster or broker's LookupServiceAddres.

Verifying this change

  1. org.apache.pulsar.admin.cli.PulsarAdminToolTest#brokers
  2. org.apache.pulsar.broker.admin.AdminApiTest#testUpdateDynamicLoadBalancerSheddingIntervalMinutes

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository: gaozhangmin#12

@gaozhangmin gaozhangmin requested review from coderzc, mattisonchao and Technoboy- and removed request for coderzc and mattisonchao April 10, 2023 07:19
@gaozhangmin gaozhangmin self-assigned this Apr 10, 2023
@github-actions
Copy link

@gaozhangmin Please add the following content to your PR description and select a checkbox:

- [ ] `doc` <!-- Your PR contains doc changes -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->

@github-actions github-actions bot added doc-required Your PR changes impact docs and you will update later. and removed doc-label-missing labels Apr 10, 2023
@gaozhangmin gaozhangmin changed the title Pulsar Dynamic Configuration Feature for Broker-level Configuration [feature][admin]Pulsar Dynamic Configuration Feature for Broker-level Configuration Apr 10, 2023
@gaozhangmin gaozhangmin changed the title [feature][admin]Pulsar Dynamic Configuration Feature for Broker-level Configuration [feature][cli]Pulsar Dynamic Configuration Feature for Broker-level Configuration Apr 10, 2023
@gaozhangmin gaozhangmin changed the title [feature][cli]Pulsar Dynamic Configuration Feature for Broker-level Configuration [improve][cli]Pulsar Dynamic Configuration Feature for Broker-level Configuration Apr 10, 2023
@gaozhangmin gaozhangmin force-pushed the dynamic-configuration branch from 5e0f321 to 57988a3 Compare April 21, 2023 04:23
@gaozhangmin gaozhangmin requested a review from tisonkun April 21, 2023 04:46
@codecov-commenter
Copy link

Codecov Report

Merging #20052 (57988a3) into master (00d09cb) will increase coverage by 0.70%.
The diff coverage is 91.30%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #20052      +/-   ##
============================================
+ Coverage     72.25%   72.95%   +0.70%     
- Complexity    31946    31959      +13     
============================================
  Files          1855     1868      +13     
  Lines        138228   138486     +258     
  Branches      15224    15241      +17     
============================================
+ Hits          99871   101034    +1163     
+ Misses        30397    29404     -993     
- Partials       7960     8048      +88     
Flag Coverage Δ
inttests 24.18% <7.97%> (?)
systests 24.75% <7.97%> (?)
unittests 72.24% <91.30%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
.../apache/pulsar/broker/resources/BaseResources.java 67.41% <ø> (ø)
...rg/apache/pulsar/broker/delayed/bucket/Bucket.java 88.70% <ø> (ø)
...ommon/protocol/schema/IsCompatibilityResponse.java 83.33% <ø> (ø)
...r/delayed/bucket/BucketDelayedDeliveryTracker.java 83.95% <75.00%> (-0.26%) ⬇️
...broker/intercept/ManagedLedgerInterceptorImpl.java 75.00% <75.00%> (ø)
...g/apache/pulsar/broker/admin/impl/BrokersBase.java 71.19% <83.33%> (+2.10%) ⬆️
...ed/bucket/TripleLongPriorityDelayedIndexQueue.java 81.25% <87.50%> (-18.75%) ⬇️
...org/apache/pulsar/broker/ServiceConfiguration.java 99.37% <100.00%> (+0.31%) ⬆️
...roker/resources/DynamicConfigurationResources.java 100.00% <100.00%> (ø)
...elayed/bucket/BookkeeperBucketSnapshotStorage.java 85.71% <100.00%> (+2.07%) ⬆️
... and 8 more

... and 186 files with indirect coverage changes

@github-actions
Copy link

The pr had no activity for 30 days, mark with Stale label.

@github-actions github-actions bot added the Stale label May 22, 2023
@@ -69,29 +69,38 @@ private class UpdateConfigurationCmd extends CliCommand {
@Parameter(names = {"-v", "--value"}, description = "service-configuration value", required = true)
private String configValue;

@Parameter(names = {"-s", "--scope"}, description = "cluster or broker address, default is cluster",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shall we use an enum Scope instead of String here? Ditto other occurance.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Scope is either cluster or broker address。 unable to enumerate

@tisonkun
Copy link
Member

cc @Technoboy- @AnonHxy maybe you can give a review also?

@github-actions github-actions bot removed the Stale label May 23, 2023
@github-actions
Copy link

The pr had no activity for 30 days, mark with Stale label.

@github-actions github-actions bot added the Stale label Jun 22, 2023
@gaozhangmin gaozhangmin force-pushed the dynamic-configuration branch from 57988a3 to 7f3e408 Compare June 25, 2023 08:41
@gaozhangmin gaozhangmin requested a review from tisonkun June 27, 2023 07:14
@Technoboy- Technoboy- added this to the 3.2.0 milestone Jul 31, 2023
@Technoboy- Technoboy- modified the milestones: 3.2.0, 3.3.0 Dec 22, 2023
@coderzc coderzc modified the milestones: 3.3.0, 3.4.0 May 8, 2024
@tisonkun
Copy link
Member

tisonkun commented May 9, 2024

Closed as stale and conflict.

@tisonkun tisonkun closed this May 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
doc-required Your PR changes impact docs and you will update later. ready-to-test Stale
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants