Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix][broker] Fix Replicated Topic unload bug when ExtensibleLoadManager is enabled #22496

Merged
merged 2 commits into from
Apr 16, 2024

Conversation

heesung-sn
Copy link
Contributor

@heesung-sn heesung-sn commented Apr 12, 2024

Motivation

Noticed that Replicated Topic is not unloaded upon setNamespaceReplicationClusters when ExtensibleLoadManager is enabled.

Modifications

  • Fix unloading bundles upon namespace policy update. (isNamespaceBundleOwned didn't check the bundle ownership properly when ExtensibleLoadManager is enabled.)
  • Do not replicate ExtensibleLoadManager internal topics (bundle state, broker and bundle load data topics) because they are only used for the local cluster.
  • Copy ReplicatorGlobalNSTest for ExtensibleLoadManager.

Verifying this change

  • Make sure that the change passes the CI checks.

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository:

Copy link

@heesung-sn Please add the following content to your PR description and select a checkbox:

- [ ] `doc` <!-- Your PR contains doc changes -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->

@heesung-sn heesung-sn self-assigned this Apr 12, 2024
@github-actions github-actions bot added doc-not-needed Your PR changes do not impact docs and removed doc-label-missing labels Apr 12, 2024
@merlimat merlimat added this to the 3.3.0 milestone Apr 13, 2024
Copy link
Member

@Demogorgon314 Demogorgon314 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we also check the other place such as producer.close(), looks like it may also get stuck forever.

@heesung-sn
Copy link
Contributor Author

Can we also check the other place such as producer.close(), looks like it may also get stuck forever.

I understand the producer.flush might get stuck if no leader broker is found in the cluster, but actually, I wonder how producer.close() can get stuck.

@Demogorgon314
Copy link
Member

I understand the producer.flush might get stuck if no leader broker is found in the cluster, but actually, I wonder how producer.close() can get stuck.

Oh, I see it has operationTimeoutMs set to 30000, so it won't be stuck.

@heesung-sn heesung-sn merged commit 203f305 into apache:master Apr 16, 2024
49 of 50 checks passed
public static Object[][] loadManagerClassName() {
return new Object[][]{
{ModularLoadManagerImpl.class.getName()},
{ExtensibleLoadManagerImpl.class.getName()}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@heesung-sn I think that this is triggering a problem in tests.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. We can comment out this line if this is problematic.

heesung-sn added a commit to heesung-sn/pulsar that referenced this pull request Jun 26, 2024
heesung-sn added a commit to heesung-sn/pulsar that referenced this pull request Jun 26, 2024
heesung-sn added a commit to heesung-sn/pulsar that referenced this pull request Jun 26, 2024
…ger is enabled (apache#22496)

(cherry picked from commit 203f305)
(cherry picked from commit f467f37)
heesung-sn added a commit to heesung-sn/pulsar that referenced this pull request Jun 26, 2024
heesung-sn added a commit to heesung-sn/pulsar that referenced this pull request Jun 26, 2024
…ger is enabled (apache#22496)

(cherry picked from commit 203f305)
(cherry picked from commit f467f37)
heesung-sn added a commit to heesung-sn/pulsar that referenced this pull request Jun 27, 2024
…ger is enabled (apache#22496)

(cherry picked from commit 203f305)
(cherry picked from commit f467f37)
nikhil-ctds pushed a commit to datastax/pulsar that referenced this pull request Jun 27, 2024
…ger is enabled (apache#22496)

(cherry picked from commit 203f305)
(cherry picked from commit f467f37)
(cherry picked from commit 2557db6)
srinath-ctds pushed a commit to datastax/pulsar that referenced this pull request Jul 1, 2024
…ger is enabled (apache#22496)

(cherry picked from commit 203f305)
(cherry picked from commit f467f37)
(cherry picked from commit 2557db6)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants