Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improve][broker]improve the getAntiAffinityNamespaceOwnedBrokers check exclude the current namespace cause count add. #5

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Nicklee007
Copy link
Owner

Motivation

As the PIP 7: Pulsar Failure domain and Anti affinity namespaces design, the anti affinity namespaces should be distributed to evenly across all domain and all the brokers.
But in method getAntiAffinityNamespaceOwnedBrokers , brokerToAntiAffinityNamespaceCount add a count even the namespace equal the given bundle's namespace which will be load. The behavior will cause the namespace easy to distributed to
a broker which has another namespace in anti affinity group, the behavior broke the anti affinity balance. It's better behavior is the same namespace should be distributed to those broker which has loaded the same namespace when all broker have load at least one namespace in ti affinity group.

there is some case
ns-0 ns-1 ns-2 are all set the same anti affinity group like 'a-group

broker-0 own ns-0 bundle-0;
broker-1 own ns-1 bundle-0;
broker-2 own ns-2 bundle-0;
then another ns-2 bundle-1 need choice a broker to load. As the old policy, broker-0 broker-1 and broker-2 are all satisfy the least NamespaceCount; but if ns-2 bundle-1 load to broker-0 or broker-1 will broke the anti affinity balance. ns-2 bundle-1 need be load by broker-2 is better.

Also, the behavior will cause give up doLoadShedding when the all broker own one namespace in anti affinity group, but the different broker owned namespace bundle count and payload is different.
So brokerToAntiAffinityNamespaceCount should exclude the given namespace count add.

Modifications

  1. In LoadManagerShared.class getAntiAffinityNamespaceOwnedBrokers method, exclude the given namespace count add.
  2. changed shouldAntiAffinityNamespaceUnload check;
  3. add some unit test.

Documentation

  • doc-not-needed

@github-actions
Copy link

github-actions bot commented Jan 8, 2023

The pr had no activity for 30 days, mark with Stale label.

@github-actions github-actions bot added the Stale label Jan 8, 2023
@Nicklee007 Nicklee007 force-pushed the improve-anti-affinity-owned-brokers-check branch from 83e5536 to 118b16b Compare January 10, 2023 03:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant