[improve][broker] PIP-192 PIP-220 Added TransferShedder

[heesung-sn]

Master Issue: #18215

Motivation

This PR implement PIP-220, #18215

Modifications

For the PIP-220(a sub-pip of pip-192, #16691), this PR implements the TransferShedder and its unit test.

Also, there are minor changes in the related classes:

• Added the recentlyUnloadedBrokers input var in NamespaceUnloadStrategy.findBundlesForUnloading(...)
• Added the TransferShedder-related configs in ServiceConfiguration

Verifying this change

• Make sure that the change passes the CI checks.

This change added tests and can be verified as follows:

• *Added unit tests for the new classes.

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

• Dependencies (add or upgrade a dependency)
• The public API
• The schema
• The default values of configurations
• The threading model
• The binary protocol
• The REST endpoints
• The admin CLI options
• Anything that affects deployment

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

We will have separate PRs to update the Doc later.

Matching PR in forked repository

PR in forked repository: heesung-sn#16

[Demogorgon314]

LGTM. Only have one comment, the configuration name is so long, can we remove the prefix?

[heesung-sn]

Please hold on reviewing this PR. I will refactor this PR according to the changes in #19154.

[heesung-sn]

Please continue the review. I applied the changes from #19154.

[gaoran10]

Does this config is necessary? Maybe we can do the same thing via modifying the broker log config file.

[heesung-sn]

Some systems do support fine-grained debugging logs for certain features.

I am curious if the pulsar follows this practice, but I think this could be useful to see debug logs from load balancer logic only.

[Demogorgon314]

Should we mark this configuration as dynamic? Then we can change this configuration in real time.

[heesung-sn]

I think we can mark all these configs as dynamic. Updating.

[gaoran10]

The Pulsar broker uses the log4j2, which supports custom log configurations by modifying the config file ${PULSAR_HOME}/conf/log4j2.yaml, which supports log control at the class level.

[heesung-sn]

That's another way of fine-configuring the log level, and I assume we can control the log level by the package as well.

But with this dynamic config, we can more easily turn on/off the debugging logs too.

[gaoran10]

OK, but it's a little strange to add a separate param to control debug log information.