[improve][broker] Exclude system topics from namespace level publish and dispatch rate limiting

[poorbarcode]

Motivation

The effect is huge if the pub&sub rate limiter takes effect for system topics, such as __change_events, transaction topics... Almost all topics can not work if the system topic encounters a throttling

Modifications

• Removes the broker-level/namespace-level/resource-group-level pub&sub throttling for system topics to avoid the issue of almost all topics not working ( load up, delete and other actions). And left the topic-level policies there( it will work if needed )

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

Matching PR in forked repository

PR in forked repository: x

[lhotari]

The only possible concern in a multi-tenant system is that a malicious user could circumvent rate-limits by using a topic name that gets considered as a system topic. I guess this is a matter of the definition of the security model in the multi-tenancy support that rate limits cannot be used to protect against malicious users. We don't currently document the security model so it's hard to resolve this as part of this PR. I guess we could create a separate issue for documenting the security model for multi-tenancy.

[lhotari]

LGTM

[gaoran10]

LGTM

[poorbarcode]

/pulsarbot rerun-failure-checks

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 80.00000% with 2 lines in your changes missing coverage. Please review.

Project coverage is 74.31%. Comparing base (bbc6224) to head (4564f67).
Report is 728 commits behind head on master.

Files with missing lines	Patch %	Lines
...sar/broker/service/DisabledPublishRateLimiter.java	50.00%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #23589      +/-   ##
============================================
+ Coverage     73.57%   74.31%   +0.73%     
- Complexity    32624    34436    +1812     
============================================
  Files          1877     1944      +67     
  Lines        139502   147080    +7578     
  Branches      15299    16216     +917     
============================================
+ Hits         102638   109299    +6661     
- Misses        28908    29354     +446     
- Partials       7956     8427     +471     

Flag	Coverage Δ
inttests	27.31% <80.00%> (+2.73%)	⬆️
systests	24.35% <80.00%> (+0.03%)	⬆️
unittests	73.70% <80.00%> (+0.86%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...rg/apache/pulsar/broker/service/AbstractTopic.java	87.93% <100.00%> (-0.06%)	⬇️
.../pulsar/broker/service/persistent/SystemTopic.java	84.21% <100.00%> (+4.21%)	⬆️
...sar/broker/service/DisabledPublishRateLimiter.java	50.00% <50.00%> (ø)

... and 653 files with indirect coverage changes

[poorbarcode]

The only possible concern in a multi-tenant system is that a malicious user could circumvent rate limits by using a topic name that gets considered as a system topic. I guess this is a matter of the definition of the security model in the multi-tenancy support that rate limits cannot be used to protect against malicious users. We don't currently document the security model so it's hard to resolve this as part of this PR. I guess we could create a separate issue for documenting the security model for multi-tenancy.

Seems we never focused on the isolation for multi-tenants before, that should be a huge PIP, which contains Transaction(all namespaces use the same Transaction metadata store) and other components. 😂

Thanks for mentioning this, let me merge the PR first