ISSUE-13756: [broker] Optimize topic policy with HierarchyTopicPolicies

[sijie]

Original Issue: apache#13756

---

This issue is created to track the process of the serial topic policy optimization with HierarchyTopicPolicies.

Motivation

Update topic policy with HierarchyTopicPolicies comes with these benefits:

• All topic policy related settings will goes into AbstractTopic#topicPolicies, easier to understand, check, review, and modify.
• Unify policy update to AbstractTopic. And easier to find which policy is not applied to non-persistent topic yet. And we can easily add support for it.
• Unify policy value with 3 level settings (topic/namespace/broker), and priority with topic > namespace > broker. And it's easier to find which level settings is missing.
• All values are updated at creation or by trigger. We can save some resource to update it or recalculate each time we use it.
• etc.

Progress

• backLogQuotaMap [Issue 12726][broker] Fix deadlock in metadata-store callback thread apache/pulsar#12753
• subscriptionTypesEnabled [broker] Optimize TopicPolicies#subscriptionTypesEnabled with HierarchyTopicPolicies apache/pulsar#13121
• replicationClusters Optimize TopicPolicies#replicationClusters with HierarchyTopicPolicies apache/pulsar#13495
• deduplicationEnabled [broker] Optimize TopicPolicy#deduplicationEnabled with HierarchyTopicPolicies apache/pulsar#13000
• messageTTLInSeconds [broker] Optimize TopicPolicies#messageTTLInSeconds with HierarchyTopicPolicies apache/pulsar#13241
• maxProducerPerTopic [broker] Optimize TopicPolicy#maxProducersPerTopic with HierarchyTopicPolicies apache/pulsar#13082
• maxConsumerPerTopic [Broker] Optimize TopicPolicies#maxConsumerPerTopic with HierarchyTopicPolicies  apache/pulsar#13361
• maxConsumersPerSubscription Optimize TopicPolicies#maxConsumersPerSubscription with HierarchyTopicPolicies apache/pulsar#13548
• delayedDeliveryTickTimeMillis Optimize TopicPolicies#delayedDelivery Enabled and TickTimeMillis with HierarchyTopicPolicies apache/pulsar#13649
• delayedDeliveryEnabled Optimize TopicPolicies#delayedDelivery Enabled and TickTimeMillis with HierarchyTopicPolicies apache/pulsar#13649
• inactiveTopicPolicies [Broker] Apply PolicyHierarchyValue to inactiveTopicPolicies apache/pulsar#12687
• compactionThreshold [broker] Optimize TopicPolicies#compactionThreshold with HierarchyTopicPolicies apache/pulsar#13710
• maxMessageSize [broker]Optimize topicMaxMessageSize with topic local cache. apache/pulsar#12830
• maxSubscriptionsPerTopic [broker] Avoid unnecessary recalculation of maxSubscriptionsPerTopic in AbstractTopic apache/pulsar#12658
• maxUnackedMessagesOnConsumer [broker] Optimize TopicPolicies#maxUnackedMessagesOnConsumer with HierarchyTopicPolicies apache/pulsar#13618
• maxUnackedMessagesOnSubscription @Jason918 [broker] optimize maxUnackedMessagesOnSubscription with HierarchyTopicPolicies apache/pulsar#14011
• retentionPolicies @Jason918 [broker] Optimize retentionPolicies with HierarchyTopicPolicies apache/pulsar#13909
• deduplicationSnapshotIntervalSeconds @Jason918 [broker] Optimize deduplicationSnapshotIntervalSeconds with HierarchyTopicPolicies apache/pulsar#13769
• dispatchRate @AnonHxy [Issue 13756][Broker] Optimize topic policy with HierarchyTopicPolicies about dispatchRate apache/pulsar#14038
• subscriptionDispatchRate @AnonHxy [Issue 13756][Broker]Optimize topic policy with HierarchyTopicPolicies about subscriptionDispatchRate apache/pulsar#14151
• publishRate @AnonHxy
• subscribeRate @AnonHxy
• replicatorDispatchRate @AnonHxy [Issue 13756][Broker]Optimize topic policy with HierarchyTopicPolicies about replicatorDispatchRate apache/pulsar#14161

These topic policies can not apply to this optimization because it's used for ML and before Topic creation.

• offloadPolicies
• persistence