PIP-245: Make subscriptions of NonPersistentTopic non-durable

[tjiuming]

Motivation

There are two types of subscriptions for a topic: Durable and Non-durable.

We create a Consumer with a Durable subscription and a Reader with a Non-durable subscription.

But for NonPersistentTopic, creating a Durable subscription is meaningless, NonPersistentSubscription doesn't have a ManagedCursor to persistent its data. After its consumer disconnected, the subscription couldn't be removed automatically if we didn't set the value of subscriptionExpirationTimeMinutes greater than 0.

For subscriptionExpirationTimeMinutes, it controls the subscription expiration of NonPersistentTopic and PersistentTopic, if we set the value of subscriptionExpirationTimeMinutes greater than 0, it may lead to data loss(The durable subscriptions of PersistentTopic also can be removed).

And the Non-durable subscriptions will be removed automatically after all the consumers disconnected, it's the existing logic.

For the purpose of removing the subscriptions which have no active consumers of NonPersistentTopic and the above reasons, we can make all the subscriptions of a NonPersistentTopic Non-durable.

Goal

Make the subscriptions of NonPersistentTopic Non-durable.

API Changes

Implementation

1. Client-side:

Add the following logic to the constructor of ConsumerImpl before this.subscriptionMode=conf.getSubscriptionMode():

        TopicName topicName = TopicName.get(topic);
        if (!topicName.isPersistent() && conf.getSubscriptionMode().equals(SubscriptionMode.Durable)) {
            conf.setSubscriptionMode(SubscriptionMode.NonDurable);
            log.warn("[{}] Cannot create a [Durable] subscription for a NonPersistentTopic, " +
                    "will use [NonDurable] to subscribe. Subscription name: {}", topic, conf.getSubscriptionName());
        }

Check the topic type, if the topic is a NonPersistentTopic, change the value of subscriptionMode to NonDurable and print WARN log, Users don't need to change their code.

2. Broker-side:

Change the constructor of NonPersistentTopic from

   public NonPersistentSubscription(NonPersistentTopic topic, String subscriptionName, boolean isDurable, 
   Map<String, String> properties) {
       this.topic = topic;
       this.topicName = topic.getName();
       this.subName = subscriptionName;
       this.fullName = MoreObjects.toStringHelper(this).add("topic", topicName).add("name", subName).toString();
       IS_FENCED_UPDATER.set(this, FALSE);
       this.lastActive = System.currentTimeMillis();
       this.isDurable = isDurable;
       this.subscriptionProperties = properties != null
               ? Collections.unmodifiableMap(properties) : Collections.emptyMap();
   }

to

   public NonPersistentSubscription(NonPersistentTopic topic, String subscriptionName,
   Map<String, String> properties) {
       this.topic = topic;
       this.topicName = topic.getName();
       this.subName = subscriptionName;
       this.fullName = MoreObjects.toStringHelper(this).add("topic", topicName).add("name", subName).toString();
       IS_FENCED_UPDATER.set(this, FALSE);
       this.lastActive = System.currentTimeMillis();
       this.isDurable = false;
       this.subscriptionProperties = properties != null
               ? Collections.unmodifiableMap(properties) : Collections.emptyMap();
   }

Remove the constructor argument isDurable, and set its value to false.

Compatibility

The PIP will deprecate Durable subscriptions on NonPersistentTopic in 3.0.0, if users want to create a Durable subscription on NonPersistentTopic we will print WARN logs and change isDurable to false.

We will not be compatible with the next version after 3.0.1.

In the next release after 3.0.0, if users want to create Durable subscriptions on NonPersistentTopic, will throw an exception.

Alternatives

No response

Anything else?

No response

[tjiuming]

Discuss thread: https://lists.apache.org/thread/2ltmyglnb25jy8nk58twkwbglws43bst

[codelipenghui]

The default subscription mode is durable. So users will see the warning log if they upgrade to the new version client? Why not just change isDurable to false for the non-persistent topic subscriptions. And if users are explicitly changed to true, then see the warning log.

[tjiuming]

The default subscription mode is durable. So users will see the warning log if they upgrade to the new version client?

Yes

Why not just change isDurable to false for the non-persistent topic subscriptions. And if users are explicitly changed to true, then see the warning log.

Because PersistentTopic and NonPersistentTopic subscriptions are using ConsumerConfigurationData,
if we change isDurable to false, when create consumers on PersistentTopic, we also need to add logics to change isDurable to true.

@codelipenghui

[dao-jun]

Vote thread: https://lists.apache.org/thread/phrktmgln08pgpddvx8vvwzvgnq9kd0w

[github-actions]

The issue had no activity for 30 days, mark with Stale label.