-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[fix][broker] Fix NPE when reset Replicator's cursor by position. #20597
[fix][broker] Fix NPE when reset Replicator's cursor by position. #20597
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch
pulsar-broker/src/test/java/org/apache/pulsar/broker/service/ReplicatorTest.java
Outdated
Show resolved
Hide resolved
pulsar-broker/src/test/java/org/apache/pulsar/broker/service/ReplicatorTest.java
Outdated
Show resolved
Hide resolved
pulsar-broker/src/test/java/org/apache/pulsar/broker/service/ReplicatorTest.java
Outdated
Show resolved
Hide resolved
2. change unit test to use admin api
5a7dfd1
to
8c9b1d3
Compare
Codecov Report
@@ Coverage Diff @@
## master #20597 +/- ##
============================================
+ Coverage 72.60% 73.11% +0.51%
+ Complexity 32018 31936 -82
============================================
Files 1855 1867 +12
Lines 138569 138764 +195
Branches 15250 15265 +15
============================================
+ Hits 100605 101458 +853
+ Misses 29945 29258 -687
- Partials 8019 8048 +29
Flags with carried forward coverage won't be shown. Click here to find out more.
|
public PersistentMessageExpiryMonitor(String topicName, String subscriptionName, ManagedCursor cursor, | ||
PersistentSubscription subscription) { | ||
this.topicName = topicName; | ||
public PersistentMessageExpiryMonitor(PersistentTopic topic, String subscriptionName, ManagedCursor cursor, | ||
@Nullable PersistentSubscription subscription) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This PR modifies the public API (even it's in the pulsar-broker
module), we should not cherry-pick it to release branches unless we make it compatible. @poorbarcode @Technoboy-
…ion. (apache#20597)" This reverts commit aa565c2.
### Motivation When I reviewed apache#20597, the unrelated changes in `PersistentTopicsBase` are hard to read. The logic could be simplified to: ```java PersistentSubscription sub = null; PersistentReplicator repl = null; if (metSomeCondition()) { repl = /* ... */; if (repl == null) { /* ... */ return; } } else { sub = /* ... */; if (repl == null) { /* ... */ return; } } final PersistentSubscription finalSub = sub; final PersistentReplicator finalRepl = repl; future.thenAccept(__ -> { if (metSomeCondition()) { repl.expireMessages(/* ... */); } else { sub.expireMessages(/* ... */); } }); ``` The code above is such a mess. It adds two final variables because the lambda can only capture final variables. The `metSomeCondition` check is performed unnecessarily twice. The original code is more hard to read because the logic in `/* ... */` takes a few lines so that the two calls of `metSomeCondition()` are not near. From the code search I see all these classes implement two `expireMessages` methods that accept an integer or a position. - PersistentMessageExpiryMonitor - PersistentSubscription - PersistentReplicator - NonPersistentSubscription The code can be simplified to introduce a new interface. ### Modifications Introduce a `MessageExpirer` interface and change the class hierarchy to: ``` // [I] is interface, [C] is class [I] MessageExpirer [I] Subscription [C] PersistentSubscription [C] NonPersistentSubscription [C] PersistentReplicator [C] PersistentMessageExpiryMonitor ``` The method invocation can be simplified much as shown in this patch. P.S. Inserting such an interface in the type hierarchy does not even break the ABI compatibility, see https://docs.oracle.com/javase/specs/jls/se7/html/jls-13.html
### Motivation When I reviewed apache#20597, the unrelated changes in `PersistentTopicsBase` are hard to read. The logic could be simplified to: ```java PersistentSubscription sub = null; PersistentReplicator repl = null; if (metSomeCondition()) { repl = /* ... */; if (repl == null) { /* ... */ return; } } else { sub = /* ... */; if (repl == null) { /* ... */ return; } } final PersistentSubscription finalSub = sub; final PersistentReplicator finalRepl = repl; future.thenAccept(__ -> { if (metSomeCondition()) { repl.expireMessages(/* ... */); } else { sub.expireMessages(/* ... */); } }); ``` The code above is such a mess. It adds two final variables because the lambda can only capture final variables. The `metSomeCondition` check is performed unnecessarily twice. The original code is more hard to read because the logic in `/* ... */` takes a few lines so that the two calls of `metSomeCondition()` are not near. From the code search I see all these classes implement two `expireMessages` methods that accept an integer or a position. - PersistentMessageExpiryMonitor - PersistentSubscription - PersistentReplicator - NonPersistentSubscription The code can be simplified to introduce a new interface. ### Modifications Introduce a `MessageExpirer` interface and change the class hierarchy to: ``` // [I] is interface, [C] is class [I] MessageExpirer [I] Subscription [C] PersistentSubscription [C] NonPersistentSubscription [C] PersistentReplicator [C] PersistentMessageExpiryMonitor ``` The method invocation can be simplified much as shown in this patch. P.S. Inserting such an interface in the type hierarchy does not even break the ABI compatibility, see https://docs.oracle.com/javase/specs/jls/se7/html/jls-13.html
…ion. (apache#20597)" This reverts commit 5abadbe.
…ache#20597) (cherry picked from commit 5abadbe) (cherry picked from commit 18f89b6)
(cherry picked from commit 7b6d1c9)
…ion. (apache#20597)" This reverts commit 18f89b6. (cherry picked from commit 209b222)
…position. (apache#20597) (apache#20781) (cherry picked from commit 643ffad)
…ion. (apache#20597)" This reverts commit aa565c2. (cherry picked from commit 35b449b)
…ion. (apache#20597)" This reverts commit aa565c2. (cherry picked from commit 35b449b)
…ion. (apache#20597)" This reverts commit aa565c2. (cherry picked from commit 35b449b)
…ion. (apache#20597)" This reverts commit aa565c2. (cherry picked from commit 35b449b)
Motivation
When init in PersistentReplicator, the field of
PersistentSubscription
is null.see :
pulsar/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentReplicator.java
Lines 119 to 120 in 37ae858
pulsar/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentMessageExpiryMonitor.java
Lines 60 to 62 in 37ae858
so an NPE throws when executing
expireMessages(Position messagePosition)
:pulsar/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentMessageExpiryMonitor.java
Lines 99 to 108 in 37ae858
Modifications
save PersistentTopic in
PersistentMessageExpiryMonitor
field.ensure when call
expireMessages(Position messagePosition)
only check topic LastPosition by topic. not null subscription.Verifying this change
add new test to verify the method wont throw npe. but not the logic.
Does this pull request potentially affect one of the following parts:
If the box was checked, please highlight the changes
Documentation
doc
doc-required
doc-not-needed
doc-complete
Matching PR in forked repository
PR in forked repository: