-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[fix] [broker] correct compaction phase one timeout meaning #21908
Conversation
pulsar-broker/src/main/java/org/apache/pulsar/compaction/TwoPhaseCompactor.java
Outdated
Show resolved
Hide resolved
pulsar-broker/src/main/java/org/apache/pulsar/compaction/TwoPhaseCompactor.java
Outdated
Show resolved
Hide resolved
I think I have introduced a bad configuration name in #11206. Actually, it was the timeout for an entry read. IMO, we should have a new configuration name and deprecate the existing one. I don't think it's reasonable to have a timeout for the whole phase one loop. After the first time the phase one loop get timeout, the subsequent phase one loop will almost always get timeout with more published messages unless changing phase one loop timeout. |
865377e
to
2f1f8de
Compare
I agree. I have introduced a new configuration name and deprecate the existing one. |
I have created a pip to change the conf name, please help review it, thanks! |
2f1f8de
to
b1cebb1
Compare
I have refactor the implementation to just modify the doc, so that we can avoid to create a pip and introduce a new conf name. PTAL, thanks. |
/pulsarbot rerun-failure-checks |
1 similar comment
/pulsarbot rerun-failure-checks |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #21908 +/- ##
=============================================
+ Coverage 36.46% 73.58% +37.11%
- Complexity 12381 32402 +20021
=============================================
Files 1725 1861 +136
Lines 131697 138589 +6892
Branches 14400 15184 +784
=============================================
+ Hits 48019 101976 +53957
+ Misses 77278 28720 -48558
- Partials 6400 7893 +1493
Flags with carried forward coverage won't be shown. Click here to find out more.
|
Motivation
Configuratioin
brokerServiceCompactionPhaseOneLoopTimeInSeconds
is defined as the total timeout of phase one.But current implementation just set a timeout for a single message read operation, which is different with the meaning of
brokerServiceCompactionPhaseOneLoopTimeInSeconds
and will result into thousands of timer tasks be scheduled and cancelled in the scheduler because we may iterate many messages.Modifications
Set a timer task for the tatol phase one.
Verifying this change
(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
Does this pull request potentially affect one of the following parts:
If the box was checked, please highlight the changes
Documentation
doc
doc-required
doc-not-needed
doc-complete
Matching PR in forked repository
PR in forked repository: thetumbled#32