Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed locking of PendingAddOp #3806

Merged
merged 3 commits into from
Feb 26, 2023
Merged

Fixed locking of PendingAddOp #3806

merged 3 commits into from
Feb 26, 2023

Conversation

merlimat
Copy link
Contributor

Motivation

Fix #3805

The test was failing because the updates to the object, done in the initiate() method, were not immediately visible to other threads. We must use synchronized consistently in all the accesses.

@merlimat merlimat added this to the 4.16.0 milestone Feb 25, 2023
@merlimat merlimat self-assigned this Feb 25, 2023
@codecov-commenter
Copy link

codecov-commenter commented Feb 25, 2023

Codecov Report

Merging #3806 (4d3b7cb) into master (80d3aac) will increase coverage by 7.83%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##             master    #3806      +/-   ##
============================================
+ Coverage     60.47%   68.31%   +7.83%     
- Complexity     5847     6744     +897     
============================================
  Files           473      473              
  Lines         40933    40934       +1     
  Branches       5235     5235              
============================================
+ Hits          24756    27965    +3209     
+ Misses        13976    10712    -3264     
- Partials       2201     2257      +56     
Flag Coverage Δ
bookie 39.90% <0.00%> (-0.03%) ⬇️
client 44.28% <100.00%> (?)
remaining 29.66% <0.00%> (-0.02%) ⬇️
replication 41.45% <0.00%> (+0.11%) ⬆️
tls 20.98% <0.00%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...ava/org/apache/bookkeeper/client/PendingAddOp.java 87.86% <ø> (+23.78%) ⬆️
...ava/org/apache/bookkeeper/client/LedgerHandle.java 79.80% <100.00%> (+26.57%) ⬆️
...va/org/apache/bookkeeper/bookie/SkipListArena.java 73.01% <0.00%> (-7.94%) ⬇️
...er/util/collections/ConcurrentLongLongHashMap.java 78.71% <0.00%> (-3.72%) ⬇️
...eeper/bookie/storage/directentrylogger/Buffer.java 67.41% <0.00%> (-3.38%) ⬇️
.../java/org/apache/bookkeeper/bookie/SyncThread.java 69.69% <0.00%> (-3.04%) ⬇️
...ache/bookkeeper/meta/NullMetadataBookieDriver.java 14.28% <0.00%> (-2.20%) ⬇️
...pache/bookkeeper/replication/AutoRecoveryMain.java 41.66% <0.00%> (-1.29%) ⬇️
...g/apache/bookkeeper/bookie/DefaultEntryLogger.java 79.93% <0.00%> (-1.20%) ⬇️
.../main/java/org/apache/bookkeeper/util/ZkUtils.java 77.31% <0.00%> (-1.04%) ⬇️
... and 128 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Contributor

@eolivelli eolivelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Usually these methods are always called on the same thread.

Adding synchronized under this assumptium shouldn't add much cost (no actual contention) so I am fine with the patch.

I wonder if you found out which is the activity who accesses this object outside of the assigned ML thread

Copy link
Contributor

@eolivelli eolivelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Forget my last comment

LGTM

@@ -358,7 +358,7 @@ public synchronized void writeComplete(int rc, long ledgerId, long entryId, Book
} else {
LOG.warn("Failed to write entry ({}, {}) to bookie ({}, {}): {}",
ledgerId, entryId, bookieIndex, addr, BKException.getMessage(rc));
lh.handleBookieFailure(ImmutableMap.of(bookieIndex, addr));
lh.executeOrdered(() -> lh.handleBookieFailure(ImmutableMap.of(bookieIndex, addr)));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We shouldn't change the thread here, it will delay the handleBookieFailure process. Please check the failed test LedgerClose2Test#testMetadataChangedDuringClose

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha!

@merlimat merlimat merged commit 8d92263 into apache:master Feb 26, 2023
@merlimat merlimat deleted the fix-locking branch February 26, 2023 19:10
@hangc0276
Copy link
Contributor

Good job!

Ghatage pushed a commit to sijie/bookkeeper that referenced this pull request Jul 12, 2024
* Fixed locking of PendingAddOp

* Fixed stack overflow

* Fixed stack overflow in different point
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

LedgerCloseTest#testLedgerCloseWithConsistentLength failed with IllegalReferenceCountException
4 participants