Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release monitor write lock in between update iterations #2528

Conversation

arik-so
Copy link
Contributor

@arik-so arik-so commented Aug 26, 2023

Fixes #2470.

@codecov-commenter
Copy link

codecov-commenter commented Aug 26, 2023

Codecov Report

Patch coverage: 95.34% and project coverage change: +0.24% 🎉

Comparison is base (3dffe54) 90.56% compared to head (b6e7486) 90.81%.
Report is 5 commits behind head on main.

❗ Current head b6e7486 differs from pull request most recent head f80284c. Consider uploading reports for the commit f80284c to get more accurate results

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2528      +/-   ##
==========================================
+ Coverage   90.56%   90.81%   +0.24%     
==========================================
  Files         109      109              
  Lines       57304    59861    +2557     
  Branches    57304    59861    +2557     
==========================================
+ Hits        51899    54362    +2463     
- Misses       5405     5499      +94     
Files Changed Coverage Δ
lightning/src/ln/monitor_tests.rs 98.45% <85.71%> (-0.08%) ⬇️
lightning/src/chain/chainmonitor.rs 94.91% <97.22%> (-0.12%) ⬇️

... and 7 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@arik-so arik-so force-pushed the arik/2023-08-2470-shorter-term-monitor-locks branch from e9b3355 to 6237706 Compare August 26, 2023 07:45
@arik-so arik-so marked this pull request as ready for review August 26, 2023 08:42
Copy link
Collaborator

@TheBlueMatt TheBlueMatt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks

lightning/src/chain/chainmonitor.rs Outdated Show resolved Hide resolved
lightning/src/chain/chainmonitor.rs Outdated Show resolved Hide resolved
lightning/src/chain/chainmonitor.rs Outdated Show resolved Hide resolved
lightning/src/ln/monitor_tests.rs Outdated Show resolved Hide resolved
@arik-so arik-so force-pushed the arik/2023-08-2470-shorter-term-monitor-locks branch from b6e7486 to bb028a3 Compare August 27, 2023 01:50
TheBlueMatt
TheBlueMatt previously approved these changes Aug 27, 2023
lightning/src/chain/chainmonitor.rs Outdated Show resolved Hide resolved
Previously, updating block data on a chain monitor
would acquire a write lock on all of its associated
channel monitors and not release it until the loop
completed.

Now, we instead acquire it on each iteration,
fixing lightningdevkit#2470.
Releasing write locks in between monitor updates
requires storing a set of cloned keys to iterate
over. For efficiency purposes, that set of keys
is an actual set, as opposed to array, which means
that the iteration order may not be consistent.

The test was relying on an event array index to
access the revocation transaction. We change that
to accessing a hash map keyed by the txid, fixing
the test.
@arik-so arik-so force-pushed the arik/2023-08-2470-shorter-term-monitor-locks branch from bb028a3 to f80284c Compare August 27, 2023 17:24
@TheBlueMatt TheBlueMatt added this to the 0.0.117 milestone Aug 28, 2023
@TheBlueMatt TheBlueMatt merged commit 2c4f824 into lightningdevkit:main Aug 28, 2023
@@ -285,7 +286,22 @@ where C::Target: chain::Filter,
where
FN: Fn(&ChannelMonitor<ChannelSigner>, &TransactionData) -> Vec<TransactionOutputs>
{
let funding_outpoints: HashSet<OutPoint> = HashSet::from_iter(self.monitors.read().unwrap().keys().cloned());
for funding_outpoint in funding_outpoints.iter() {
let monitor_lock = self.monitors.read().unwrap();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: rename monitor_lock -> monitors

let funding_outpoints: HashSet<OutPoint> = HashSet::from_iter(self.monitors.read().unwrap().keys().cloned());
for funding_outpoint in funding_outpoints.iter() {
let monitor_lock = self.monitors.read().unwrap();
if let Some(monitor_state) = monitor_lock.get(funding_outpoint) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: rename monitor_state -> monitor

}
}

// do some followup cleanup if any funding outpoints were added in between iterations
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"cleanup" doesn't sound right in this case.
Re-iterate if any funding outpoints were added in between write locked iterations above.

@G8XSU
Copy link
Contributor

G8XSU commented Aug 28, 2023

There are some hidden implications,
Noting for record keeping and review:

  • Not all monitors will be at same chain-state (some monitors could have updated block processed and some waiting on write lock)
  • Monitors can be ahead of chain_monitor.highest_chain_height
  • SpendableOutputs from various monitors can be in in arbitrary order. (there was an implicit fixed ordering earlier though not any specific order)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update monitors for new chain data without blocking the world
5 participants