Improve on (potentially) blocking wallet behaviors #281

tnull · 2024-04-12T12:58:13Z

~~Based on #141.~~

Due to our onchain wallet requiring us to hold a Mutex for the duration of the chain sync, many operations might be blocked for seconds at at time.

Here, we make several changes that hopefully alleviate any blocking issues from holding the wallet lock(s). In particular we introduce a balance cache, drop our 'immediate retry sync' logic and drop the previous wallet_lock Condvar in favor of a pub/sub pattern that is beneficial in async runtime environments.

tnull · 2024-06-01T07:57:23Z

Rebased on #141 to resolve minor conflicts with main.

tnull · 2024-06-11T18:58:03Z

Rebased on main after #141 landed.

tnull · 2024-06-14T07:31:23Z

Rebased after #307 landed.

jbesraa

I tested this branch locally with an app with GUI that previously lagged when the wallet synced and I can confirm that with this PR things are much smoother

tnull · 2024-06-14T08:03:19Z

I tested this branch locally with an app with GUI that previously lagged when the wallet synced and I can confirm that with this PR things are much smoother

Yeah, without these workarounds the post-Anchors code is borderline unusable as event handling might grind to a standstill whenever we need to check the balance and wallet syncing is ongoing.

tnull · 2024-06-17T07:36:40Z

Rebased to resolve minor conflict after #303 landed.

tnull · 2024-06-18T09:31:29Z

Now also added a commit dropping the immediate-retry behavior in TxBroadcaster, and applying the timeout to tx_sync.sync() in sync_wallets

G8XSU · 2024-06-18T18:55:59Z

src/wallet.rs

+				*self.balance_cache.write().unwrap() = balance.clone();
+				balance
+			},
+			Err(_) => self.balance_cache.read().unwrap().clone(),


can you expand on some of the implications of using a cached balance ?

iiuc, we might bypass some of the balance checks during open-channel (outbound and inbound) and send_to_address.(does bdk allow building tx greater than balance?)

can you expand on some of the implications of using a cached balance ?

It essentially has no implications: generally the wallet balance would always just be updated when a sync succeeds and here we make sure to always update the cache afterwards, while still holding the wallet Mutex. So the cache should always be up to date. Plus, we'll actually call through and update the cache whenever we can.

iiuc, we might bypass some of the balance checks during open-channel (outbound and inbound) and send_to_address.(does bdk allow building tx greater than balance?)

Yes, but IIUC BDK's wallet would always only pick up on any changes after a successful sync, i.e., it would show the old balance anyways.

Btw, see relatedly: #41

G8XSU · 2024-06-18T18:56:19Z

src/wallet.rs

@@ -175,13 +199,24 @@ where
 	pub(crate) fn get_balances(


might be worth to add doc that result might be cached.

I honestly don't think it matters: a) this is pub(crate), and b) it shouldn't result in any noticeable behavior changes, as discussed above.

G8XSU · 2024-06-18T19:24:38Z

src/wallet.rs

 			Err(e) => match e {
 				bdk::Error::Esplora(ref be) => match **be {
 					bdk::blockchain::esplora::EsploraError::Reqwest(_) => {
+						// Drop lock, sleep for a second, retry.
+						drop(wallet_lock);


nit: we this and retry here, only to remove it in later commit, can be simplified if we just drop retry before this.

Fair enough. Are you fine with me leaving it like this or would you prefer to invert the commit ordering?

src/wallet.rs

tnull · 2024-06-20T16:40:26Z

Kicked CI.

@G8XSU Let me know if I can squash the fixups.

G8XSU · 2024-06-20T17:08:57Z

Lgtm!
feel free to squash fixups.

Unfortunately BDK's current wallet design requires us to have it live in `Mutex` that is locked for long periods of time during syncing. This is especially painful for short-lived operations that just operate locally, such as retrieving the current balance, which we now do in several places to be able to check Anchor channels limitations, e.g., in event handling. In order to avoid blocking during balance retrieval, we introduce a `balance` cache that will be refreshed whenever we're done with syncing *or* when we can successfully get the wallet lock. Otherwise, we'll just return the cached value, allowing us to make progress even though a background sync of the wallet might be in-progress.

Using a `Condvar` could be potentially dangerous in async contexts as `wait`ing on it might block the current thread potentially hosting more than one task. Here, we drop the `Condvar` and adopt a pub/sub scheme instead, similar to the one we already implemented in `ConnectionManager`.

It's not super clear that it achieves much in the face of a rate-limited Esplora server, and having a custom sleep there is just awkward. So we drop it and hope we still get a chance to sync our on-chain wallet now and then.

.. as we're not sure it actually increases reliability. We now only log failures, ignoring HTTP 400 as this is bitcoind's error code for "transaction already in mempool".

.. to make progress and unblock the `Mutex` even if BDK's wallet `sync` would never return.

.. even though we don't expect this to block, we're better safe than sorry and start to introduce timeouts for any calls we make to remote servers.

.. before initiating the Runtime shutdown.

.. as we use `Clone` for `tokio::sync::watch::Sender`, which was only introduced with 1.37.

tnull · 2024-06-20T17:10:36Z

Lgtm! feel free to squash fixups.

Squashed without further changes.

tnull force-pushed the 2024-04-improve-on-wallet-friction branch from 3d70262 to c0f13bc Compare April 12, 2024 13:00

tnull changed the title ~~Improve on wallet (potentially) blocking wallet behaviors~~ Improve on (potentially) blocking wallet behaviors Apr 12, 2024

tnull force-pushed the 2024-04-improve-on-wallet-friction branch 13 times, most recently from 8edbac9 to d4649e8 Compare May 17, 2024 12:42

This was referenced May 20, 2024

Expose NetworkGraph accessors #293

Merged

LDK Node v0.3 Tracking Issue #192

Closed

tnull added this to the 0.3 milestone May 20, 2024

tnull force-pushed the 2024-04-improve-on-wallet-friction branch from d4649e8 to abb6926 Compare June 1, 2024 07:57

tnull mentioned this pull request Jun 11, 2024

Add anchor support #141

Merged

tnull force-pushed the 2024-04-improve-on-wallet-friction branch 2 times, most recently from 85affcf to 4abd601 Compare June 11, 2024 18:57

tnull requested a review from jkczyz June 11, 2024 19:02

tnull force-pushed the 2024-04-improve-on-wallet-friction branch from 4abd601 to fa7690b Compare June 14, 2024 07:27

jbesraa reviewed Jun 14, 2024

View reviewed changes

tnull force-pushed the 2024-04-improve-on-wallet-friction branch from fa7690b to 840ca4f Compare June 17, 2024 07:35

tnull force-pushed the 2024-04-improve-on-wallet-friction branch from 840ca4f to a85fede Compare June 18, 2024 09:15

tnull force-pushed the 2024-04-improve-on-wallet-friction branch 2 times, most recently from c1ff7a1 to 0e9caca Compare June 18, 2024 13:06

tnull requested a review from G8XSU June 18, 2024 17:58

G8XSU reviewed Jun 18, 2024

View reviewed changes

tnull force-pushed the 2024-04-improve-on-wallet-friction branch 2 times, most recently from afee329 to fcb824f Compare June 20, 2024 16:39

tnull added 13 commits June 20, 2024 19:10

Drop immediate-retry logic in wallet

fd4b33f

It's not super clear that it achieves much in the face of a rate-limited Esplora server, and having a custom sleep there is just awkward. So we drop it and hope we still get a chance to sync our on-chain wallet now and then.

Drop immediate-retry logic in tx_broadcaster

82ab9ac

.. as we're not sure it actually increases reliability. We now only log failures, ignoring HTTP 400 as this is bitcoind's error code for "transaction already in mempool".

Add timeout for on-chain syncing

f58f00f

.. to make progress and unblock the `Mutex` even if BDK's wallet `sync` would never return.

Add timeout for Lightning syncing

746014c

.. even though we don't expect this to block, we're better safe than sorry and start to introduce timeouts for any calls we make to remote servers.

Add timeout for fee rate cache updates

02e4b3f

.. even though we don't expect this to block, we're better safe than sorry and start to introduce timeouts for any calls we make to remote servers.

Add timeout for RGS updates

b0a1dfc

.. even though we don't expect this to block, we're better safe than sorry and start to introduce timeouts for any calls we make to remote servers.

Add timeout for broadcasting transactions

d67a3af

Log shutdowns of background tasks

de69c75

Shutdown: Wait for event processing to fully stop

0a0ccb1

.. before initiating the Runtime shutdown.

Bump tokio version to 1.37

5095d42

.. as we use `Clone` for `tokio::sync::watch::Sender`, which was only introduced with 1.37.

Also apply a general 10 second socket timeout for the Esplora client

f839015

tnull force-pushed the 2024-04-improve-on-wallet-friction branch from fcb824f to f839015 Compare June 20, 2024 17:10

G8XSU approved these changes Jun 20, 2024

View reviewed changes

tnull merged commit ca44721 into lightningdevkit:main Jun 20, 2024
6 of 13 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve on (potentially) blocking wallet behaviors #281

Improve on (potentially) blocking wallet behaviors #281

tnull commented Apr 12, 2024 •

edited

Loading

tnull commented Jun 1, 2024

tnull commented Jun 11, 2024

tnull commented Jun 14, 2024

jbesraa left a comment

tnull commented Jun 14, 2024 •

edited

Loading

tnull commented Jun 17, 2024

tnull commented Jun 18, 2024 •

edited

Loading

G8XSU Jun 18, 2024

tnull Jun 19, 2024 •

edited

Loading

G8XSU Jun 18, 2024

tnull Jun 19, 2024

G8XSU Jun 18, 2024

tnull Jun 19, 2024

tnull commented Jun 20, 2024

G8XSU commented Jun 20, 2024

tnull commented Jun 20, 2024

Improve on (potentially) blocking wallet behaviors #281

Improve on (potentially) blocking wallet behaviors #281

Conversation

tnull commented Apr 12, 2024 • edited Loading

tnull commented Jun 1, 2024

tnull commented Jun 11, 2024

tnull commented Jun 14, 2024

jbesraa left a comment

Choose a reason for hiding this comment

tnull commented Jun 14, 2024 • edited Loading

tnull commented Jun 17, 2024

tnull commented Jun 18, 2024 • edited Loading

G8XSU Jun 18, 2024

Choose a reason for hiding this comment

tnull Jun 19, 2024 • edited Loading

Choose a reason for hiding this comment

G8XSU Jun 18, 2024

Choose a reason for hiding this comment

tnull Jun 19, 2024

Choose a reason for hiding this comment

G8XSU Jun 18, 2024

Choose a reason for hiding this comment

tnull Jun 19, 2024

Choose a reason for hiding this comment

tnull commented Jun 20, 2024

G8XSU commented Jun 20, 2024

tnull commented Jun 20, 2024

tnull commented Apr 12, 2024 •

edited

Loading

tnull commented Jun 14, 2024 •

edited

Loading

tnull commented Jun 18, 2024 •

edited

Loading

tnull Jun 19, 2024 •

edited

Loading