Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Batching of HTLC transactions for anchor output channels #3064

Open
wvanlint opened this issue May 13, 2024 · 3 comments · May be fixed by #3340
Open

Batching of HTLC transactions for anchor output channels #3064

wvanlint opened this issue May 13, 2024 · 3 comments · May be fixed by #3340
Assignees
Milestone

Comments

@wvanlint
Copy link
Contributor

HTLC transactions for anchor output channels can be aggregated with SIGHASH_SINGLE|SIGHASH_ANYONECANPAY. However, if I understand correctly, LDK currently performs only limited aggregation: only for HTLC success transactions, and only for preimages provided before force-closure, not afterwards.

In the ideal case, all HTLC transactions can be aggregated per block, and a change output from one block can be used for the next block. This could significantly reduce the number of UTXOs (as opposed to the amount) that need to be reserved for anchor output channels.

Happy to take a look as well!

cc @wpaulino

@wpaulino
Copy link
Contributor

While we can definitely aggregate more aggressively, we may want to split up packages as HTLCs get close (e.g., 6-12 blocks left) to their deadline and the package they were in remains unconfirmed.

@wvanlint
Copy link
Contributor Author

While we can definitely aggregate more aggressively, we may want to split up packages as HTLCs get close (e.g., 6-12 blocks left) to their deadline and the package they were in remains unconfirmed.

Would we want to split up packages due to any transaction pinning concerns or to choose a higher fee rate for those HTLCs? If the latter, there could also be a trade-off to fee bump the entire package with the higher fee rate.

@TheBlueMatt
Copy link
Collaborator

For received HTLCs that haven't expired, the counterparty can't claim them so there's not much to talk about there (but indeed if they're getting close to expiry it may make sense to split them off and start giving them higher fee, though you can of course just assign higher fee to the whole package).

For sent HTLCs (or received HTLCs once they expire or revoked outputs if our counterparty broadcasts a stale state), the counterparty could be able to spend the same output, which introduces pinning questions.

However, its not clear to me that its worth trying to materially change the claiming logic for them - if a counterparty is trying to exploit you by pinning, they're going to just push the maximum possible amount as pending HTLC(s) and then try to pin them. If we assume pinning is 100% reliable at delaying an HTLC claim until it has expired then they will just always succeed and we should address pinning with mempool monitoring and human intervention (eg transaction accelerator services) instead. If, on the other hand, we assign some probability of succeeding at pinning, and we assume trials are uncorrelated (they almost certainly are not), then we should not combine claims too much to at least force the counterparty to do a handful of trials so that we get some successes.

Of course none of that means we should aggregate spends of outputs that we believe are not pin-able (unexpired received HTLCs and revoked counterparty balance, basically, see lightning/bolts#803) with ones that we believe may be pin-able, but it seems like a reasonable policy might be to just claim everything in two buckets - pin-able and not-pin-able outputs.

@TheBlueMatt TheBlueMatt added this to the 0.1 milestone Jul 15, 2024
@TheBlueMatt TheBlueMatt added the Take a Friday Leave a Friday Stomp the Bugs, Without Much Commitment label Jul 15, 2024
TheBlueMatt added a commit to TheBlueMatt/rust-lightning that referenced this issue Aug 28, 2024
When we first added batch claiming, we only did so for claims which
we thought were not pinnable, i.e. those for which only we can
claim the output.

This was the conservative choice - any outputs which we think are
potentially pinnable might be pinned, leaving our entire batch
unable to confirm on chain. However, if we assume that pinnable
outputs are, indeed pinnable, and attempts to pin a transaction and
keep it from confirming have highly correlated success rates,
there's no reason we shouldn't also batch claims of pinnable
outputs separately.

Here we do so, grouping batch-claimable outputs into pinnable and
unpinnable `AggregationCluster`s. We aggregate all outputs (which
have some time left before expiry) in each cluster jointly.

Fixes lightningdevkit#3064
TheBlueMatt added a commit to TheBlueMatt/rust-lightning that referenced this issue Sep 7, 2024
When we first added batch claiming, we only did so for claims which
we thought were not pinnable, i.e. those for which only we can
claim the output.

This was the conservative choice - any outputs which we think are
potentially pinnable might be pinned, leaving our entire batch
unable to confirm on chain. However, if we assume that pinnable
outputs are, indeed pinnable, and attempts to pin a transaction and
keep it from confirming have highly correlated success rates,
there's no reason we shouldn't also batch claims of pinnable
outputs separately.

Sadly, aggregating other types of inputs is rather nontrivial, as
evidenced by the size of this commit -

HTLC-Timeout claims have locktimes fixed by our counterparty's
signature and thus can only be aggregated with other HTLCs of the
same CLTV, which we have to check for.

Further, our concept of "pinnable" is deliberately somewhat fuzzy -
outputs which we believe are not pinnable will eventually become
pinnable at the height where our counterparty can spend them. Thus,
we treat outputs as pinnable if they're over that height, and if
they're within `COUNTERPARTY_CLAIMABLE_WITHIN_BLOCKS_PINNABLE` of
that height we treat them as neither pinnable nor not-pinnable,
aggregating such claims only with other claims which will become
pinnable at the same height.

However, the complexity required is worth it - aggregation can save
our users a lot of money in the case of a force-close, and directly
impacts the amount of UTXOs they need as reserve for anchors, so we
want to aggregate as much as possible.

Fixes lightningdevkit#3064
wvanlint pushed a commit to wvanlint/rust-lightning that referenced this issue Sep 17, 2024
When we first added batch claiming, we only did so for claims which
we thought were not pinnable, i.e. those for which only we can
claim the output.

This was the conservative choice - any outputs which we think are
potentially pinnable might be pinned, leaving our entire batch
unable to confirm on chain. However, if we assume that pinnable
outputs are, indeed pinnable, and attempts to pin a transaction and
keep it from confirming have highly correlated success rates,
there's no reason we shouldn't also batch claims of pinnable
outputs separately.

Sadly, aggregating other types of inputs is rather nontrivial, as
evidenced by the size of this commit -

HTLC-Timeout claims have locktimes fixed by our counterparty's
signature and thus can only be aggregated with other HTLCs of the
same CLTV, which we have to check for.

Further, our concept of "pinnable" is deliberately somewhat fuzzy -
outputs which we believe are not pinnable will eventually become
pinnable at the height where our counterparty can spend them. Thus,
we treat outputs as pinnable if they're over that height, and if
they're within `COUNTERPARTY_CLAIMABLE_WITHIN_BLOCKS_PINNABLE` of
that height we treat them as neither pinnable nor not-pinnable,
aggregating such claims only with other claims which will become
pinnable at the same height.

However, the complexity required is worth it - aggregation can save
our users a lot of money in the case of a force-close, and directly
impacts the amount of UTXOs they need as reserve for anchors, so we
want to aggregate as much as possible.

Fixes lightningdevkit#3064
TheBlueMatt added a commit to TheBlueMatt/rust-lightning that referenced this issue Sep 18, 2024
When we first added batch claiming, we only did so for claims which
we thought were not pinnable, i.e. those for which only we can
claim the output.

This was the conservative choice - any outputs which we think are
potentially pinnable might be pinned, leaving our entire batch
unable to confirm on chain. However, if we assume that pinnable
outputs are, indeed pinnable, and attempts to pin a transaction and
keep it from confirming have highly correlated success rates,
there's no reason we shouldn't also batch claims of pinnable
outputs separately.

Sadly, aggregating other types of inputs is rather nontrivial, as
evidenced by the size of this commit -

HTLC-Timeout claims have locktimes fixed by our counterparty's
signature and thus can only be aggregated with other HTLCs of the
same CLTV, which we have to check for.

Further, our concept of "pinnable" is deliberately somewhat fuzzy -
outputs which we believe are not pinnable will eventually become
pinnable at the height where our counterparty can spend them. Thus,
we treat outputs as pinnable if they're over that height, and if
they're within `COUNTERPARTY_CLAIMABLE_WITHIN_BLOCKS_PINNABLE` of
that height we treat them as neither pinnable nor not-pinnable,
aggregating such claims only with other claims which will become
pinnable at the same height.

However, the complexity required is worth it - aggregation can save
our users a lot of money in the case of a force-close, and directly
impacts the amount of UTXOs they need as reserve for anchors, so we
want to aggregate as much as possible.

Fixes lightningdevkit#3064
wvanlint pushed a commit to wvanlint/rust-lightning that referenced this issue Sep 19, 2024
When we first added batch claiming, we only did so for claims which
we thought were not pinnable, i.e. those for which only we can
claim the output.

This was the conservative choice - any outputs which we think are
potentially pinnable might be pinned, leaving our entire batch
unable to confirm on chain. However, if we assume that pinnable
outputs are, indeed pinnable, and attempts to pin a transaction and
keep it from confirming have highly correlated success rates,
there's no reason we shouldn't also batch claims of pinnable
outputs separately.

Sadly, aggregating other types of inputs is rather nontrivial, as
evidenced by the size of this commit -

HTLC-Timeout claims have locktimes fixed by our counterparty's
signature and thus can only be aggregated with other HTLCs of the
same CLTV, which we have to check for.

Further, our concept of "pinnable" is deliberately somewhat fuzzy -
outputs which we believe are not pinnable will eventually become
pinnable at the height where our counterparty can spend them. Thus,
we treat outputs as pinnable if they're over that height, and if
they're within `COUNTERPARTY_CLAIMABLE_WITHIN_BLOCKS_PINNABLE` of
that height we treat them as neither pinnable nor not-pinnable,
aggregating such claims only with other claims which will become
pinnable at the same height.

However, the complexity required is worth it - aggregation can save
our users a lot of money in the case of a force-close, and directly
impacts the amount of UTXOs they need as reserve for anchors, so we
want to aggregate as much as possible.

Fixes lightningdevkit#3064
wvanlint added a commit to wvanlint/rust-lightning that referenced this issue Sep 21, 2024
When we first added batch claiming, we only did so for claims which
we thought were not pinnable, i.e. those for which only we can
claim the output.

This was the conservative choice - any outputs which we think are
potentially pinnable might be pinned, leaving our entire batch
unable to confirm on chain. However, if we assume that pinnable
outputs are, indeed pinnable, and attempts to pin a transaction and
keep it from confirming have highly correlated success rates,
there's no reason we shouldn't also batch claims of pinnable
outputs separately.

Sadly, aggregating other types of inputs is rather nontrivial, as
evidenced by the size of this commit.

HTLC-Timeout claims have locktimes fixed by our counterparty's
signature and thus can only be aggregated with other HTLCs of the
same CLTV, which we have to check for.

Further, our concept of "pinnable" is deliberately somewhat fuzzy -
outputs which we believe are not pinnable will eventually become
pinnable at the height where our counterparty can spend them. Thus,
we treat outputs as pinnable if they're over that height, and if
they're within `COUNTERPARTY_CLAIMABLE_WITHIN_BLOCKS_PINNABLE` of
that height we treat them as neither pinnable nor not-pinnable,
aggregating such claims only with other claims which will become
pinnable at the same height.

However, the complexity required is worth it - aggregation can save
our users a lot of money in the case of a force-close, and directly
impacts the amount of UTXOs they need as reserve for anchors, so we
want to aggregate as much as possible.

Fixes lightningdevkit#3064

Co-authored-by: Matt Corallo <git@bluematt.me>
wvanlint added a commit to wvanlint/rust-lightning that referenced this issue Sep 24, 2024
When we first added batch claiming, we only did so for claims which
we thought were not pinnable, i.e. those for which only we can
claim the output.

This was the conservative choice - any outputs which we think are
potentially pinnable might be pinned, leaving our entire batch
unable to confirm on chain. However, if we assume that pinnable
outputs are, indeed pinnable, and attempts to pin a transaction and
keep it from confirming have highly correlated success rates,
there's no reason we shouldn't also batch claims of pinnable
outputs separately.

Sadly, aggregating other types of inputs is rather nontrivial, as
evidenced by the size of this commit.

HTLC-Timeout claims have locktimes fixed by our counterparty's
signature and thus can only be aggregated with other HTLCs of the
same CLTV, which we have to check for.

Further, our concept of "pinnable" is deliberately somewhat fuzzy -
outputs which we believe are not pinnable will eventually become
pinnable at the height where our counterparty can spend them. Thus,
we treat outputs as pinnable if they're over that height, and if
they're within `COUNTERPARTY_CLAIMABLE_WITHIN_BLOCKS_PINNABLE` of
that height we treat them as neither pinnable nor not-pinnable,
aggregating such claims only with other claims which will become
pinnable at the same height.

However, the complexity required is worth it - aggregation can save
our users a lot of money in the case of a force-close, and directly
impacts the amount of UTXOs they need as reserve for anchors, so we
want to aggregate as much as possible.

Fixes lightningdevkit#3064

Co-authored-by: Matt Corallo <git@bluematt.me>
@TheBlueMatt TheBlueMatt removed the Take a Friday Leave a Friday Stomp the Bugs, Without Much Commitment label Oct 19, 2024
@TheBlueMatt TheBlueMatt linked a pull request Oct 19, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants