Add the ability to hold HTLCs before forwarding (FEAT 52/53) #989
Conversation
This is an incredibly simple first start towards the protocol sketched out at [1]. It adds the ability to have a counteraprty hold an HTLC before forwarding it. Specifically, the HTLC sender sets a required TLV on the `update_add_htlc` message and sends an onion message to the final recipient. From there, the final recipient uses the included `reply_path` to notify the sender's counterparty that they're online and ready to receive the HTLC. In order to fully flesh out the protocol as sketched, we'll need to add an onion message mailbox service (which should be as simple as a new feature bit), add PTLCs, and extensions to BOLT 12 to allow signaling that the final recipient is often-offline. While we could add such signaling to BOLT 11, there's not a whole lot of reason to - if the recipient is able to provide an invoice, they're currently online! [1] https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-October/003307.html
TheBlueMatt
changed the title
There was a problem hiding this comment.
For this to work, we must have Sender -> Hub A -> ... -> Hub B -> Recipient where Hub A and B are always online. Hub A would hold the HTLC.
You can't ask the recipient to tell Hub A when it gets online because for that you need the recipient to be online already to receive the instruction, it has to be Hub B that tells Hub A when recipient gets online.
So when you create the payment, you need to know that the recipient is behind Hub B and that Hub B supports this hold protocol. And in the message to Hub B (with a reply path to Hub A), you need to say who is the recipient.
| 1. `tlv_stream`: `update_add_htlc_tlvs` | ||
| 2. types: | ||
| 1. type: 0 (`hold_htlc`) | ||
| 2. data: | ||
| * [`32*bytes`:`payment_release_secret`] |
There was a problem hiding this comment.
This conflicts with payments to blinded routes (https://github.com/lightning/bolts/pull/765/files), you can use 2.
| @@ -43,6 +43,7 @@ The Context column decodes as follows: | |||
| | 26/27 | `option_shutdown_anysegwit` | Future segwit versions allowed in `shutdown` | IN | | [BOLT #2][bolt02-shutdown] | | |||
| | 44/45 | `option_channel_type` | Node supports the `channel_type` field in open/accept | IN | | [BOLT #2](02-peer-protocol.md#the-open_channel-message) | | |||
| | 48/49 | `option_payment_metadata` | Payment metadata in tlv record | 9 | | [BOLT #11](11-payment-encoding.md#tagged-fields) | |||
| | 52/53 | `option_htlc_hold` | Hold HTLCs and forward on receipt of an onion message | IN | `option_onion_messages` | | |||
There was a problem hiding this comment.
Do we want to advertise this option to everyone? I can imagine that we would only accept to hold HTLCs for our customers but not for other nodes. I think it should be per channel.
There was a problem hiding this comment.
I don't see why you'd want such a restriction? The hold option can only be set by your counterparty, never a rando node, and if your counterparty wants to not use some of their channel balance...they can always do that?
There was a problem hiding this comment.
I believe we do want to advertise this to everyone, because it's a good selling point for people to open channels with you: by just looking at your node_announcement they know that you offer this feature that will be useful to them if they want to receive payments and expect to be offline sometimes.
| - MUST NOT include a `hold_htlc` TLV unless the sending node expects to be | ||
| offline for an extended duration starting soon. | ||
| - If the `hold_htlc` TLV is present: | ||
| - MUST immediately send at least two onion messages across at least two |
There was a problem hiding this comment.
Why 2? It seems like it should be an implementation decision.
Anyway, sending an onion message to a node that we expect to be offline is pointless, the message will never be delivered.
There was a problem hiding this comment.
See PR text - this assumes store-and-forward mailbox support at the recipient's LSP.
| #### TLV fields for `held_htlc_available` | ||
| 1. `tlv_stream`: `held_htlc_available` | ||
| 2. types: | ||
|
|
||
| #### TLV fields for `release_held_htlc` | ||
|
|
||
| 1. `tlv_stream`: `release_held_htlc` | ||
| 2. types: | ||
| 1. type: 0 (`payment_release_secret`) | ||
| 2. data: | ||
| * [`32*bytes`:`payment_release_secret`] |
There was a problem hiding this comment.
From the next section I get that these are onion messages. They should go in the onionmsg_payload in 04-onion-routing.md
There was a problem hiding this comment.
I kinda disagree? I mean not super strongly, I'm happy to put them anywhere, but generally I don't think we should be shoving every onion message into 04-onion-routing, that makes no sense, IMO. The BOLT 12 (somewhat obviously) PR puts its onion messages into BOLT 12, so I shoved these into the relevant place. We could also define a more general "13-onion-messages` BOLT and shove all the messages there, but I don't think that's super useful either - we generally group things by functionality, not type.
There was a problem hiding this comment.
I didn't mean that everything should go into 04-onion-routing but they need to be mentioned there as possible onion messages. Just like for BOLT 12, the details of the bolt 12 types are in there own file but they are referenced from 04-onion-routing. I'm saying this because for release_held_htlc, it's not a standard onion message TLV and it would be good to make it clear exactly where it needs to go.
|
Two more things that I think need more work:
If the recipient is online, this scheme adds an extra round trip for nothing. An alternative design would be try to relay the payment normally but add a reply path to Hub A in the payment onion, in the payload to Hub B. If the recipient is online, the payment goes through, if not then Hub B sends a |
Yes, not sure how else it'd work.
Yes you can, you just have to assume Hub B supports "onion message mailbox" support - ie store-and-forward. I noted this in the PR description/commit text.
Only on the first hop - given its only useful/used if Sender is often-offline, there's not really any reason to care about a high CLTV on the first hop - that capital wasn't going to be useful for anything anyway. You're right that we'll need a further extension to allow Hub A to send the HTLC onward with a lower CLTV than the one in the onion, add that to the list of things we need to flesh this out fully 🤷♂️ .
See above, I don't think so.
Correct, that's part of why its super explicit that "this shouldn't be used if you think the recipient is online", that's also why its not specified for BOLT 11 - see above, it only makes sense with a BOLT 12 extension where you indicate that "Hub B can provide PTLC-based invoices on my behalf". |
You should specify this "onion message mailbox" in more details, I don't think it is as obvious and simple at you imply.
That's what I meant and as I said I have already suggested a solution for that in the context of payments to blinded routes : use CLTV deltas in payment onions instead of absolute CLTVs.
Hub B needs to support "onion message mailbox", it won't work without it. So you need a way to advertise this support.
The main (only?) use case I can see for private wallet to private wallet payment is sending money to someone you trust (family or friends typically). And this does not require BOLT 12 or PTLC, it can be done today with keysend. |
With AMP, the payee can generate a static invoice and put it in an "always online" place (a QR in a website or can send an email or use any other asynchronous message mechanism). Without PTLC, the payer doesn't have any proof of payment, so she has to trust the payee. With PTLC no trust is needed. |
Not sure why this is complicated? Just store-and-forward. There's DoS questions, but not a lot we can do about that. In any case, yes, it should be specified, I'm just taking this one small step at a time.
Ah! Okay apologies, I misread. Yes, that option would address the issue as well.
Yes, to be clear, this PR as-is is maybe 1/5th of what we need. It's not useful on its own, but it's a first step towards something great.
Huh? Ability to generate a reusable invoice and receive tips is a substantial use-case that people generally expect to work when they receive a QR code, and which doesn't work almost at all today. This is how bitcoin works and is often how its used. More generally, here is the US, this is how Cash App and Venmo non-bitcoin QR codes work, and people share those as well. Further, ability to release HTLCs on receipt of a message could shift the mobile failure rate substantially - today a ton of the payment failures seen are just mobile nodes that happen to be offline. If the sender had an indication of when the recipient was online they could avoid a lot of the pain of terrible mobile-node-sending UX.
Ah, good point, we can just use keysend to avoid needing PTLCs. Of course we see still have to define a number of invoice extensions either way, and I'd generally prefer to do it on BOLT12 (a keysend option on BOLT 12 may annoy @rustyrussell but....) given we need onion messages to make this work anyway. |
That's exactly what I wrote. I understand perfectly well why this hold HTLC scheme is a good feature for sending money to people you trust (which includes tipping because you trust the person you tip not to pretend that you didn't pay them). I'm just saying that there is no need to wait for PTLC, it can be done today with a new invoice extension, the scheme described here and keysend. And even though I like BOLT 12, I don't see how it makes things better in this scenario. |
Ah, okay, I totally misunderstood what you wrote there. Seems we're talking past each other a good bit here :(
Right, I think there are use-cases for proof-of-payment that make it useful to have PTLCs for this, though I agree its not required in most use-cases. Proof of donation is a valuable thing in some contexts, as is proof-of-payment as a scheme to improve security of offline/verifying signing in some cases.
Right, I suggest it largely because it seems the obvious place to extend things. Given this work seeks to provide support for nodes which are behind an LSP, BOLT 12 is important for the same use cases - without BOLT 12, invoices for nodes behind LSPs are painfully large, with lots of readability issues. If this work didn't (a) already require new invoice extensions and (b) require onion messages anyway, I'd be considering BOLT 11 extensions or other invoice format extensions, but most of the work in implementing BOLT 12 is already required here (onion messages + blinded paths) and BOLT 12 is independently important for this use-case. |
| - MUST NOT include a `hold_htlc` TLV unless the sending node expects the | ||
| final recipient of the HTLC to be offline at the time the HTLC would arrive |
There was a problem hiding this comment.
Would it be useful for the recipient to signal that they want the sender to make use of hold_htlc in their invoice, since the sender may not know they're paying to a mostly-offline user?
Downside of course being that it's bad for privacy and you generally don't want to tell people you're offline in LN. But without some indication, your options as a sender are:
- Use
hold_htlcby default
-> Unnescesary overhead for online destinations
-> Won't work for offline users without LSPs (onion message fails quietly at offline node?) - Try without first, fail and then retry with
hold_htlc
-> Bad UX / possible stuck payment
There was a problem hiding this comment.
Would it be useful for the recipient to signal that they want the sender to make use of hold_htlc in their invoice, since the sender may not know they're paying to a mostly-offline user?
Yep! This isn't a fully-usable protocol on its own, its one of 3 or 4 parts you'd need to use this - invoice signaling being one part.
|
I believe the latest thinking for the name that people can refer to this idea is "async payments." |
|
It is worth exploring further how this proposal would work if it included support for the suggestion from @thomash-acinq to use keysend. Support for keysend would allow people to test much of the scheme before PTLCs become widely deployed by the network. A few questions about using keysend:
An argument against keysend support as an intermediate solution is that it requires work that PTLCs will make obsolete, but if the effort is minimal and/or can be reused when PTLC support is available, it seems worth it. |
|
Yea, we could consider it definitely. First we do need onion messages to be broadly available, which is still a ways off, then we'll need to define the onion message format in a second PR. Once the OM stuff lands as BOLTs I'll take a look at it (if I remember). |
|
We could make a more general way for recipients to deliver data to intermediate hops, without relying on the sender to support the feature. The hop hint in the invoice could contain an encrypted TLV payload for the intermediate hop. This way recipients would be able to encode If in the future there are other schemes that require the recipient to communicate something to an intermediate hop, this structure could be reused, and the sender wouldn't have to implement anything new. |
|
That would still result in an HTLC getting sent a few hops to a blinded path introduction point and then holding liquidity on those hops into the recipient comes online. |
It doesn't change this general flow at all (I think). I didn't mean blinded paths even. But it would provide a general way to let the recipient encode some data for an intermediate hop. So the recipient and the intermediate hop can make arbitrary agreements, without the sender having to know about it. The sender just puts the data in the payload for that hop. In this case, the recipient can encode that it wants the htlc to be held. Another case (totally unrelated to this PR) could be a non-interactive lightning proxy. The recipient can encode the actual destination in the payload for the proxy. There could be many different kinds of agreements between the recipient and the intermediate hop, that the sender doesn't need to know about. |
|
Even in that case I don't think its really acceptable - senders really need to be aware that their capital is about to be locked up for potentially-hours before they hit send on the HTLC. An intermediate node should never do that to someone unless they're aware (or being malicious). |
Point taken. Agreed. |
| 2. types: | ||
| 1. type: 0 (`hold_htlc`) | ||
| 2. data: | ||
| * [`32*bytes`:`payment_release_secret`] |
There was a problem hiding this comment.
Per our work in LDK (cc lightningdevkit/rust-lightning@6e27aec), this needs an update to remove the payment_release_secret and authenticate these messages via the blinded path only.
This is an incredibly simple first start towards the protocol
sketched out at [1]. It adds the ability to have a counteraprty
hold an HTLC before forwarding it.
Specifically, the HTLC sender sets a required TLV on the
update_add_htlcmessage and sends an onion message to the finalrecipient. From there, the final recipient uses the included
reply_pathto notify the sender's counterparty that they'reonline and ready to receive the HTLC.
In order to fully flesh out the protocol as sketched, we'll need
to add an onion message mailbox service (which should be as simple
as a new feature bit), add PTLCs, and extensions to BOLT 12 to
allow signaling that the final recipient is often-offline. While
we could add such signaling to BOLT 11, there's not a whole lot of
reason to - if the recipient is able to provide an invoice,
they're currently online!
I didn't bother basing this on the onion messages PR, but its a somewhat implied dependency.
[1] https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-October/003307.html