Skip to content

Conversation

@Lagrang3
Copy link

@Lagrang3 Lagrang3 commented Apr 19, 2023

Overview

This PR about implementing a payment plugin called renepay that seeks to
construct optimal Multi-Path-Payments (o-MPP) in terms of fees and reliability.
The original idea was published by Rene Pickhardt and Stefan Richter [1].
There exists a python implementation of o-MPP by Pickhardt (pickhardtpayments).

This work was possible thanks to a Build on L2 Grant.

Pickhardtpayments

Any node in the Lightning Network (LN) has knowledge of the existence of public
channels and their capacities, but the channel liquidity is unknown unless that
channel is local to the node.
Therefore, when a node constructs a route to send a payment it is assuming that
all channels along the path have enough liquidity to forward the payment amount.
If the payment fails, alternative routes can be tried and the process repeats.

It is reasonable to assume that the smaller the payment amount it is more likely
that the payment attempt succeeds. With the use of MPP one could split the
payment into smaller parts and send it along different routes. However, also the
more routes a MPP has, the more likely it will fail. One must find a compromise
between the number of routes and the size of each payment part as well as an
strategy to select routes if one wishes to maximize the probability of success
of the payment.

Pickhardt-Richter [1] paper formalizes this problem stating that the liquidity
of every channel is a random variable distributed in the integer range between
0 and C, where C is the capacity of the channel. For simplicity we are
neglecting the channel reserved fund.
A MPP can also be modeled as flow in the LN, that is an integer value is
associated to each channel, which corresponds to the amount of satoshis this
channel must forward so that this MPP can be completed. Notice that a channel
could be used by two different routes within the same MPP and the flow
associated to that channel is the sum of all funds that need to get through it.
Formally a flow is an integer function in the domain of the channels.

For a realistic payment the Flow constraints include:

  1. for any channel c the flow cannot exceed the channel's capacity:
    0 <= Flow(c) <= Capacity(c);
  2. there is satoshi conservation, hence each node has a satoshi balance computed
    as the sum of all incoming flows minus the outgoing flows,
    the sum of all nodes balance must be zero.
  3. also for the payment to considered valid one must have that
    the destination node have an balance which equals the requested payment
    amount.
    The payer node have a negative balance and must correspond to the
    payment amount plus the routing fees.
    Routing nodes will have a positive balance that corresponds to the total fees
    they collect.

The probability of success of a certain MPP, or its associated Flow,
is then the multiplication of the
probabilities that each channel is able to forward the corresponding Flow:

    Prob(Flow) = Product_{channel c} Prob( c can forward Flow(c))

Maximizing this probability is equivalent to minimizing the function -log Prob(Flow)

    - log Prob(Flow) = - sum_{channel c} log Prob( c can forward Flow(c))

If we assume that:

  1. the probability distribution of the liquidity of every channel c is such that
    -log Prob(c can forward x) is always a convex function of x;
  2. and the routing fees are negligible, hence we can assume that the balance of
    every node different from the source and the destination is zero;

then the problem of finding an optimal Flow that satisfies the capacity and balance
constraints, while maximizing Prob(Flow) can be solved efficiently using
polynomial algorithms. Chapter 14 of Ahuja-Magnanti-Orlin's Network Flows book [2]
is dedicated to this class of problems.

Files

soon

Tests

soon

To-do list

soon

References

[1] Rene Pickhardt, Stefan Richter. Optimally Reliable & Cheap Payment Flows on the Lightning Network https://arxiv.org/abs/2107.05322
[2] R.K. Ahuja, T.L. Magnanti, and J.B. Orlin. Network Flows: Theory, Algorithms, and Applications. Prentice Hall, 1993.

Lagrang3 added 30 commits April 14, 2023 14:02
- add ccan/lqueue
- use an lqueue to traverse the residual network graph in order to seach
for an admissible path from `source` to `target`.
Lagrang3 added 12 commits May 29, 2023 14:05
If two flows satisfy all bounds or no bounds, then we select the one
with the lowest fee. But if the fees are the same we prefer the highest
probability.
Signed-off-by: Lagrang3 <eduardo.quintana@pm.me>
Signed-off-by: Lagrang3 <eduardo.quintana@pm.me>
Signed-off-by: Lagrang3 <eduardo.quintana@pm.me>
Signed-off-by: Lagrang3 <eduardo.quintana@pm.me>
@Lagrang3 Lagrang3 marked this pull request as ready for review June 15, 2023 09:58
Lagrang3 added 13 commits June 20, 2023 09:42
Signed-off-by: Lagrang3 <eduardo.quintana@pm.me>
Signed-off-by: Lagrang3 <eduardo.quintana@pm.me>
We've just replicated the workflow of the standard pay command.
We first query listsendpays for previous attempts to pay for the same
hash.
Knowledge update must consider total amount of liquidity commited to a
channel in the form of HTLC.
- add a renepay object that lives throughout the life of the command.
- make a clear distinction between struct renepay and struct payment.
- add a payment module to keep the methods of renepay and payment.
- add a uncertainty_network module
- waitsendpay calls now are detached from the command, allowing to
  receive responses even after the command finishes
  (yet I think this could have been done in a more robust way using
  notifications instead of waitsendpay).
Instead of waitsendpay
@Lagrang3 Lagrang3 closed this Jul 4, 2023
endothermicdev pushed a commit that referenced this pull request Aug 8, 2025
Changelog-None: Currently, the `BOLT #12` invrequest parsing test
only tests the invrequest decode function. Add a test for the
encoding function as well by making the test roundtrip.
rustyrussell added a commit that referenced this pull request Oct 23, 2025
And add a check for new uses creeping in, since it got cut & paste
everywhere.

This means "this is a valid string, but truncate it to this many characters"
vs "%.*s" which means "only read this many characters of string":

```
['lightningd-3 2025-10-23T02:31:40.890Z **BROKEN** plugin-funder: Plugin marked as important, shutting down lightningd!']
--------------------------- Captured stderr teardown ---------------------------
    #0 0x557da58ad1dc in printf_common(void*, char const*, __va_list_tag*) asan_interceptors.cpp.o
    #1 0x557da5aff814 in json_out_addv /home/runner/work/lightning/lightning/ccan/ccan/json_out/json_out.c:239:11
    #2 0x557da59740ce in plugin_logv /home/runner/work/lightning/lightning/plugins/libplugin.c:1777:2
    #3 0x557da5969b6f in plugin_log /home/runner/work/lightning/lightning/plugins/libplugin.c:1934:2
    #4 0x557da595c4f6 in datastore_del_success /home/runner/work/lightning/lightning/plugins/funder.c:161:2
    #5 0x557da598b837 in handle_rpc_reply /home/runner/work/lightning/lightning/plugins/libplugin.c:1072:10
    #6 0x557da598a4b0 in rpc_conn_read_response /home/runner/work/lightning/lightning/plugins/libplugin.c:1361:3
    #7 0x557da5adbea5 in next_plan /home/runner/work/lightning/lightning/ccan/ccan/io/io.c:60:9
    #8 0x557da5ae06ff in do_plan /home/runner/work/lightning/lightning/ccan/ccan/io/io.c:422:8
    #9 0x557da5adfb58 in io_ready /home/runner/work/lightning/lightning/ccan/ccan/io/io.c:439:10
    #10 0x557da5aec2ce in io_loop /home/runner/work/lightning/lightning/ccan/ccan/io/poll.c:455:5
    #11 0x557da59757ac in plugin_main /home/runner/work/lightning/lightning/plugins/libplugin.c:2409:3
    #12 0x557da594fe23 in main /home/runner/work/lightning/lightning/plugins/funder.c:1723:2
    #13 0x7f6572229d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #14 0x7f6572229e3f in __libc_start_main csu/../csu/libc-start.c:392:3
    #15 0x557da588b584 in _start (/home/runner/work/lightning/lightning/plugins/funder+0x10d584) (BuildId: 71ba63ab577fc6fa60573d3e8555f6db7d5c584d)

0x624000009d28 is located 0 bytes to the right of 7208-byte region [0x624000008100,0x624000009d28)
allocated by thread T0 here:
    #0 0x557da590e7f6 in __interceptor_realloc (/home/runner/work/lightning/lightning/plugins/funder+0x1907f6) (BuildId: 71ba63ab577fc6fa60573d3e8555f6db7d5c584d)
    #1 0x557da5b2149b in tal_resize_ /home/runner/work/lightning/lightning/ccan/ccan/tal/tal.c:755:13
    #2 0x557da59f2032 in membuf_tal_resize /home/runner/work/lightning/lightning/common/utils.c:203:2
    #3 0x557da5b03934 in membuf_prepare_space_ /home/runner/work/lightning/lightning/ccan/ccan/membuf/membuf.c:45:12
    #4 0x557da59d4289 in jsonrpc_io_read_ /home/runner/work/lightning/lightning/common/jsonrpc_io.c:127:2
    #5 0x557da598a635 in rpc_conn_read_response /home/runner/work/lightning/lightning/plugins/libplugin.c:1366:9
    #6 0x557da5adbea5 in next_plan /home/runner/work/lightning/lightning/ccan/ccan/io/io.c:60:9
    #7 0x557da5ae06ff in do_plan /home/runner/work/lightning/lightning/ccan/ccan/io/io.c:422:8
    #8 0x557da5adfb58 in io_ready /home/runner/work/lightning/lightning/ccan/ccan/io/io.c:439:10
    #9 0x557da5aec2ce in io_loop /home/runner/work/lightning/lightning/ccan/ccan/io/poll.c:455:5
    #10 0x557da59757ac in plugin_main /home/runner/work/lightning/lightning/plugins/libplugin.c:2409:3
    #11 0x557da594fe23 in main /home/runner/work/lightning/lightning/plugins/funder.c:1723:2
    #12 0x7f6572229d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

SUMMARY: AddressSanitizer: heap-buffer-overflow asan_interceptors.cpp.o in printf_common(void*, char const*, __va_list_tag*)
Shadow bytes around the buggy address:
  0x0c487fff9350: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c487fff9360: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c487fff9370: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c487fff9380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c487fff9390: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c487fff93a0: 00 00 00 00 00[fa]fa fa fa fa fa fa fa fa fa fa
  0x0c487fff93b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c487fff93c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c487fff93d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c487fff93e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c487fff93f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==26122==ABORTING
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this pull request Oct 24, 2025
And add a check for new uses creeping in, since it got cut & paste
everywhere.

This means "this is a valid string, but truncate it to this many characters"
vs "%.*s" which means "only read this many characters of string":

```
['lightningd-3 2025-10-23T02:31:40.890Z **BROKEN** plugin-funder: Plugin marked as important, shutting down lightningd!']
--------------------------- Captured stderr teardown ---------------------------
    #0 0x557da58ad1dc in printf_common(void*, char const*, __va_list_tag*) asan_interceptors.cpp.o
    #1 0x557da5aff814 in json_out_addv /home/runner/work/lightning/lightning/ccan/ccan/json_out/json_out.c:239:11
    #2 0x557da59740ce in plugin_logv /home/runner/work/lightning/lightning/plugins/libplugin.c:1777:2
    #3 0x557da5969b6f in plugin_log /home/runner/work/lightning/lightning/plugins/libplugin.c:1934:2
    #4 0x557da595c4f6 in datastore_del_success /home/runner/work/lightning/lightning/plugins/funder.c:161:2
    #5 0x557da598b837 in handle_rpc_reply /home/runner/work/lightning/lightning/plugins/libplugin.c:1072:10
    #6 0x557da598a4b0 in rpc_conn_read_response /home/runner/work/lightning/lightning/plugins/libplugin.c:1361:3
    #7 0x557da5adbea5 in next_plan /home/runner/work/lightning/lightning/ccan/ccan/io/io.c:60:9
    #8 0x557da5ae06ff in do_plan /home/runner/work/lightning/lightning/ccan/ccan/io/io.c:422:8
    #9 0x557da5adfb58 in io_ready /home/runner/work/lightning/lightning/ccan/ccan/io/io.c:439:10
    #10 0x557da5aec2ce in io_loop /home/runner/work/lightning/lightning/ccan/ccan/io/poll.c:455:5
    #11 0x557da59757ac in plugin_main /home/runner/work/lightning/lightning/plugins/libplugin.c:2409:3
    #12 0x557da594fe23 in main /home/runner/work/lightning/lightning/plugins/funder.c:1723:2
    #13 0x7f6572229d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #14 0x7f6572229e3f in __libc_start_main csu/../csu/libc-start.c:392:3
    #15 0x557da588b584 in _start (/home/runner/work/lightning/lightning/plugins/funder+0x10d584) (BuildId: 71ba63ab577fc6fa60573d3e8555f6db7d5c584d)

0x624000009d28 is located 0 bytes to the right of 7208-byte region [0x624000008100,0x624000009d28)
allocated by thread T0 here:
    #0 0x557da590e7f6 in __interceptor_realloc (/home/runner/work/lightning/lightning/plugins/funder+0x1907f6) (BuildId: 71ba63ab577fc6fa60573d3e8555f6db7d5c584d)
    #1 0x557da5b2149b in tal_resize_ /home/runner/work/lightning/lightning/ccan/ccan/tal/tal.c:755:13
    #2 0x557da59f2032 in membuf_tal_resize /home/runner/work/lightning/lightning/common/utils.c:203:2
    #3 0x557da5b03934 in membuf_prepare_space_ /home/runner/work/lightning/lightning/ccan/ccan/membuf/membuf.c:45:12
    #4 0x557da59d4289 in jsonrpc_io_read_ /home/runner/work/lightning/lightning/common/jsonrpc_io.c:127:2
    #5 0x557da598a635 in rpc_conn_read_response /home/runner/work/lightning/lightning/plugins/libplugin.c:1366:9
    #6 0x557da5adbea5 in next_plan /home/runner/work/lightning/lightning/ccan/ccan/io/io.c:60:9
    #7 0x557da5ae06ff in do_plan /home/runner/work/lightning/lightning/ccan/ccan/io/io.c:422:8
    #8 0x557da5adfb58 in io_ready /home/runner/work/lightning/lightning/ccan/ccan/io/io.c:439:10
    #9 0x557da5aec2ce in io_loop /home/runner/work/lightning/lightning/ccan/ccan/io/poll.c:455:5
    #10 0x557da59757ac in plugin_main /home/runner/work/lightning/lightning/plugins/libplugin.c:2409:3
    #11 0x557da594fe23 in main /home/runner/work/lightning/lightning/plugins/funder.c:1723:2
    #12 0x7f6572229d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

SUMMARY: AddressSanitizer: heap-buffer-overflow asan_interceptors.cpp.o in printf_common(void*, char const*, __va_list_tag*)
Shadow bytes around the buggy address:
  0x0c487fff9350: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c487fff9360: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c487fff9370: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c487fff9380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c487fff9390: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c487fff93a0: 00 00 00 00 00[fa]fa fa fa fa fa fa fa fa fa fa
  0x0c487fff93b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c487fff93c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c487fff93d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c487fff93e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c487fff93f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==26122==ABORTING
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant