paymod: Feature parity, finally (part 03) #3773
Conversation
I say go for it, and keep the old pay as the command |
|
Rebased on top of paymod-02: range-diff |
There was a problem hiding this comment.
Excellent series!
I believe we should do as @ZmnSCPxj suggested, and move the old one to oldpay, and simply call this pay.
We also need to remove the over-pay change: it reveals an actual issue with our implementation, which should not overpay when doing MPP of more than 1 part, IMHO (i.e. adjust CLTV only on shadow routes)
cdecker
force-pushed
the
paymod-02
branch
2 times, most recently
from
9e3bb95 to
cf3dfa7
Compare
I wonder if we should even provide a I have also been gathering some results by probing the network with the old and the new payment flow, and comparing the outcomes. This is mainly to bolster my own confidence in the new code, and get a comparison between the behavior and stability of the two. So after doing 50k payments with the new code I feel much more comfortable doing a hard switch. I also found a rare border case that would crash the
Like mentioned in the above comment, I'll move the fix in the form of an opt-out flag for amount-fuzzing to |
cdecker
force-pushed
the
paymod-03
branch
2 times, most recently
from
8715eb1 to
ff3798b
Compare
cdecker
force-pushed
the
paymod-03
branch
2 times, most recently
from
a91dd6f to
5849969
Compare
There was a problem hiding this comment.
Ack f0602e8
Minor style changes and a few questions only...
cdecker
force-pushed
the
paymod-03
branch
4 times, most recently
from
f97fd2c to
8ebcef8
Compare
We need to set some variables as well, and the transition to FAILED is a bit special in that quite some modifiers will skip into FAILED.
This proved to be rather difficult given the tight coupling of the old structs and the output of the command.
This is a slight change in interface in that the string no longer spells out that a specific node was excluded.
This is necessary for the routehint modifier.
Keeping the arguments in their non-serialized form allows us to amend them as we apply modifiers and learn new information.
So far we got away with not caring about these but since we're implementing modifiers that impact these limits, we better keep track of them.
These are primarily the fee and cltv constraints that we need to keep up to date in order to give modifiers a correct view of what is and what isn't allowed.
Since we change the interface for `pay` and `paystatus` we need a way to determine which one to expect. This just parses the COMPAT_Vxyz flags and allows us to check if, based on the configuration, we are in compat mode for that set of changes or not.
Add/remove the HTLC amount from the channel hints so concurrent getroute calls have the correct exclusions. This can sometimes underflow, if we're unlucky and call getroute too closely, but that's not a big issue, it can only result in a failed MPP attempt too many, nothing fatal, and it'll get corrected based on the result returned by the failing node.
This makes it easier to stash a human readable failure message in an attempt.
This is necessary otherwise we would not be calling modifiers for the newly set state which can lead to unexpected results.
We handle these in a number of different ways, and regularly get the parsing and logic for optional fields wrong, so let's consolidate them here.
We want to differentiate a wrong block-height from other failure reasons, such as an unknown `payment_hash`, so we skip the `waitblockheight` if we're already at the correct height.
These get reflected in the `listsendpays` command, and are quite useful.
We've been adding modifiers and arguments out of order, and we need the arguments order to match up if we want `paymod` to be a drop-in replacement.
This commit collects the changes required to the tests caused by the changes to the `pay` and `paystatus` commands. They are also rather good hints as to what these changes entail.
I wrongly used a pointer to the array that'd move on append, so an extra dereference was required here.
It turns out that the `failcodename` doesn't get populated if the `failcode` isn't a known error from the enum (duh...) so don't fail parsing if it's missing.
|
Rebased on top of |
|
Ack b606f46 |
This PR merges into the branch of the second part
paymod-02and notmaster. This should reduce the number of commits that are to be reviewed,but I'm still unclear if merging the first part will re-address this PR to
point to
master. It's an experiment, so let's see if this works for stackingmultiple PRs 😉
This is the final part of the large refactor into a new payment flow (see
#3753 and #3760 for the previous parts). It implements most of the missing
functionality as
payment_modifiers and reaches feature parity with the oldpaycommand.The PR is not as clean as it could be, in particular some things get moved
around a bit in commits to consolidate their location, but rebasing to get
them into their final position right away would be more work than its worth.
common/rpcclient.[ch]in order to cut down on future need to manuallydeserialize those results. It parses a bit more than any single use might
need, but it frees us from having to juggle JSON dict keys and the like in
the future.
New Features
I added a new
direct_pay_modwhich overrides the call togetroutealtogether if we know we have a direct channel that can be attempted for
the payment, thus disabling any form of fuzzing or shadow-routing in cases
we definitely don't need it. On failure we will still fall back to the
normal behavior.
We now track
channel_hintsby applying routes when we start an attempt,and un-applying them if the attempt fails. While we do not share these
hints across payments due to concerns of acting on stale information, these
updated hints will become very important once we have an MPP modifier which
will result in multiple concurrent attempts needing up-to-date estimates on
the state of a channel. These hints are still only best effort estimates,
and we will update them as we learn more information on capacity and other
constraints.
We move from a "compute-constraints-on-the-fly" system to a precomputed
constraints model, in which we precompute the fee and cltv budget at the
beginning and can then split that budget for retries and parallel
attempts. This reduces the number of places in which the fees and CTLV
deltas get computed considerably.
Activation
For testing purposes the new implementation is guarded behind the
COMPATflag, we are discussing however whether we just activate the new
implementation by default, and provide an escape hatch in the form of a CLI
flag to use the old implementation. We can likely gather some feedback once
merged into
mastersince the plugin repository has nightly CI runsconfigured to run against
COMPAT=0onmaster.Next steps
And now for the reason we started out on this long and tortuous journey in the
first place. We can now implement
keysendsending support and MPP sendingsupport with relative ease, by implementing them as modifiers 😉 I will
queue up those as separate pull requests.
Changelog-None