liquidity ads #4639

niftynei · 2021-07-09T18:01:06Z

rustyrussell

Ack e364d06

rustyrussell · 2021-07-11T03:38:23Z

gossipd/gossipd.h

@@ -60,6 +61,9 @@ struct daemon {

 	/* Features lightningd told us to set. */
 	struct feature_set *our_features;
+
+	/* The channel lease rates we're advertising */
+	struct lease_rates *rates;


Probably could be const?

rustyrussell · 2021-07-11T03:40:03Z

lightningd/gossip_msg.c

+		fromwire_lease_rates(pptr, max, entry->rates);
+	} else
+		entry->rates = NULL;
+


Oops, these functions are no longer used and should have been totally deleted. Will do after.

rustyrussell · 2021-07-11T03:41:15Z

gossipd/gossip_generation.c

@@ -125,26 +126,57 @@ static void get_nannounce_parts(const u8 *node_announcement,
 	sizes[0] = 2 + fromwire_u16(&flen, &len);
 	assert(flen != NULL && len >= 4);

+	/* BOLT-0fe3485a5320efaa2be8cfa0e570ad4d0259cec3 #7:


FWIW, I've stared using BOLT= instead, rather than commit message. Makes it a bit harder to check, but good fi spec is in flux.

rustyrussell · 2021-07-11T03:42:16Z

gossipd/gossip_generation.c

 	const u8 *orig;

 	/* Get last one we have. */
 	orig = gossip_store_get(tmpctx, gs, node->bcast.index);
 	get_nannounce_parts(orig, oparts, osizes);
 	get_nannounce_parts(nannounce, nparts, nsizes);

+	*only_missing_tlv = memeq(oparts[0], osizes[0], nparts[0], nsizes[0])


Let this be NULL? One caller wants that...

rustyrussell · 2021-07-11T04:41:21Z

gossipd/gossip_generation.c

+		}
+	}
+
+	sign_and_send_nannounce(daemon, nannounce, timestamp);


THis looks like a lot of cut and paste in what could be a flag? Can be fixed after though.

rustyrussell · 2021-07-11T05:43:04Z

bitcoin/psbt.c

+	 *
+	 * The minimum witness weight for an input is 110.
+	 */
+	weight += 110;


+= ?? This seems wrong?

Also, psbt_input_weight is a very generic name for something containing this: suggest minweight as a parameter?

Whatever code that I was using for this no longer exists, so I'm deleting this commit.

rustyrussell · 2021-07-11T05:55:24Z

contrib/pyln-client/pyln/client/lightning.py

+            "funding_feerate": funding_feerate,
+        }
+        return self.call("queryrates", payload)
+


You can avoid adding it to lightning.py: it converts unknown calls automatically.

rustyrussell · 2021-07-11T05:58:43Z

openingd/dualopend_wire.csv

+msgdata,dualopend_validate_lease,their_pubkey,pubkey,
+
+msgtype,dualopend_validate_lease_reply,7127
+msgdata,dualopend_validate_lease_reply,err_msg,?wirestring,


I probably would have left dualopend to validate the signature, and hand it up when it says channel is open, but this works too (and can be simplified later if we want).

it seems made the (incorrect) assumption that only lightningd could verify these. Ooof. Will leave as is for now.

rustyrussell · 2021-07-11T06:01:32Z

bitcoin/script.c

+ *  OP_ENDIF
+ *  OP_CHECKSIG
+ */
+u8 *bitcoin_wscript_to_local(const tal_t *ctx, u16 to_self_delay, u32 csv,


csv is a terrible name for this parameter! lease_remaining?

rustyrussell · 2021-07-11T06:10:21Z

doc/lightning-parsefeerate.7.md

+
+The **parsefeerate** command returns the current feerate for any valid
+*feerate_str*. This is useful for finding the current feerate that a
+**fundpsbt** or **utxopsbt** command might use.


This whole commit seems unnecessary? feerates does this already?

Not exactly, feerates returns the named current feerates; this takes any arbitrary feerate string and returns its value in perkw. I mean, technically, I could halfway parse the feerate string, figure out if it's a plain number or a named fee rate and if it's the latter match it up with the returned fields from feerates. Except that's a bit more work than just piping it through to lightningd to do it for me instead?

rustyrussell · 2021-07-11T07:00:17Z

Added some neatening commits on the end, and trivial rebase now first commit is merged.

niftynei · 2021-07-14T17:43:54Z

lightningd/gossip_control.c

+	u32 *lease_basis, *channel_fee_max_ppt, *funding_weight;
+
+	if (!param(cmd, buffer, params,
+		   p_req("lease_fee_base_msat", param_sat, &lease_base_sat),


I really struggled with what to name this. The field itself is satoshi only, but given that we print our RPCs out uniformly as msats, I went with the msat designation... (even though only sats are valid.)

niftynei · 2021-07-14T17:45:39Z

common/lease_rates.c

+	/* FIXME: why can't i do memeqzero? */
+	return rates->funding_weight == 0
+		&& rates->channel_fee_max_base_msat == 0
+		&& rates->channel_fee_max_proportional_thousandths == 0
+		&& rates->lease_fee_base_sat == 0
+		&& rates->lease_fee_basis == 0;


acking your later fix :)

niftynei · 2021-07-14T17:55:36Z

tests/test_opening.py

-    assert funder_opts['fuzz_percent'] == 5
+    assert funder_opts['fuzz_percent'] == 0


Broken out to separate commit.

niftynei · 2021-07-14T17:56:58Z

plugins/funder.c

+		 || chan_fee_msats)
+		policy->rates = default_lease_rates(policy);
+	else
+		policy->rates = NULL;


The true default is NULL; we only use the defaults iff any one of the parameters is set.

niftynei · 2021-07-14T17:59:47Z

bitcoin/psbt.c

+	 *
+	 * The minimum witness weight for an input is 110.
+	 */
+	weight += 110;


Whatever code that I was using for this no longer exists, so I'm deleting this commit.

niftynei · 2021-07-14T18:20:43Z

openingd/dualopend_wire.csv

+msgdata,dualopend_validate_lease,their_pubkey,pubkey,
+
+msgtype,dualopend_validate_lease_reply,7127
+msgdata,dualopend_validate_lease_reply,err_msg,?wirestring,


it seems made the (incorrect) assumption that only lightningd could verify these. Ooof. Will leave as is for now.

niftynei · 2021-07-14T18:25:47Z

bitcoin/script.c

+ *  OP_ENDIF
+ *  OP_CHECKSIG
+ */
+u8 *bitcoin_wscript_to_local(const tal_t *ctx, u16 to_self_delay, u32 csv,


niftynei · 2021-07-14T18:32:08Z

doc/lightning-parsefeerate.7.md

+
+The **parsefeerate** command returns the current feerate for any valid
+*feerate_str*. This is useful for finding the current feerate that a
+**fundpsbt** or **utxopsbt** command might use.


Not exactly, feerates returns the named current feerates; this takes any arbitrary feerate string and returns its value in perkw. I mean, technically, I could halfway parse the feerate string, figure out if it's a plain number or a named fee rate and if it's the latter match it up with the returned fields from feerates. Except that's a bit more work than just piping it through to lightningd to do it for me instead?

niftynei · 2021-07-14T18:33:08Z

plugins/libplugin.c

-	      const struct json_out *params TAKES,
-	      const char *guide,
-	      ...)
+static const char *rpc_scan_list(const tal_t *ctx,


I don't actually end up using this commit, I'm throwing it away.

Import the wires from spec. Here we go!

When we get a node announcement out, we can now pull out its offer characteristic

Correctly mark whether or not the TLV's are the same/different, given two node announcements

If there's a rate-card for liquidity, we don't know about it until after startup (the plugin *should* call us at init to tell us what their current rates are)

Changelog-Experimental: JSON-RPC: new RPC `setleaserates`, for passing in the rates to advertise for a channel lease (option_will_fund)

Note that we use the names from the spec. Changelog-Experimental: JSON-RPC: `listnodes` now includes the `lease_rates`, if available

A couple of the fields for liquidity_ads are u16

If received lease rates are empty (all zeroes), turn them off

As we move to advertising liquidity, set the default to not have any fuzz. This means we won't randomly reject an offer to buy liquidity.

Implement support for liquidity ads in `funder` plugin. We set the command line options for the leases, as well as sending the updated ads to lightningd (who then passes them through to gossipd)

Make sure everything updates/flows through as expected.

convenience, mostly

When we accept a bid to create a channel lease, we send back a signature committing to our max channel lease amounts.

When a request comes through, we forward it over to the funder who uses the currently set policy to figure out how to handle it. Includes small update to the policy engine which decides whether or not to fund a request. Changelog-Experimental: Plugins: `openchannel2` hook now includes optional fields for a channel lease request

Changelog-Experimental: EXPERIMENTAL-DUAL-FUND: JSON-RPC: openchannel_init now takes a `requested_amt`, which is an amount to request from peer

Changelog-Added: JSON-RPC: `fundchannel` now takes optional `request_amt` parameter

Adds new tables to database, backfills, basically copies the fee_rates state machine for channeld.

Useful for parsing a passed in feerate before calling lightningd with it, e.g. when you need to know what the feerate is for a fundpsbt before calling fundpsbt Changelog-Added: JSON-RPC: new command `parsefeerate` which takes a feerate string and returns the calculated perkw/perkb

If the channel is under lease, the lessor's (accepter, really) funds have a CSV lock that reflects the length of the remainder of the lease.

Channel leases modify the CSV height that an output is eligible for being spent at, persist this to the database

If an output's CSV lock hasn't been surpassed yet, don't try to include it in a transaction

Typically we forget a channel if 2016 blocks have passed and the funding transaction hasn't been mined yet, however we SHOULD NOT forget these channels if we've got funds in them!

If the channel is leased and our peer is too far behind, fail the channel.

By default, we won't close a channel that we leased to a peer. You can override this with the `force_lease_closed` flag. Changelog-Added: JSON-RPC: close now has parameter to force close a leased channel (option_will_fund)

We need to update the peer wrt our blockheight on reconnect

We're gonna need them

We need to know what the lease we're expecting is. To do this we pass around the hex encoded portion of the wire format. We can use this passed in expected lease rates to confirm that the peer is, in fact, using the same rates as what we have currently. Changelog-Added: JSON-RPC: fundchannel, multifundchannel, and openchannel_init now accept a 'compact_lease' for any requested funds

We need to parse the feerate string, so we can figure out what our weight fee will be for a leased channel, so we go get the feerate and use that to calculate what our expected lease fee will be for the requested amount.

If there's no plugin currently in place, we simply won't return any funding at all, in which case we'd expect them to handle however they want. (our implementation would fail the open, as we only accept opens that have at least as much as we've requested provided)

Make it easier to figure out what the inputs were to check_balances on two failure cases

We need the 'actual' accepter's funding for the reserve calculations, which includes the lease fee that the opener is paying them, so we calculate it before doing all that jazz. However, we MUST send the actual "on paper" (e.g. without lease fee) amount to the peer in accept_channel2, so we stash the original amount and send it.

with channel leases, we use the open_tlv data after a round of talking to the peer (which clears out the tmpctx). It'll get cleaned up when this peer opens/fails.

Check that channel leases work as expected for unilateral closes and cheats!

Since we now use 'compact_lease' to gate an open (if the rates have changed, we fail), we no longer need to rely on query rates for figuring things out, so we make it dev-only. Changelog-Changed: JSON-API: queryrates is now developer only

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

We don't actually use the NULL-is-equal in the one caller, so ignore that. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

And fix up lease_rates_fromhex to be the obvious calling convention. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

niftynei · 2021-07-20T17:28:30Z

ACK 7be8e71

niftynei requested a review from cdecker as a code owner July 9, 2021 18:01

niftynei added this to the v0.10.1 milestone Jul 9, 2021

rustyrussell approved these changes Jul 11, 2021

View reviewed changes

rustyrussell force-pushed the nifty/liquid-ad-open branch from e364d06 to 5829b2f Compare July 11, 2021 07:00

niftynei force-pushed the nifty/liquid-ad-open branch from 5829b2f to 396cd51 Compare July 14, 2021 18:45

niftynei commented Jul 14, 2021

View reviewed changes

niftynei force-pushed the nifty/liquid-ad-open branch 4 times, most recently from ed51715 to ac8e879 Compare July 19, 2021 19:06

niftynei added 7 commits July 19, 2021 16:46

liquidity-ads: import from spec

463626d

Import the wires from spec. Here we go!

lease_rates: parse them out of node_announcements

b5fc2ac

When we get a node announcement out, we can now pull out its offer characteristic

gossip/liquidity-ad: node_ann comparison, optional TLV check

cdb934e

Correctly mark whether or not the TLV's are the same/different, given two node announcements

gossipd: signal whether node_ann is updated, are we starting up?

cbfd35e

If there's a rate-card for liquidity, we don't know about it until after startup (the plugin *should* call us at init to tell us what their current rates are)

setleaserates: new RPC to pass in lease rates

1adadee

Changelog-Experimental: JSON-RPC: new RPC `setleaserates`, for passing in the rates to advertise for a channel lease (option_will_fund)

option_will_fund: print out lease rates in listnodes

de11c5d

Note that we use the names from the spec. Changelog-Experimental: JSON-RPC: `listnodes` now includes the `lease_rates`, if available

libplugin: add u16_option parsing

0227cd1

A couple of the fields for liquidity_ads are u16

niftynei force-pushed the nifty/liquid-ad-open branch from ac8e879 to 314e1b6 Compare July 19, 2021 22:04

niftynei added 11 commits July 19, 2021 20:05

gossipd: turn off lease offers if the rates are empty

e4b4020

If received lease rates are empty (all zeroes), turn them off

funder: make policy a pointer, most places

d93b193

funder_policy: set default to 0

5e5987b

As we move to advertising liquidity, set the default to not have any fuzz. This means we won't randomly reject an offer to buy liquidity.

funder-liqudity-ads: support options + setliquidity ad

3e40c83

Implement support for liquidity ads in `funder` plugin. We set the command line options for the leases, as well as sending the updated ads to lightningd (who then passes them through to gossipd)

contrib: offer a liquidity-ad lease in developer mode

5c5e7a1

tests: add test for setting/updating node announce

8aedc13

Make sure everything updates/flows through as expected.

amount: add 'is_zero' helper

1dd656c

convenience, mostly

hsmd: method to sign liquidity ad offer

c94b4e6

When we accept a bid to create a channel lease, we send back a signature committing to our max channel lease amounts.

liquidity-ad: request amount, pass through to dualopend

26e739a

Changelog-Experimental: EXPERIMENTAL-DUAL-FUND: JSON-RPC: openchannel_init now takes a `requested_amt`, which is an amount to request from peer

liquidity-ad: pipe request_amt all the way out to fundchannel

25499e0

Changelog-Added: JSON-RPC: `fundchannel` now takes optional `request_amt` parameter

niftynei and others added 26 commits July 19, 2021 20:05

liquidity-ads: persist channel blockheight states to disk

817e5ce

Adds new tables to database, backfills, basically copies the fee_rates state machine for channeld.

channel: wire blockheight updates from channeld in to database

1e6950b

liquidity-ad: wire in csv updates to commitment txs

2493bcc

channel lease: set csv_lock for commitment transactions

484bcbe

If the channel is under lease, the lessor's (accepter, really) funds have a CSV lock that reflects the length of the remainder of the lease.

channel utxo: persist the 'csv' lock value to database

7d756e0

Channel leases modify the CSV height that an output is eligible for being spent at, persist this to the database

utxo: don't add outputs that aren't currently eligible to be spent

4ff926a

If an output's CSV lock hasn't been surpassed yet, don't try to include it in a transaction

channeld: don't forget channels that we've put funds into

27657cf

Typically we forget a channel if 2016 blocks have passed and the funding transaction hasn't been mined yet, however we SHOULD NOT forget these channels if we've got funds in them!

channel lease: fail leased channel if peer offline + blockheight behind

9c39e6a

If the channel is leased and our peer is too far behind, fail the channel.

close: param to force-close a leased channel

8097937

By default, we won't close a channel that we leased to a peer. You can override this with the `force_lease_closed` flag. Changelog-Added: JSON-RPC: close now has parameter to force close a leased channel (option_will_fund)

channel lease: update the blockheight on reconnect, also

74ccf75

We need to update the peer wrt our blockheight on reconnect

lease rates: helpers for parsing, serializing + equating lease_rates

8666cbb

We're gonna need them

channel leases: add expected lease fee to opening PSBT

93854d5

We need to parse the feerate string, so we can figure out what our weight fee will be for a leased channel, so we go get the feerate and use that to calculate what our expected lease fee will be for the requested amount.

nit: better error messages on dualopend's check_balances failures

02644be

Make it easier to figure out what the inputs were to check_balances on two failure cases

dualopend: hold onto open_tlv for longer than the tmpctx persists

64123f1

with channel leases, we use the open_tlv data after a round of talking to the peer (which clears out the tmpctx). It'll get cleaned up when this peer opens/fails.

lease-channels: tests for the test gods

86dacd9

Check that channel leases work as expected for unilateral closes and cheats!

queryrates: make it dev-only

8729d6b

Since we now use 'compact_lease' to gate an open (if the rates have changed, we fail), we no longer need to rely on query rates for figuring things out, so we make it dev-only. Changelog-Changed: JSON-API: queryrates is now developer only

reestablish_tlvs: don't lose them btw tmpctx cleanup

7049c30

overflows: helpers for integer assignment overflows.

48a46a3

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

lease_rates, funder: use overflow helpers

26474c7

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

gossipd: de-duplicate code.

4a967ba

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

lease_rates: lease_rates_eq.

f8f4e43

We don't actually use the NULL-is-equal in the one caller, so ignore that. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

common/lease_rates: more things const.

103d3f1

And fix up lease_rates_fromhex to be the obvious calling convention. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

niftynei force-pushed the nifty/liquid-ad-open branch from e37f3ac to 103d3f1 Compare July 20, 2021 01:06

printlogs for failing tests

7be8e71

niftynei merged commit 35bec51 into ElementsProject:master Jul 20, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

liquidity ads #4639

liquidity ads #4639

niftynei commented Jul 9, 2021

rustyrussell left a comment

rustyrussell Jul 11, 2021

rustyrussell Jul 11, 2021

rustyrussell Jul 11, 2021

rustyrussell Jul 11, 2021

rustyrussell Jul 11, 2021

rustyrussell Jul 11, 2021

niftynei Jul 14, 2021

rustyrussell Jul 11, 2021

rustyrussell Jul 11, 2021

niftynei Jul 14, 2021

rustyrussell Jul 11, 2021

niftynei Jul 14, 2021

rustyrussell Jul 11, 2021

niftynei Jul 14, 2021

rustyrussell commented Jul 11, 2021

niftynei Jul 14, 2021

niftynei Jul 14, 2021

niftynei Jul 14, 2021

niftynei Jul 14, 2021

niftynei Jul 14, 2021

niftynei Jul 14, 2021

niftynei Jul 14, 2021

niftynei Jul 14, 2021

niftynei Jul 14, 2021

niftynei commented Jul 20, 2021

		assert funder_opts['fuzz_percent'] == 5
		assert funder_opts['fuzz_percent'] == 0

liquidity ads #4639

liquidity ads #4639

Conversation

niftynei commented Jul 9, 2021

rustyrussell left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rustyrussell commented Jul 11, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

niftynei commented Jul 20, 2021