diff --git a/_includes/articles/cardcoins-rbf-batching.md b/_includes/articles/cardcoins-rbf-batching.md new file mode 100644 index 0000000000..d72021b8c5 --- /dev/null +++ b/_includes/articles/cardcoins-rbf-batching.md @@ -0,0 +1,54 @@ +{:.post-meta} +*by [CardCoins][]* + +_"Additive batching" is a scheme in which additional outputs are +added to unconfirmed transactions in the mempool. This field report outlines +efforts [CardCoins][] has taken in introducing a reorg- and DoS-safe implementation +of such a scheme in its customer payout workflow._ + +[Replace By Fee][topic rbf] (RBF, BIP125) and [batching][payment batching] are +two important tools for any enterprises directly interacting with Bitcoin's +mempool. Fees go up, fees go down, but the business must always fight for fee efficiency. + +Each tool, while powerful, has its own complexities and nuances. For example, +batching customer withdrawals may save on fees for the enterprise, but will likely make +[child pays for parent][topic cpfp] (CPFP) uneconomical for a customer who wishes to speed up the transaction. +Similarly, RBF is useful for an enterprise who takes a fee-underbidding strategy +(their initial transaction broadcast starts at a low fee, and is slowly bid +upwards), but it exposes their customers to [potential confusion][rbf blog] as their +withdrawal transaction updates in their wallet. It would also be messy for the +customer to spend from this transaction while it remains unconfirmed, as the +enterprise will have to pay for this child spend when attempting to replace +the parent. Even worse, the enterprise may have a withdrawal [pinned][pinning] by another +service which received the customer's withdrawal. + +When combining these two tools, a service provider unlocks new functionality but +is similarly exposed to novel forms of complexity. In the base case, combining +RBF and a single, static batch carries a simple combination of the complexities +that RBF and batching carry discretely. However, when you combine RBF and +"additive batching," emergent edge cases and dangerous failure scenarios present +themselves. + +In additive RBF batching, the service provider introduces new outputs (and +confirmed inputs) to a transaction in the mempool to incorporate new customer +withdrawals into an unconfirmed transaction. This enables the service provider +to give +users the experience of an instantaneous withdrawal while still retaining much +of the fee savings from doing large batches of customer withdrawals at once. As +each customer requests a withdrawal, an output is added to the transaction in +the mempool. This transaction continues to be updated until it confirms or +reaches some other local optimum. + +There are many strategies to this type of additive RBF batching. At [CardCoins][] we +took a safety-first approach to our implementation (with the help of [Matthew +Zipkin][]), the details of which we described in a blog post, [RBF +Batching at CardCoins: Diving into the Mempool's Dark Reorg Forest][cardcoins +rbf blog]. + +{% include references.md %} +[CardCoins]: https://www.cardcoins.co/ +[payment batching]: /en/payment-batching/ +[rbf blog]: /en/rbf-in-the-wild/#some-usability-examples +[pinning]: /en/topics/transaction-pinning/ +[Matthew Zipkin]: https://twitter.com/MatthewZipkin +[cardcoins rbf blog]: https://blog.cardcoins.co/rbf-batching-at-cardcoins-diving-into-the-mempool-s-dark-reorg-forest diff --git a/_posts/en/2021-06-16-cardcoins-rbf-batching.md b/_posts/en/2021-06-16-cardcoins-rbf-batching.md new file mode 100644 index 0000000000..287579c9f1 --- /dev/null +++ b/_posts/en/2021-06-16-cardcoins-rbf-batching.md @@ -0,0 +1,17 @@ +--- +title: 'Field Report: Using RBF and Additive Batching' +permalink: /en/cardcoins-rbf-batching/ +name: 2021-06-16-cardcoins-rbf-batching +slug: 2021-06-16-cardcoins-rbf-batching +type: posts +layout: post +lang: en +version: 1 +excerpt: > + "Additive batching" is a scheme whereby additional outputs are + included to unconfirmed transactions in the mempool. This field report outlines + efforts CardCoins has taken in introducing a reorg and DoS safe implementation + of such a scheme in its customer payout workflow. + +--- +{% include articles/cardcoins-rbf-batching.md %} diff --git a/_posts/en/newsletters/2021-06-16-newsletter.md b/_posts/en/newsletters/2021-06-16-newsletter.md new file mode 100644 index 0000000000..7afd0dc7cc --- /dev/null +++ b/_posts/en/newsletters/2021-06-16-newsletter.md @@ -0,0 +1,134 @@ +--- +title: 'Bitcoin Optech Newsletter #153' +permalink: /en/newsletters/2021/06/16/ +name: 2021-06-16-newsletter +slug: 2021-06-16-newsletter +type: newsletter +layout: newsletter +lang: en +--- +This week's newsletter celebrates the lock-in of the taproot soft fork, +describes a draft BIP for improving transaction +privacy by varying the fields used to implement anti fee sniping, and +features an article about the challenges of combining transaction +replacement with payment batching. Also included are our regular +sections with announcements of new software releases and release +candidates, plus notable changes to popular Bitcoin infrastructure +software. + +## News + +- **🟩 Taproot locked in:** the [taproot][topic taproot] soft fork and + related changes specified in BIPs [340][bip340], [341][bip341], and + [342][bip342] were locked in by signaling miners last weekend. + Taproot will be safe to use after block 709,632, which is expected in + early or mid November. The delay gives time for users to upgrade + their nodes to a release (such as Bitcoin Core 0.21.1 or later) that will + enforce taproot's rules, ensuring that funds received to taproot + scripts after block 709,632 are safe even if there's a problem with + miners. + + Developers are encouraged to start [implementing taproot][taproot + uses] so they can + be ready to take advantage of greater efficiency, privacy, and + fungibility as soon as the activation is complete. + + Readers celebrating the lock-in of taproot may also wish to read a + [short thread][wuille taproot] about taproot's origins and history by + developer Pieter Wuille. + +- **BIP proposed for wallets to set nSequence by default on taproot transactions:** + Chris Belcher [posted][belcher post] a draft BIP to the Bitcoin-Dev + mailing list suggesting an alternative way wallets can implement [anti + fee sniping][topic fee sniping]. The alternative method would enhance + the privacy and fungibility of transactions made by single-sig users, + [multisignature][topic multisignature] users, and users of certain + contract protocols such as taproot-enabled LN or advanced + [coinswaps][topic coinswap]. + + Anti fee sniping is a technique some wallets implement to discourage + miners from trying to steal fees from each other in a way that would + reduce the amount of proof of work expended on securing Bitcoin and + limit users' ability to rely on confirmation scores. All wallets + that implement anti fee sniping today use nLockTime height locks, + but it's also possible to implement the same protection using + [BIP68][] nSequence height locks. This wouldn't be any more + effective at preventing fee sniping, but it would provide a good + reason for regular wallets to set their nSequence values to the same + values that are required for transactions in certain + multisignature-based contract protocols, such as ideas for coinswaps + and taproot-enabled LN. This helps make regular wallet transactions + look like contract protocol transactions and vice versa. + + Belcher's proposal suggests wallets randomly choose between using + either nLockTime or nSequence with 50% probability when both options + are available. Overall, if the proposal is implemented, it will + allow users of regular single-sig transactions or uncomplicated + multisignatures to join together with users of contract protocols to + mutually improve each others' privacy and fungibility. + +## Field Report: Using RBF and Additive Batching + +{% include articles/cardcoins-rbf-batching.md %} + +## Releases and release candidates + +*New releases and release candidates for popular Bitcoin infrastructure +projects. Please consider upgrading to new releases or helping to test +release candidates.* + +- [Rust Bitcoin 0.26.2][] is the project's latest minor release. + Compared to the previous major version, it contains a several API + improvements and bug fixes. See the [changelog][rb changelog] for + details. + +- [Rust-Lightning 0.0.98][] is a minor release containing several + improvements and bug fixes. + +- [LND 0.13.0-beta.rc5][LND 0.13.0-beta] is a release candidate that + adds support for using a pruned Bitcoin full node, allows receiving + and sending payments using Atomic MultiPath ([AMP][topic multipath payments]), + and increases its [PSBT][topic psbt] capabilities, among other improvements + and bug fixes. + +## Notable code and documentation changes + +*Notable changes this week in [Bitcoin Core][bitcoin core repo], +[C-Lightning][c-lightning repo], [Eclair][eclair repo], [LND][lnd repo], +[Rust-Lightning][rust-lightning repo], [libsecp256k1][libsecp256k1 +repo], [Hardware Wallet Interface (HWI)][hwi repo], +[Rust Bitcoin][rust bitcoin repo], [BTCPay Server][btcpay server repo], +[Bitcoin Improvement Proposals (BIPs)][bips repo], and [Lightning +BOLTs][bolts repo].* + +- [Bitcoin Core GUI #4][] adds initial support for using [Hardware Wallet + Interface (HWI)][topic hwi] external signers via the GUI. Once this feature is + finalized, users will be able to use their HWI-compatible hardware wallets + directly from the Bitcoin Core GUI. + + {:.center} + ![Screenshot of HWI path configuration option](/img/posts/2021-06-gui-hwi.png) + +- [Bitcoin Core #21573][] updates the version of libsecp256k1 included + in Bitcoin Core. The most notable change is the use of the + optimized modular inverse code described in Newsletters [#136][news136 + safegcd] and [#146][news146 safegcd]. Performance evaluations posted + to the PR found it accelerated old block verification by about 10%. + +- [C-Lightning #4591][] adds support for parsing [bech32m][topic bech32] + addresses. C-Lightning will now permit a peer that has negotiated the feature + `option_shutdown_anysegwit` to specify any v1+ native segwit address as + a closing or withdrawal destination. + +{% include references.md %} +{% include linkers/issues.md issues="4,21573,4591" %} +[Rust Bitcoin 0.26.2]: https://github.com/rust-bitcoin/rust-bitcoin/releases/tag/0.26.2 +[Rust-Lightning 0.0.98]: https://github.com/rust-bitcoin/rust-lightning/releases/tag/v0.0.98 +[LND 0.13.0-beta]: https://github.com/lightningnetwork/lnd/releases/tag/v0.13.0-beta.rc5 +[belcher post]: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-June/019048.html +[news136 safegcd]: /en/newsletters/2021/02/17/#faster-signature-operations +[news146 safegcd]: /en/newsletters/2021/04/28/#libsecp256k1-906 +[taproot uses]: https://en.bitcoin.it/wiki/Taproot_Uses +[wuille taproot]: https://twitter.com/pwuille/status/1403725170993336322 +[rb changelog]: https://github.com/rust-bitcoin/rust-bitcoin/blob/master/CHANGELOG.md#0262---2021-06-08 diff --git a/_topics/en/bech32.md b/_topics/en/bech32.md index 6a20829225..c6f69e8c0f 100644 --- a/_topics/en/bech32.md +++ b/_topics/en/bech32.md @@ -90,6 +90,9 @@ optech_mentions: - title: Electrum 4.1.0 adds support for bech32m url: /en/newsletters/2021/05/19/#electrum-4-1-0-enhances-lightning-features + - title: C-Lightning #4591 adds support for sending to bech32m addresses + url: /en/newsletters/2021/06/16/#c-lightning-4591 + ## Optional. Same format as "primary_sources" above see_also: - title: Javascript bech32 demo decoder diff --git a/_topics/en/fee-sniping.md b/_topics/en/fee-sniping.md new file mode 100644 index 0000000000..fea9379718 --- /dev/null +++ b/_topics/en/fee-sniping.md @@ -0,0 +1,185 @@ +--- +title: Fee sniping + +## Optional. Shorter name to use for reference style links e.g., "foo" +## will allow using the link [topic foo][]. Not case sensitive +# shortname: foo + +## Optional. An entry will be added to the topics index for each alias +aliases: + - Anti fee sniping + +## Required. At least one category to which this topic belongs. See +## schema for options +categories: + - Mining + - Security Problems + - Security Enhancements + +## Optional. Produces a Markdown link with either "[title][]" or +## "[title](link)" +primary_sources: + - title: "Bitcoin Core #2340: discourage fee sniping with nLockTime" + link: https://github.com/bitcoin/bitcoin/pull/2340 + + - title: Anti-fee-sniping protection with nSequence in taproot transactions + link: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-June/019048.html + +## Optional. Each entry requires "title", "url", and "date". May also use "feature: +## true" to bold entry +optech_mentions: + - title: "LND #2063 updates sweeper to use nLockTime anti fee sniping" + url: /en/newsletters/2018/10/23/#lnd-1978 + + - title: "Suggestion to use anti fee sniping for interactive LN funding transactions" + url: /en/newsletters/2020/02/05/#interactive-construction-of-ln-funding-transactions + + - title: "C-Lightning #3465 implements anti fee sniping for withdrawal transactions" + url: /en/newsletters/2020/02/12/#c-lightning-3465 + + - title: BIP proposed for wallets to set nSequence by default on taproot transactions + url: /en/newsletters/2021/06/16/#bip-proposed-for-wallets-to-set-nsequence-by-default-on-taproot-transactions + +## Optional. Same format as "primary_sources" above +# see_also: +# - title: +# link: + +## Required. Use Markdown formatting. Only one paragraph. No links allowed. +## Should be less than 500 characters +excerpt: > + **Fee sniping** occurs when a miner deliberately re-mines one or more + previous blocks in order to take the fees from the miners who + originally created those blocks. Although re-mining a previous block + is less likely to succeed than simply extending the chain with a new + block, it can be more profitable if the previous block is worth much + more in transaction fees than the transactions currently in the + miner's mempool. + +--- +Fee sniping is a problem that may occur as Bitcoin's subsidy +continues to diminish and transaction fees begin to dominate +Bitcoin's block rewards. If transaction fees are all that matter, +then a miner with `x` percent of the hash rate has a `x` percent +chance of mining the next block, so the expected value to them of +honestly mining is `x` percent of the [best feerate set of +transactions][csb] in their mempool. + +Alternatively, a miner could dishonestly attempt to re-mine the +previous block plus a wholly new block to extend the +chain. This behavior is referred to as fee sniping, and the dishonest +miner's chance of succeeding at it if every +other miner is honest is `(x/(1-x))^2`. Even though fee sniping has an +overall lower probability of success than honest mining, attempting +dishonest mining could be the more profitable choice if transactions in +the previous block paid significantly higher feerates than the +transactions currently in the mempool---a small chance at a large amount +can be worth more than a large chance at a small amount. + +The problem is actually worse than described above because every miner +who chooses to mine dishonestly reduces the number of honest +miners trying to extend the chain. The smaller the share of hash +rate controlled by honest miners, the greater the probability that a +dishonest miner will be successful, so a single large miner +rationally choosing to mine dishonestly can set off a cascade of +ever smaller miners also rationally defecting to dishonest mining. +If that persists for an extended period of time, confirmation scores +cease to be a proxy for transaction finality and Bitcoin becomes +unusable until the problem is resolved. We expect the most likely +resolution would be centralization of mining---a cartel representing +a majority of hash rate agreeing to never reorg each others' blocks +can restore stability to the system, but that comes with the +increased risk that they'll later +censor certain transactions. + +### Mitigations + +- **Block transaction limit:** without a limit on the number of + transactions that can be contained within a block, such as a block + size or block weight limit, dishonest miners could take all the + transactions they know about now and try to put them into the oldest block + they're working to re-mine. All other blocks would be empty, with + miners only creating them to bury their re-mined block under as much + proof of work as possible. + + {:.center} + ![Illustration of fee sniping without block limits](/img/posts/2021-06-sniping-size-limit.png) + + Limiting the number of transactions that can be contained within a + Bitcoin block produces two desirable effects: + + 1. It tends to prevent any new block at the tip of the chain from + containing all pending transactions, leaving some transactions + for the next block. If the amount of transaction fee expected + from honestly mining the next block is close to the amount of + transaction fee available from dishonestly re-mining the previous + block, all rational miners will behave honestly. + + 2. It ensures that, even if dishonest miners do re-mine blocks, they + won't be able to achieve maximum revenue by leaving the blocks + near the tip empty---those blocks will need to contain fee-paying + transactions. Other dishonest miners may attempt themselves to + fee snipe those transactions, reducing the revenue of the initial + fee sniping miner and possibly discouraging them from fee sniping + in the first place. + +- **Rearrangement protection (anti fee sniping):** even with a block size + limit, a dishonest miner doesn't need to use the exact same set of + transactions included in the block that they're attempting to re-mine. + They can replace any low feerate transactions in that block with + higher feerate transactions from their mempool. This has the benefit + of burying higher feerate transactions further back in the chain where + other dishonest miners will have to work harder to re-mine them a + second time. + + {:.center} + ![Illustration of honest mining compared to fee sniping](/img/posts/2021-06-afs.png) + + It's possible to limit this abuse by preventing miners from being + able to include pending high-feerate transactions in the re-mined + version of any previous block. Miners would only be allowed to + include pending transactions in blocks after the current chain tip. + In other words, in an ideal situation, there wouldn't be any + difference between the regular transactions in blocks created by + economically rational dishonest miners and blocks created by honest + miners. This can reduce the revenue available to the dishonest + miner and make them more vulnerable to fee sniping by other + dishonest miners---again reducing the initial dishonest miner's + expected rewards possibly enough to keep them mining honestly in + the first place. + + This rearrangement protection is commonly called **anti fee + sniping.** Originally [implemented][bitcoin core #2340] in Bitcoin + Core, anti fee sniping is now also used by several other wallets. + + All wallets that implement anti fee sniping today use nLockTime + height locks to prevent a transaction from being included in the + re-mined version of a previous block. It's also [possible][belcher + post] to implement the same protection using [BIP68][] nSequence + height locks, which could help make regular wallet transactions look + like contract protocol transactions and vice versa. + +We're unaware of any developers who think the above mechanisms are a +complete solution to the fee sniping problem, but every other mitigation +that has been proposed so far seems to have significant downsides. None +of those alternatives appears to be an active area of research and +development. + + + +{% include references.md %} +{% include linkers/issues.md issues="2340" %} +[csb]: /en/newsletters/2021/06/02/#candidate-set-based-csb-block-template-construction +[belcher post]: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-June/019048.html +[news18 lnd afs]: /en/newsletters/2018/10/23/#lnd-1978 +[news84 cl afs]: /en/newsletters/2020/02/12/#c-lightning-3465 diff --git a/_topics/en/hwi.md b/_topics/en/hwi.md index dedc6ff088..405133740b 100644 --- a/_topics/en/hwi.md +++ b/_topics/en/hwi.md @@ -52,6 +52,9 @@ optech_mentions: - title: "Bitcoin Core #16546 adds external signer interface compatible with HWI" url: /en/newsletters/2021/03/03/#bitcoin-core-16546 + - title: "Bitcoin Core GUI #4 adds initial support for using HWI external signers via the GUI" + url: /en/newsletters/2021/06/16/#bitcoin-core-gui-4 + ## Optional. Same format as "primary_sources" above see_also: - title: Partially-Signed Bitcoin Transactions (PSBTs) diff --git a/_topics/en/taproot.md b/_topics/en/taproot.md index c77d73a00e..0e57772354 100644 --- a/_topics/en/taproot.md +++ b/_topics/en/taproot.md @@ -176,6 +176,9 @@ optech_mentions: - title: "Bitcoin Core #22051 adds support for importing descriptors for taproot outputs" url: /en/newsletters/2021/06/09/#bitcoin-core-22051 + - title: "Taproot locked in; activation to occur at block 709,632" + url: /en/newsletters/2021/06/16/#taproot-locked-in + ## Optional see_also: - title: MAST diff --git a/img/posts/2021-06-afs.plantuml b/img/posts/2021-06-afs.plantuml new file mode 100644 index 0000000000..a89f24673e --- /dev/null +++ b/img/posts/2021-06-afs.plantuml @@ -0,0 +1,40 @@ +@startditaa + Public Information + + Block n Mempool ++------+------+------+ +------+------+------+ +| c000 | c111 | c222 | | c111 | c222 | c666 | ++------+------+------+ +------+------+------+ +| c333 | c444 | c555 | | c888 | cAAA | cBBB | ++------+------+------+ +------+------+------+ +| c666 | c777 | cCCC | | cCCC | cDDD | cEEE | ++------+------+------+ +------+------+------+ + +------------------------------------------------ + + Honest miner works on Block n+1 + using mempool txes + + Block n Block n+1 ++------+------+------+ +------+------+------+ +| c000 | c111 | c222 | | c111 | c222 | c666 | ++------+------+------+ +------+------+------+ +| c333 | c444 | c555 | | c888 | cAAA | cBBB | ++------+------+------+ +------+------+------+ +| c666 | c777 | cCCC | | cCCC | cDDD | cEEE | ++------+------+------+ +------+------+------+ + +------------------------------------------------ + + Dishonest miner works to move high feerate +transactions from mempool into alternative block n' + + Block n' Block n'+1 ++------+------+------+ +------+------+------+ +| c000 | c111 | c111 | | c666 | c777 | c888 | ++------+------+------+ +------+------+------+ +| c222 | c222 | c333 | | cAAA | cBBB | cCCC | ++------+------+------+ +------+------+------+ +| c444 | c555 | c666 | | cCCC | cDDD | cEEE | ++------+------+------+ +------+------+------+ +@endditaa diff --git a/img/posts/2021-06-afs.png b/img/posts/2021-06-afs.png new file mode 100644 index 0000000000..bf5c1ba931 Binary files /dev/null and b/img/posts/2021-06-afs.png differ diff --git a/img/posts/2021-06-gui-hwi.png b/img/posts/2021-06-gui-hwi.png new file mode 100644 index 0000000000..ab11a18b4b Binary files /dev/null and b/img/posts/2021-06-gui-hwi.png differ diff --git a/img/posts/2021-06-sniping-size-limit.plantuml b/img/posts/2021-06-sniping-size-limit.plantuml new file mode 100644 index 0000000000..2835518150 --- /dev/null +++ b/img/posts/2021-06-sniping-size-limit.plantuml @@ -0,0 +1,46 @@ +@startditaa + Public Information + + Block n Mempool ++------+------+------+ +------+------+------+ +| c000 | c111 | c222 | | c111 | c222 | c666 | ++------+------+------+ +------+------+------+ +| c333 | c444 | c555 | | c888 | cAAA | cBBB | ++------+------+------+ +------+------+------+ +| c666 | c777 | cCCC | | cCCC | cDDD | cEEE | ++------+------+------+ +------+------+------+ + +------------------------------------------------ + + Honest miner works on Block n+1 + using mempool txes + + Block n Block n+1 ++------+------+------+ +------+------+------+ +| c000 | c111 | c222 | | c111 | c222 | c666 | ++------+------+------+ +------+------+------+ +| c333 | c444 | c555 | | c888 | cAAA | cBBB | ++------+------+------+ +------+------+------+ +| c666 | c777 | cCCC | | cCCC | cDDD | cEEE | ++------+------+------+ +------+------+------+ + +------------------------------------------------ + + Dishonest miner works to move everything + into alternative block n' + + Block n' Block n'+1 ++------+------+------+ +| c000 | c111 | c111 | (empty) ++------+------+------+ +| c222 | c222 | c333 | ++------+------+------+ +| c444 | c555 | c666 | ++------+------+------+ +| c666 | c777 | c888 | ++------+------+------+ +| cAAA | cBBB | cCCC | ++------+------+------+ +| cCCC | cDDD | cEEE | ++------+------+------+ +@endditaa diff --git a/img/posts/2021-06-sniping-size-limit.png b/img/posts/2021-06-sniping-size-limit.png new file mode 100644 index 0000000000..5ba9ebe903 Binary files /dev/null and b/img/posts/2021-06-sniping-size-limit.png differ