diff --git a/_includes/articles/suredbits-enterprise-ln.md b/_includes/articles/suredbits-enterprise-ln.md new file mode 100644 index 0000000000..fece295247 --- /dev/null +++ b/_includes/articles/suredbits-enterprise-ln.md @@ -0,0 +1,59 @@ +{:.post-meta} +*by [Roman Taranchenko][], Engineer at [Suredbits][]* + +After the first excitement of sending and, more importantly, receiving a +payment over the Lightning Network has faded away, it’s always good to +think about how to operate your node in a safe and reliable way. +Failures almost always happen unexpectedly. How do you recover after a +possible failure? How do you make backups reliable? How do you keep the +seed in a secure location? Et cetera, et cetera… + +At [Suredbits][] we use Eclair for our nodes. Even though Eclair is +pretty robust on its own, we took some steps to make it even more +reliable---such as using PostgreSQL as a database backend [(using this PR +)][db pr] and [AWS Secrets Manager][] to +store private keys. + +Eclair has a built-in online backup feature, but it requires manual +setup and script writing to automate, which is not really scalable and +is error prone. Running PostgreSQL at AWS RDS allows us to automate +backups and replication in a way that is familiar to many DevOps engineers +and which makes restoring the database state easier. + +Using PostgreSQL as a remote database backend makes node failover +simpler to implement because, if the node crashes for some reason, +there’s no need to restore the database from a backup---all you need is +to point a new Eclair instance to the correct database server. Here’s a +[quick demo][failover demo] of an automated failover implemented with two Eclair +instances plus AWS's RDS, ELB, and NAT Gateway. + +In the failover scenario depicted in the demo, we needed a secure way to +allow the node’s seed for its private keys to be shared between the Eclair instances. Eclair +stores the seed in a file on the local file system which should be +backed up somewhere and restored when needed. The current Eclair implementation +requires extra steps to do so in an automated fashion. We instead use AWS Secrets +Manager---an encrypted key/value store specifically designed to securely hold +various kinds of secrets including database passwords and encryption +keys. Now all you need to do to share the seed between the instances is to +point them to the correct secrets location in the config file. Once +configured, the instance can be stored as an AMI image which can re-imaged as +many times as needed without manual configuration. + +The measures we took are just the first steps to building +enterprise-grade Lightning nodes. There are still some more problems +that need to be solved. For example, which Hardware Security Module +(HSM) can be used for a Lightning node, or how to failover a Bitcoin +Core node in a multi-instance setting. But we believe that our work is a +solid base for scaling out Eclair and making it more fault-tolerant. + +For more information about this topic, see our [presentation][enterprise ln vid]. + +*Disclaimer: Since private keys are involved, don't use third party cloud +services without a thorough risk assessment.* + +[Roman Taranchenko]: https://github.com/rorp +[suredbits]: https://suredbits.com +[db pr]: https://github.com/ACINQ/eclair/pull/1249 +[aws secrets manager]: https://github.com/rorp/eclair/tree/aws_secretsmanager +[failover demo]: https://youtu.be/L2DtolwS8ew +[enterprise ln vid]: https://www.youtube.com/watch?v=tbwy9mJIrZE diff --git a/_posts/en/2020-05-06-suredbits-enterprise-ln.md b/_posts/en/2020-05-06-suredbits-enterprise-ln.md new file mode 100644 index 0000000000..47538d78e2 --- /dev/null +++ b/_posts/en/2020-05-06-suredbits-enterprise-ln.md @@ -0,0 +1,17 @@ +--- +title: 'Field Report: Running a Lightning node in an enterprise environment' +permalink: /en/suredbits-enterprise-ln/ +name: 2020-05-06-suredbits-enterprise-ln +slug: 2020-05-06-suredbits-enterprise-ln +type: posts +layout: post +lang: en +version: 1 + +excerpt: > + A field report about how Suredbits uses Eclair, AWS, and a remote + PostgreSQL database to operate an LN node with fast failover + capability. + +--- +{% include articles/suredbits-enterprise-ln.md %} diff --git a/_posts/en/newsletters/2020-05-06-newsletter.md b/_posts/en/newsletters/2020-05-06-newsletter.md new file mode 100644 index 0000000000..c429563ad6 --- /dev/null +++ b/_posts/en/newsletters/2020-05-06-newsletter.md @@ -0,0 +1,240 @@ +--- +title: 'Bitcoin Optech Newsletter #96' +permalink: /en/newsletters/2020/05/06/ +name: 2020-05-06-newsletter +slug: 2020-05-06-newsletter +type: newsletter +layout: newsletter +lang: en +--- +This week's newsletter links to a discussion about using enhanced QR +codes for communicating large transactions, includes a field report from +Suredbits about building a high-availability LN node, and briefly +summarizes several recently transcribed talks and conversations. Also +included are our regular sections with releases, release candidates, and +notable code changes from popular Bitcoin infrastructure software. + +## Action items + +{:.center style="font-size: 1.5em"} + +![Happy halving!](/img/posts/2020-05-halving.png) + +## News + +- **QR codes for large transactions:** QR codes can practically contain + up to about 3 kilobytes of data, which is enough to accommodate + typical user transactions but far too small to contain the largest + transactions users can normally send. Riccardo + Casatta and Christopher Allen each posted to the Bitcoin-Dev mailing + list ([1][casatta qr], [2][allen qr]) a request for discussion + hopefully leading to standardization of a method for visual + communication of [Partially Signed Bitcoin Transactions][topic psbt] + (PSBTs) and other potentially large data blobs related to Bitcoin + wallet interaction. See the [previous discussion][qr old] in the + Specter DIY repository and [continue the discussion][qr new] in the + Airgapped Signing repository. + +## Field report: running a Lightning node in an enterprise environment + +{% include articles/suredbits-enterprise-ln.md %} + +## Recently transcribed talks and conversations + +*[Bitcoin Transcripts][] is the home for transcripts of technical +Bitcoin presentations and discussions. In this new monthly feature, we +highlight a selection of the transcripts from the previous month.* + +- **Simplicity---Next-Generation Smart Contracting** Adam Back presented at a + Blockstream webinar on Simplicity, a next-generation, low-level + alternative to Bitcoin Script focused on provable security and + expressiveness. Back discussed how Simplicity, if enabled in Bitcoin + today, would allow developers to implement new functionality like + [SIGHASH_NOINPUT][topic sighash_noinput] without necessarily needing a soft fork. He also + displayed a [demo][simplicity demo] showing what you can do with + Simplicity today. ([transcript][simplicity xs], [video][simplicity + vid], [slides][simplicity slides]) + +- **Attacking Bitcoin Core** Amiti Uttarwar presented at LA BitDevs. + Uttarwar discussed how changes to Bitcoin’s peer-to-peer layer should + be assessed according to five goals: reliability, timeliness, + accessibility, privacy, and upgradability. She discussed the dangers of + network partitions and [eclipse attacks][topic eclipse attacks]---and then explained why + block-relay-only connections and anchor nodes are effective + mitigations. ([transcript][attacking xs], [video][attacking vid]) + +- **LND v0.10** Laolu Osuntokun, Joost Jager, and Oliver Gugger + presented in virtual reality at Reckless VR. Osuntokun presented Tor + and RPC enhancements in the latest release of LND plus a new channel + feature called [anchor outputs][topic anchor outputs] that addresses + the challenge of estimating onchain fees months in advance. + Jager discussed the challenges of [multipart payments][topic multipath + payments] including the splitting algorithm, what happens when the + shards of the payment arrive at different times, and strategies for + handling multipart payment failures. Gugger ended by discussing Partially + Signed Bitcoin Transaction ([PSBT][topic psbt]) channel funding and + the channel abstraction work that made this possible. + ([transcript][lnd10 xs], [video][lnd10 vid]) + +- **Grokking Bitcoin** Kalle Rosenbaum participated in a Bitcoin developer meetup + and gave a presentation at London Bitcoin Devs. The + meetup discussion focused on the role of Bitcoin technical + education, [BIP32][] HD wallets, and soft fork upgrades. For the + presentation, Rosenbaum used the content in his book to discuss how + the segwit upgrade of 2017 addressed transaction malleability and + quadratic hashing. ([Meetup transcript][grok xs2], [presentation + transcript][grok xs1], [presentation video][grok vid], [presentation + slides][grok slides]) + +## Releases and release candidates + +*New releases and release candidates for popular Bitcoin infrastructure +projects. Please consider upgrading to new releases or helping to test +release candidates.* + +- [C-Lightning 0.8.2][c-lightning 0.8.2] is a new version release + that adds support for opening channels of any size (using the + `--large-channels` configuration parameter), provides a keysend plugin + for receiving [spontaneous payments][topic spontaneous payments], and + contains several other new features and bug fixes. Additionally, both + novice and experienced users are encouraged to read the project's new + [FAQ][cl faq]. + +- [LND 0.10.0-beta][lnd 0.10.0-beta] is a major version release that + adds support for sending [multipath payments][topic multipath + payments], funding channels using an external wallet via [Partially + Signed Bitcoin Transactions][topic psbt] (PSBTs), the ability to create + invoices larger than 0.043 BTC, and several other new features and bug + fixes. Additionally, users may wish to read the new [Operational + Safety documentation][lnd op safety]. + +- [Bitcoin Core 0.20.0rc1][bitcoin core 0.20.0] is a release candidate + for the next major version of Bitcoin Core. + +## Notable code and documentation changes + +*Notable changes this week in [Bitcoin Core][bitcoin core repo], +[C-Lightning][c-lightning repo], [Eclair][eclair repo], [LND][lnd repo], +[libsecp256k1][libsecp256k1 repo], [Rust-Lightning][rust-lightning repo], +[Bitcoin Improvement Proposals (BIPs)][bips repo], and [Lightning +BOLTs][bolts repo].* + +*Note: the commits to Bitcoin Core mentioned below apply to its master +development branch and so those changes will likely not be released +until version 0.21, about six months after the release of the upcoming +version 0.20.* + +- [Bitcoin Core #16528][] allows the `createwallet` RPC to create a + wallet that uses [output script descriptors][topic descriptors] to + derive just the particular scriptPubKeys that the wallet uses to + receive payments. This is a major improvement over the way legacy + wallets scan for payments by deriving every type of script handled by + the wallet for each public key in the wallet---a technique that dates + back to the original Bitcoin 0.1 release's support for receiving to + both P2PK and P2PKH scriptPubKeys. Descriptor wallets should be more + efficient (because they don't need to scan for unused script types), + easier to upgrade to new types of script (e.g. for [taproot][topic + taproot]), and easier to use with external tools (e.g. multisig + wallets or [HWI][topic hwi]-compatible hardware wallets via + [partially-signed bitcoin transactions][topic psbt]). + + By default, descriptor wallets use the popular [BIP32][] HD wallet + paths specified by BIPs [44][BIP44], [49][BIP49], and [84][BIP84] + rather than the non-standardized fully-hardened path used in legacy + Bitcoin Core HD wallets. A number of wallet RPCs are unavailable + with descriptor wallets, either because they don't make sense with + descriptors or because developers are still adapting them to new + edge cases. The merge of this PR early in the 0.21 development + cycle and the decision to make descriptor wallets a non-default + option will give the new features six months to mature before + their expected release. + +- [Bitcoin Core #18038][] improves privacy when initially broadcasting a + transaction by reducing the frequency that a wallet will attempt to resend + from about 30 minutes to about once a day. Previously, an entity surveilling + the network could see multiple broadcasts of the same transaction during these + resend periods from the same node and conclude that wallet was the originator. + By reducing the frequency of resending attempts from the wallet, the + originator of the transaction has less chance of being identified. + + To ensure new transactions reach the network even without frequent + indiscriminate rebroadcast by the wallet, this PR also adds a type of + _unbroadcast_ transaction within the mempool. An unbroadcast transaction is + one that has been submitted locally via the wallet or RPC but which has not + yet been successfully relayed to peers on the network. Such unbroadcast + transactions remain in the mempool and will be rebroadcast every 10-15 minutes + until a peer has fetched the transaction by sending the node a `getdata` P2P + message for that transaction. + +- [BIPs #893][] makes several changes to the [BIP340][] specification of + [schnorr][topic schnorr signatures] pubkeys and signatures, with + related changes also being made to the [BIP341][] specification of + [taproot][topic taproot]. The major changes include: + + - *Alternative x-only pubkey tiebreaker:* this changes which variant + of a public key to use when only its x-coordinate is known, + as previously discussed (see [Newsletter #83][news83 alt + tiebreaker]). + + - *Updated nonce generation recommendations:* the suggested method + to use for generating a signature nonce has been updated to + prevent implementation-specific vulnerabilities (see previous + discussion in Newsletters [#83][news83 precomputed] and + [#91][news91 power analysis]). + + - *Updated tagged hashes:* the tag prefixed to hash function input + for schnorr signatures has been updated to deliberately break + compatibility with the old draft specification. Otherwise, + libraries written for the old code might sometimes generate + signatures valid under the new code, and sometimes might not, + leading to confusion. This was also previously mentioned in + [Newsletter #83][news83 alt tiebreaker]. + +- [BIPs #903][] simplifies the [BIP322][] specification of [generic + signed messages][topic generic signmessage] as previously proposed + (see [Newsletter #91][news91 bip322 update]), the change mainly + removes details that allowed signing the same message for multiple + scripts (addresses) in the same proof. + +- [BIPs #900][] updates the [BIP325][] specification of [signet][topic + signet] so that all signets use the same hardcoded genesis block + (block 0) but independent signets can be differentiated by their + [network magic][] (message start bytes). In the updated protocol, the + message start bytes are the first four bytes of a hash digest of the + network's challenge script (the script used to determine whether a + block has a valid signature). The change was [motivated][dorier + signet] by a desire to simplify the development of applications that + want to use multiple signets but which need to call libraries that + hardcode the genesis blocks for the networks they support. + +{% include references.md %} +{% include linkers/issues.md issues="16528,18038,893,903,900" %} +[bitcoin core 0.20.0]: https://bitcoincore.org/bin/bitcoin-core-0.20.0 +[lnd 0.10.0-beta]: https://github.com/lightningnetwork/lnd/releases/tag/v0.10.0-beta +[c-lightning 0.8.2]: https://github.com/ElementsProject/lightning/releases/tag/v0.8.2 +[news83 alttiebreaker]: /en/newsletters/2020/02/05/#alternative-x-only-pubkey-tiebreaker +[dorier signet]: https://github.com/bitcoin/bitcoin/pull/16411#issuecomment-577999888 +[cl faq]: https://github.com/ElementsProject/lightning/blob/master/doc/FAQ.md +[news91 bip322 update]: /en/newsletters/2020/04/01/#proposed-update-to-bip322-generic-signmessage +[casatta qr]: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020-April/017794.html +[allen qr]: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020-April/017795.html +[qr old]: https://github.com/cryptoadvance/specter-diy/issues/57 +[qr new]: https://github.com/BlockchainCommons/AirgappedSigning/issues/4 +[lnd op safety]: https://github.com/lightningnetwork/lnd/blob/master/docs/safety.md +[news83 alt tiebreaker]: /en/newsletters/2020/02/05/#alternative-x-only-pubkey-tiebreaker +[news83 precomputed]: /en/newsletters/2020/02/05/#safety-concerns-related-to-precomputed-public-keys-used-with-schnorr-signatures +[news91 power analysis]: /en/newsletters/2020/04/01/#mitigating-differential-power-analysis-in-schnorr-signatures +[network magic]: https://btcinformation.org/en/glossary/start-string +[bitcoin transcripts]: https://twitter.com/btctranscripts +[simplicity demo]: https://asciinema.org/a/rhIsJBixoB3k8yuFQQr2UGAQN +[simplicity xs]: https://diyhpl.us/wiki/transcripts/blockstream-webinars/2020-04-08-adam-back-simplicity/ +[simplicity vid]: https://www.youtube.com/watch?v=RZNCk-nyx_A +[simplicity slides]: https://docsend.com/view/svs27jr +[attacking xs]: https://diyhpl.us/wiki/transcripts/la-bitdevs/2020-04-16-amiti-uttarwar-attacking-bitcoin-core/ +[attacking vid]: https://www.youtube.com/watch?v=8TaY730YlMg +[lnd10 xs]: https://diyhpl.us/wiki/transcripts/vr-bitcoin/2020-04-18-laolu-joost-oliver-lnd0.10/ +[lnd10 vid]: https://www.youtube.com/watch?v=h34fUGuDjMg +[grok xs1]: https://diyhpl.us/wiki/transcripts/london-bitcoin-devs/2020-04-29-kalle-rosenbaum-grokking-bitcoin/ +[grok vid]: https://www.youtube.com/watch?v=6tHnYyaw0qw +[grok slides]: http://rosenbaum.se/ldnbitcoindev/drawing.sozi.html +[grok xs2]: https://diyhpl.us/wiki/transcripts/london-bitcoin-devs/2020-04-22-socratic-seminar/ diff --git a/img/posts/2020-05-halving.png b/img/posts/2020-05-halving.png new file mode 100644 index 0000000000..a60ae13159 Binary files /dev/null and b/img/posts/2020-05-halving.png differ