Skip to content

Commit 94bf69e

Browse files
hardingharding
committed
Xapo Consolidation: make text includable in multiple pages
Except for a couple extra invisible newline in the HTML source, renders identically to before. This will allow us to keep the article text synchronised with the text in the newsletter. - Moves text of article to _includes/ - Adds copyright statement to top, at least until we get permission to MIT license - Defines a variable that allows changing the subhead depth - Includes that file in the previous file
1 parent b57b238 commit 94bf69e

File tree

2 files changed

+83
-78
lines changed

2 files changed

+83
-78
lines changed

‎_includes/articles/towns-xapo-consolidation.md‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
{% comment %}<!--
2+
Copyright 2018 Anthony Towns
3+
-->{% endcomment %}
4+
5+
As mentioned in [newsletter #3][newsletter 3], the past few months of low
6+
transaction fees has made it a great time to do UTXO consolidation!
7+
Consolidation has been one of a variety of activities [Xapo][Xapo] has been
8+
undertaking to be prepared for the next time fees spike like they did in the
9+
last few months of 2017.
10+
11+
{% assign img1_label = "Plot of total Bitcoin UXTOs, January - July 2018" %}
12+
13+
{:.center}
14+
![{{img1_label}}](/img/posts/utxo-consolidation-2018.png)<br>
15+
*{{img1_label}},
16+
source: [Statoshi](https://statoshi.info/dashboard/db/unspent-transaction-output-set?panelId=6&fullscreen&from=1514759562608&to=1532039707168)*
17+
18+
[newsletter 3]: https://bitcoinops.org/en/newsletters/2018/07/10/#dashboard-items
19+
[Xapo]: https://www.xapo.com/
20+
21+
The idea behind UTXO consolidation is essentially this: when your average
22+
outgoing payment is larger than your average incoming payment (or when they’re
23+
the same, but you’re batching outgoing payments), you’ll often have to combine
24+
many UTXOs in order to fund an outgoing transaction, which increases the size
25+
of your transactions and hence the fees you pay. By consolidating UTXOs in
26+
advance, you can move most of that data forward in time, giving you more
27+
control over when most of those costs are incurred, and if you can do it when
28+
fees are low, that lets you reduce those costs pretty substantially. For
29+
example, if you would have spent a dozen 2-of-3 multisig inputs at 100 s/B
30+
(satoshis per byte), that would cost around 360,000 satoshis; while if you
31+
consolidated those inputs beforehand at 2 s/B, and then spent the single
32+
consolidated input later at 100 s/B, your total cost for the two transactions
33+
is only about 41,000 satoshis. And if fees don’t rise the risk isn’t huge: if
34+
fees just sat at 2 s/B, you’d be spending 7,900 satoshis across two
35+
transactions if you consolidated, rather than spending 7,200 satoshis in a
36+
single transaction if you did nothing.
37+
38+
Consolidation also gives an opportunity to update the addresses you use for
39+
your UTXOs, for example to roll keys over, switch to multisig, or switch to
40+
segwit or bech32 addresses. And reducing the number of UTXOs makes it easier to
41+
run a full node too, marginally improving Bitcoin’s decentralisation and
42+
overall security, which is always nice.
43+
44+
Of course, one thing you really don’t want to have happen is for your
45+
consolidation transactions to somehow fill up the blockchain and cause fees to
46+
immediately start rising! There are two metrics to watch to avoid this risk:
47+
one is whether the [mempool][mempool] is full (which causes the minimum
48+
acceptable fee to rise), and the other is how much empty space there has been
49+
in recent blocks (which gives an indication of whether miners will accept more
50+
transactions at the minimum fee). Both these metrics have been very promising
51+
most of the time over the past few months: the mempool has regularly been close
52+
to empty, meaning the transactions paying as little as 1 s/B have been
53+
propagated to miners, and many blocks have not been full, meaning cheap
54+
consolidation transactions will get mined reasonably quickly, rather than
55+
creating a backlog that will cause fees to rise.
56+
57+
[mempool]: https://statoshi.info/dashboard/db/memory-pool?panelId=1&fullscreen&from=1509458400000&to=1531813659334&theme=dark
58+
59+
The approach we took to actually doing the consolidation was to have a script
60+
that would select groups of small UTXOs, and create a consolidation transaction
61+
spending them to a single pool address at a fee rate of 1.01 satoshis per byte.
62+
The script gradually feeds consolidation transactions into the network, so it
63+
doesn’t cause too large a spike in the mempool, and perhaps more importantly so
64+
we don’t risk having our transactions get dropped because they have low fees
65+
and the mempool has filled up. We triggered this manually when we were
66+
comfortable it won’t interfere with our operations, and when there doesn’t seem
67+
to be much load on the Bitcoin network in general.
68+
69+
All in all, this has worked out pretty well; we’ve reduced our UTXO count by
70+
something like [4 million UTXOs][4 million UTXOs] this year, and aside from
71+
some [concerned][redditors1] [redditors][redditors2], the cost to the network
72+
as a whole has been minimal, as has the cost to us.
73+
74+
[4 million UTXOs]: https://www.oxt.me/entity/Xapo
75+
[redditors1]: https://www.reddit.com/r/BitcoinDiscussion/comments/8ocyc9/massive_consolidation_currently_underway/
76+
[redditors2]: https://www.reddit.com/r/Bitcoin/comments/8p3y5b/does_xapo_spamming_the_blockchain/
77+
78+
{{include.hlevel | default: '##'}} Additional resources
79+
80+
- [Techniques to reduce transaction fees: consolidation](https://en.bitcoin.it/wiki/Techniques_to_reduce_transaction_fees#Consolidation) - Bitcoin Wiki
81+
- [How to cheaply consolidate coins to reduce miner fees](https://en.bitcoin.it/wiki/How_to_cheaply_consolidate_coins_to_reduce_miner_fees) - Bitcoin Wiki
82+
- [What are some best practices regarding the usage of consolidations and fanouts?](https://bitgo.freshdesk.com/support/solutions/articles/27000044185-what-are-some-best-practices-regarding-the-usage-of-consolidations-and-fanouts-) - BitGo

‎_posts/en/2018-07-30-xapo-consolidation.md‎

Lines changed: 1 addition & 78 deletions
Original file line numberDiff line numberDiff line change
@@ -12,81 +12,4 @@ version: 1
1212
*by [Anthony Towns](https://twitter.com/ajtowns)<br>Developer on Bitcoin Core
1313
at [Xapo][]*
1414

15-
As mentioned in [newsletter #3][newsletter 3], the past few months of low
16-
transaction fees has made it a great time to do UTXO consolidation!
17-
Consolidation has been one of a variety of activities [Xapo][Xapo] has been
18-
undertaking to be prepared for the next time fees spike like they did in the
19-
last few months of 2017.
20-
21-
{% assign img1_label = "Plot of total Bitcoin UXTOs, January - July 2018" %}
22-
23-
{:.center}
24-
![{{img1_label}}](/img/posts/utxo-consolidation-2018.png)<br>
25-
*{{img1_label}},
26-
source: [Statoshi](https://statoshi.info/dashboard/db/unspent-transaction-output-set?panelId=6&fullscreen&from=1514759562608&to=1532039707168)*
27-
28-
[newsletter 3]: https://bitcoinops.org/en/newsletters/2018/07/10/#dashboard-items
29-
[Xapo]: https://www.xapo.com/
30-
31-
The idea behind UTXO consolidation is essentially this: when your average
32-
outgoing payment is larger than your average incoming payment (or when they’re
33-
the same, but you’re batching outgoing payments), you’ll often have to combine
34-
many UTXOs in order to fund an outgoing transaction, which increases the size
35-
of your transactions and hence the fees you pay. By consolidating UTXOs in
36-
advance, you can move most of that data forward in time, giving you more
37-
control over when most of those costs are incurred, and if you can do it when
38-
fees are low, that lets you reduce those costs pretty substantially. For
39-
example, if you would have spent a dozen 2-of-3 multisig inputs at 100 s/B
40-
(satoshis per byte), that would cost around 360,000 satoshis; while if you
41-
consolidated those inputs beforehand at 2 s/B, and then spent the single
42-
consolidated input later at 100 s/B, your total cost for the two transactions
43-
is only about 41,000 satoshis. And if fees don’t rise the risk isn’t huge: if
44-
fees just sat at 2 s/B, you’d be spending 7,900 satoshis across two
45-
transactions if you consolidated, rather than spending 7,200 satoshis in a
46-
single transaction if you did nothing.
47-
48-
Consolidation also gives an opportunity to update the addresses you use for
49-
your UTXOs, for example to roll keys over, switch to multisig, or switch to
50-
segwit or bech32 addresses. And reducing the number of UTXOs makes it easier to
51-
run a full node too, marginally improving Bitcoin’s decentralisation and
52-
overall security, which is always nice.
53-
54-
Of course, one thing you really don’t want to have happen is for your
55-
consolidation transactions to somehow fill up the blockchain and cause fees to
56-
immediately start rising! There are two metrics to watch to avoid this risk:
57-
one is whether the [mempool][mempool] is full (which causes the minimum
58-
acceptable fee to rise), and the other is how much empty space there has been
59-
in recent blocks (which gives an indication of whether miners will accept more
60-
transactions at the minimum fee). Both these metrics have been very promising
61-
most of the time over the past few months: the mempool has regularly been close
62-
to empty, meaning the transactions paying as little as 1 s/B have been
63-
propagated to miners, and many blocks have not been full, meaning cheap
64-
consolidation transactions will get mined reasonably quickly, rather than
65-
creating a backlog that will cause fees to rise.
66-
67-
[mempool]: https://statoshi.info/dashboard/db/memory-pool?panelId=1&fullscreen&from=1509458400000&to=1531813659334&theme=dark
68-
69-
The approach we took to actually doing the consolidation was to have a script
70-
that would select groups of small UTXOs, and create a consolidation transaction
71-
spending them to a single pool address at a fee rate of 1.01 satoshis per byte.
72-
The script gradually feeds consolidation transactions into the network, so it
73-
doesn’t cause too large a spike in the mempool, and perhaps more importantly so
74-
we don’t risk having our transactions get dropped because they have low fees
75-
and the mempool has filled up. We triggered this manually when we were
76-
comfortable it won’t interfere with our operations, and when there doesn’t seem
77-
to be much load on the Bitcoin network in general.
78-
79-
All in all, this has worked out pretty well; we’ve reduced our UTXO count by
80-
something like [4 million UTXOs][4 million UTXOs] this year, and aside from
81-
some [concerned][redditors1] [redditors][redditors2], the cost to the network
82-
as a whole has been minimal, as has the cost to us.
83-
84-
[4 million UTXOs]: https://www.oxt.me/entity/Xapo
85-
[redditors1]: https://www.reddit.com/r/BitcoinDiscussion/comments/8ocyc9/massive_consolidation_currently_underway/
86-
[redditors2]: https://www.reddit.com/r/Bitcoin/comments/8p3y5b/does_xapo_spamming_the_blockchain/
87-
88-
## Additional resources
89-
90-
- [Techniques to reduce transaction fees: consolidation](https://en.bitcoin.it/wiki/Techniques_to_reduce_transaction_fees#Consolidation) - Bitcoin Wiki
91-
- [How to cheaply consolidate coins to reduce miner fees](https://en.bitcoin.it/wiki/How_to_cheaply_consolidate_coins_to_reduce_miner_fees) - Bitcoin Wiki
92-
- [What are some best practices regarding the usage of consolidations and fanouts?](https://bitgo.freshdesk.com/support/solutions/articles/27000044185-what-are-some-best-practices-regarding-the-usage-of-consolidations-and-fanouts-) - BitGo
15+
{% include articles/towns-xapo-consolidation.md hlevel='##' %}

0 commit comments

Comments
 (0)