blockchain: Optimize exported header access. #1273
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jrick
approved these changes
Jun 12, 2018
davecgh
force-pushed
the
blockchain_header_by_hash
branch
from
02b5b14 to
08c43f0
Compare
This optimizes the FetchHeader function to make use of the fact that all block nodes are now in memory and therefore it is no longer necessary to consult the database for them. It also renames the function to HeaderByHash so it is consistent with other similar functions such as HeaderByHeight, BlockByHash, and BlockByHeight and updates all callers in the repo accordingly. This speeds up fetching all of the headers in the chain by roughly 3x to 4x for a chain height of 246,000 headers. Longer chains will benefit more. For example, here is some timing information before and after this commit plus the full block index in memory commit for 246,000 headers: 7200 RPM HDD: ------------- Previous fetch time: ~15.76s New fetch time: ~4.17s SSD: ---- Previous fetch time: ~12.63s New fetch time: ~4.17s
davecgh
force-pushed
the
blockchain_header_by_hash
branch
from
08c43f0 to
a235b83
Compare
cfromknecht
added a commit
to cfromknecht/btcd
that referenced
this pull request
Aug 1, 2018
backport of decred/dcrd#1273 Notable difference being that btcd mainline currenlty doesn't have a blockchain/blockindex_test file, so those changes are ommitted. Great work @davecgh :)
cfromknecht
added a commit
to cfromknecht/btcd
that referenced
this pull request
Aug 1, 2018
backport of decred/dcrd#1273 Notable difference being that btcd mainline currenlty doesn't have a blockchain/blockindex_test file, so those changes are ommitted. Great work @davecgh :)
cfromknecht
added a commit
to cfromknecht/btcd
that referenced
this pull request
Aug 1, 2018
backport of decred/dcrd#1273 Notable difference being that btcd mainline currenlty doesn't have a blockchain/blockindex_test.go file, so those changes are omitted. Great work @davecgh :)
cfromknecht
added a commit
to cfromknecht/btcd
that referenced
this pull request
Aug 1, 2018
backport of decred/dcrd#1273 Notable difference being that btcd mainline currenlty doesn't have a blockchain/blockindex_test.go file, so those changes are omitted. Great work @davecgh :)
anchaj
pushed a commit
to phoreproject/btcd
that referenced
this pull request
Sep 24, 2018
backport of decred/dcrd#1273 Notable difference being that btcd mainline currenlty doesn't have a blockchain/blockindex_test.go file, so those changes are omitted. Great work @davecgh :)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This optimizes the
FetchHeaderfunction to make use of the fact that all block nodes are now in memory and therefore it is no longer necessary to consult the database for them.It also renames the function to
HeaderByHashso it is consistent with other similar functions such asHeaderByHeight,BlockByHash, andBlockByHeightand updates all callers in the repo accordingly.This speeds up fetching all of the headers in the chain by roughly 3x to 4x for a chain height of 246,000 headers. Longer chains will benefit more.
For example, here is some timing information before and after this commit plus the full block index in memory commit for 246,000 headers:
7200 RPM HDD:
Previous fetch time: ~15.76s
New fetch time: ~4.17s
SSD:
Previous fetch time: ~12.63s
New fetch time: ~4.17s
This is work towards #1145.