Work with the new dcrd reorg ntfn sequence #828
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
OnBlockConnected now relays the previous block hash from the new block header to the collection queue for verification. After sync, the best block height and hash are set in the collection queue via (*collectionQueue).SetPreviousBlock. When (*collectionQueue).ProcessBlocks verifies a block is insequence, and calls each synchronous block connected hander, it then sets the new block as the previous for validation of the next block. Add (*wiredDB).GetBestBlockHeightHash to retrieve the height and hash of the base DB's best block. Used after sync to initialize the collection queue's previous block record.
Required dcrd JSON RPC version is 5.0.0
In ntfnhandlers.go, create the basic ReorgData type without the full chains or the common ancestor, and create a reorg signal queue using a chan ReorgData. Add ReorgSignaler, which processes ReorgData sent on reorgDataChan (use queueReorg to send), allowing each package's reorg handler to finish before proceeding to the next reorg in the queue. This should be launched as a goroutine after starting the RPC client. Add queueReorg to queue a new reorg via the reorgDataChan.
Factor out data collection into (*chainMonitor).collect. Remove the reorganizing flag since the block connected handler does not help with the reorg anymore. Blocks that do not connect after the last-stored block should not be sent to BlockConnectedHandler. It is not capable of checking that the blocks it receives are next for dcrdata. It only only collects data and calles each BlockDataSaver's Store function.
The first block in the slice is now the lowest after the common ancestror, while the last is the chain tip.
Use full chain data in (*ChainMonitor).switchToSideChain. No more RPCs. Remove reorg-related fields of ChainMonitor, except the RWMutex. Just call switchToSideChain, passing the reorgData from the signal. Add reorg results checks, ensuring new tip is the expected hash at the expected height.
Remove the unused BlockConnected(|Sync) functions since they are no longer used. Only the ReorgHandler is needed now. Call switchToSide chain directly from ReorgHandler. Add checks in switchToSideChain to ensure correct starting DB height. Use the new and faster GetBestBlockHeightHash instead of GetBestBlockSummary to get just the height and hash. Add more checks on the reorg result in ReorgHandler.
Just like dcrsqlite reorg handling.
|
PR description updated with design and implementation details. Will be ready testing by other team members shortly. I will update the description when ready. Note this PR also bumps the required dcrd JSON RPC version to 5.0.0 (and only 5.0.0). Previously dcrd JSON RPC versions are incompatible! dcrd had it's JSON RPC version bumped to 5.0.0 a few hours ago in PR decred/dcrd#1531... rebuild dcrd from latest master to test! |
Add superQueue function and anyQueue (chan interface{}) for queueing
the handling of different types of event notifications, while keeping the
order in which they were received from the daemon. superQueue
manages the event notification queue, keeping the events in the
same order they were received by the rpcclient.NotificationHandlers, and
sending them to the appropriate handlers. It should be a goroutine.
MakeNodeNtfnHandlers launches superQueue, and sends both new block
data and reorg data on the anyQueue channel to superQueue.
Make package level block queue (instance of collectionQueue).
Make package level dcrd RPC client.
Create (*collectionQueue).processBlock for synchronous block processing.
Remove reorgDataChan, queueReorg, and ReorgSignaler. Instead,
perform reorg synchronously in signalReorg.
In signalReorg, make stakedb reorg run first, so that blockdata and
dcrsqlite will be able to get the ticket pool info for the new blocks.
Allow reorg handling in dcrsqlite, blockdata, and dcrpg to run
concurrently since they only need the stakedb update first.
Set the collectionQueue's previous/best block at the end of the reorg.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolving #823
collectionQueueto verify that new blocks are in sequencerpcutils.CommonAncestorfunction to provide full reorg details to each reorg handler and avoid redundant RPCs.ChainMonitor's reorg handling. Each ofdcrsqlite,dcrpg, andstakedbuse aswitchToSideChainfunction. Withblockdata,ReorgHandlersimply stores the new chain tip's block data in thereorgDataSavers, which is presently just the explorer UI update.There is no longer a need for
BlockConnectedHandlerin dcrpg and dcrsqlite, so it is removed. The goroutines are no longer started of course, and the synchronouscollectionQueuenow includes onlysdbChainMonitor.BlockConnectedSyncandwsChainMonitor.BlockConnectedSync(the synchronous collection and storage functions for the stakedb and blockdata packages.).In short, the roles of each data saver's
ReorgHandlerare:The entire process is much simpler overall. One source of complexity is the new reorg queue in the
notificationpackage, and the new and expandedtxhelpers.ReorgDatatype that adds the entire lists of blocks in the old and new chains as well as the common ancestor. The reorg queue is processed bynotification.ReorgSignaler, which uses the newrpcutils.CommonAncestorto get the common ancestor and the full list of block in the old and new chains.rpcutils.CommonAncestorfinds the common ancestor and full list of block in the old and new chain by iteratively interrogating dcrd via RPC for blocks' previous block hash until the same hash is encountered for both chains at the same height. This approach is used as opposed to two other ways: