peer: Implement stall detection. #518
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
davecgh
force-pushed
the
stalldetection
branch
5 times, most recently
from
9c30c84 to
f9540f2
Compare
davecgh
force-pushed
the
stalldetection
branch
from
f9540f2 to
03b027b
Compare
|
Updated to improve log message on stalled peer and made it debug level. |
davecgh
force-pushed
the
stalldetection
branch
4 times, most recently
from
92965ce to
cbbe3a8
Compare
This commit implements stall detection logic at the peer level to detect and disconnect peers that are either not following the protocol in regards to expected response messages or have otherwise stalled. This is accomplished by setting deadlines for each message type which expects a response and periodically checking them while properly taking into account processing time. There are an increasing number of nodes on the network which claim to be full nodes, but don't actually properly implement the entire p2p protocol even though they implement it enough to cause properly implemented nodes to make data requests to which they never respond. Since btcd currently only syncs new blocks via single sync peer and, prior to this commit only had very basic stall detection, this could lead to a situation where the block download became stalled indefinitely due to one of these misbehaving peers. This commit fixes that issue since the stalled peer will now be detected and disconnected which leads to a new sync peer being selected. This logic will also fit nicely with the future multi-peer sync model which is on the roadmap and for which infrastructure work is underway. Fixes btcsuite#486 and fixes btcsuite#229.
|
I used to have 'orphan block only' symptoms multiple times per day, but definately every 12-24hrs at the minimum... with this pull request, my btcd has not skipped a beat since. Why is this not merged yet? ;-) ;-) ;-) |
|
No issues in hours with dozens of hosts. OK |
|
Reads like a hack but solves the immediate problem. Although it's not a fool proof solution (deadlines can be overwritten for example) it will be updated when multipeer syncing is added. ok |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Requires #445
This is an alternative approach to #501 that is a more general solution and also fixes some concurrency-related issues with that PR as a side effect.
This pull request implements stall detection logic at the peer level to detect and disconnect peers that are either not following the protocol in regards to expected response messages or have otherwise stalled. This
is accomplished by setting deadlines for each message type which expects a response and periodically checking them while properly taking into account processing time.
There are an increasing number of nodes on the network which claim to be full nodes, but don't actually properly implement the entire p2p protocol even though they implement it enough to cause properly implemented nodes to make data requests to which they never respond.
Since btcd currently only syncs new blocks via single sync peer and, prior to this commit only had very basic stall detection, this could lead to a situation where the block download became stalled indefinitely due to one of these misbehaving peers. This commit fixes that issue since the stalled peer will now be detected and disconnected which leads to a new sync peer being selected.
This logic will also fit nicely with the future multi-peer sync model which is on the roadmap and for which infrastructure work is underway.
Fixes #486 and #229 .