Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow syncing from pruned nodes #143

Open
3 tasks
cpacia opened this issue Nov 18, 2018 · 3 comments
Open
3 tasks

Allow syncing from pruned nodes #143

cpacia opened this issue Nov 18, 2018 · 3 comments
Labels
enhancement New feature or request

Comments

@cpacia
Copy link
Contributor

cpacia commented Nov 18, 2018

Pruned nodes signal themselves with the NodeNetworkLimited service flag. Since they are capable of serving a least a limited number of blocks, we should allow syncing from them if appropriate.

Things to do:

  • Allow nodes signaling NodeNetworkLimited to be selected as a sync candidate if we are less than 30 days from the tip. 30 days is just a guess at an appropriate number I suppose it could be a little longer.

  • We don't currently allow outgoing connections to nodes not signaling NodeNetwork. Ideally we would allow outgoing connections to NodeNetworkLimited nodes if we are less than 30 days from the tip, but at startup we don't really know how far we are from the tip. Maybe the best way to handle this is to allow at most half of our outgoing slots to be taken up by pruned nodes.

  • If we send a GetBlocks message to a pruned node and they don't have the block, they will respond with a NotFound message. We don't currently handle the NotFound and instead will eventually timeout. We need to make it handle the NotFound response and disconnect the peer.

@cpacia cpacia added the enhancement New feature or request label Nov 18, 2018
@zquestz
Copy link
Contributor

zquestz commented Nov 18, 2018

Do we really want to sync from pruned nodes even though there are plenty of full nodes on the network? Seems like a lot of complexity for relatively little gain.

@cpacia
Copy link
Contributor Author

cpacia commented Nov 18, 2018

It's not a high priority right now but if the average blocksize goes up I'd expect to see more pruned nodes and fewer full nodes to sync from.

At the end of the day the most bandwidth intensive operation is uploading historical blocks to other peers. It's like 98% of your bandwidth usage. If we can spread that cost around to more nodes (pruned nodes in this case) then we can lighten the load on everyone else.

@cpacia
Copy link
Contributor Author

cpacia commented Oct 20, 2019

We have more issues with this.

  • Fast synced nodes calculate the filter index incorrectly. Each filter header has the hash of the previous filter header, but since fast sync nodes start from a checkpoint, they don't know the filter header hash prior to the checkpoint. So all filter headers calculated by fast synced nodes are incorrect. Fixing this might require putting the filter header hash in the checkpoint.

  • Pruned, but not fastsynced, nodes can serve filters since they calculated the index from genesis and it should be correct. However, there are problems here. First, we only save a pruned flag in the database when running in either pruned mode or fast synced mode. Hence, we can't currently tell the difference between pruned and fast synced nodes to know if we can serve filters. Second, the Migrate method in the database does not work with pruned nodes so we skip migrations. To make it work would require figuring out the first saved block and migrating everything afterwards.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants