eth/downloader, trie: pull head state concurrently with chain

[karalabe]

Currently fast sync works in two phases:

• Download the blockchain + transaction receipts
• Download a pivot state and import blocks after it

Downloading the blockchain and tx data is a fairly resilient operation which can be aborted and resumed at will. However the state trie is downloaded at a random block close to the chain head, the failure of which is considered a potential attack, resulting in the disabling of fast sync and restarting with slow.

However, most of the time it's just a connectivity issue, maybe bad peers, maybe local connectivity problem, which can result in a very unpleasant sync restart. Still, the downloader has no way to decide why peers dropped off, only that noone was able to serve the promised data, which could mean a bad actor at play.

---

This PR works around this issue not by making phase 2 more resilient to failure, but rather by shortening phase two to becoming almost instantaneous. It does this by starting a concurrent head (!!!) state trie download already in phase one - where errors are permitted - so that when the sync algo reached phase 2, almost the entire state is downloaded (head) and only the difference between the ~1500 head blocks needs to be pulled (currently about 23K states), versus the entire state (currently about 1.3M). This makes phase two ~0.17% of it's original length, considerably shorter for connectivity issues to wreck havoc.

Please note, the reason we pull the head state and not the pivot state is because we need the pivot to remain random and the head is deterministic and fixed either way, so there's no harm in downloading it.

This does pose a possibility where an attacker might try to send us garbage data while we're pulling the blockchain (as we can't verify the root hash without all the headers leading up to it), but when we do finish downloading the headers, we just notice it's junk and pull the correct state. Given that the most an attacker can achieve this way is to waste a bit of space for newly (and only newly) joining nodes, which should be cleaned up by state pruning, there's not much to gain. Also this attack can only be done in a targeted way and cannot really be magnified by the network. Thus given that the worst attack vector is annoying someone, this change should be safe to include.

---

Further on the positive side, by pulling most of the state during phase one already:

• We don't need to be aggressive in dropping useless nodes (from a state perspective) as we have enough time to pull the states, there's no significant hurry. This resulting in a most stable network.
• Newly joining nodes which just began to sync can already help each other out as they can pull states from one another too and not have to rely on fully synced large nodes for this data.
  • Mental note: maybe we should do state sync randomly, not in order. This could allow a more distributed availability of data. Though this should be done in another PR :)

[robotally]

Vote	Count	Reviewers
👍	1	@zsfelfoldi
👎	0

Updated: Mon May 30 20:33:37 UTC 2016

[karalabe]

@obscuren @zsfelfoldi Please review :)

[karalabe]

Just a note: syncStatsStateTotal was an unused variable that we can just drop off and syncStatsStateDone is better not to be reset so that pulling states in multiple cycles (i.e. reset after some failure) does not look as if it's restarting (causing people to freak out) but rather explicitly display that it's continuing.

[fjl]

I have tried this PR two times now and it seems to work. Code looks also OK. 👍

[zsfelfoldi]

LGTM too 👍