Syncer: don't fetch 900 blocks each time

[arajasek]

As noted in the comment here, we always fetch 900 blocks from our peers when asked to syncFork. This is wasteful, and can cause us to fall out of sync / pressure our peers.

This isn't trivial to resolve, however. I propose the following optimizations:

• check if incoming is already built on a chain we have synced. If so, fetch nothing.
  • Note that this isn't easy to do, and must be reasoned about carefully. In particular, simply finding incoming.Parents() in your store is NOT sufficient evidence that we have the entire chain.
• fetch blocks with some exponential increase
  • a good "starting" number of blocks would be incoming.Height() - known.Height() + 10
  • when those are fetched, check if we have enough to find our common ancestor (if so, we're done)
  • if not, double the amount and fetch again (max of 900)

[Stebalien]

In particular, simply finding incoming.Parents() in your store is NOT sufficient evidence that we have the entire chain.

Could you expand on this? I thought we relied on this guarantee.

fetch blocks with some exponential increase

This sounds like the right approach. Honestly, we can probably start at 2 epochs, most forks are very short.

[arajasek]

In particular, simply finding incoming.Parents() in your store is NOT sufficient evidence that we have the entire chain.

Could you expand on this? I thought we relied on this guarantee.

I think it's likely the case that you have the entire chain, but not necessarily so. Say that you're at height 100, and were "stuck" for some reason for 10 epochs. When you return, one peer gossips the tipset at height 109, while another gossips height 110. In this case, when processing 110 as incoming, we WILL have its parents, but not 101-108.

fetch blocks with some exponential increase

This sounds like the right approach. Honestly, we can probably start at 2 epochs, most forks are very short.

I agree. I'm going to start by optimizing what appear to be nearly 100% of calls to syncFork, which is the 1 epoch case. In this case you actually don't need to fetch either zero blocks (if you already have all the parents, which is 90% of the time, based on analysis), or fetch one tipset (incoming.Parents()).

[Stebalien]

I think it's likely the case that you have the entire chain, but not necessarily so. Say that you're at height 100, and were "stuck" for some reason for 10 epochs. When you return, one peer gossips the tipset at height 109, while another gossips height 110. In this case, when processing 110 as incoming, we WILL have its parents, but not 101-108.

Do we not have a rule where we won't write tipset 108 to disk until we have tipset 109 written?