Fix for the pruning proof rebuild issue (issue #444)

[michaelsutton]

The following describes the pruning bug and the fix assuming sufficient prior knowledge.

Some notation:

• High levels: block levels at which there are less than 2000 blocks since genesis (so genesis is part of that proof level). Currently this is approx around level 39 and above.
• Low levels: block levels close to current difficulty and hence it is likely that all 2000 proof blocks are within the last day before the corresponding pruning point.
• Intermediate levels: block levels which are not that rare but are still unlikely to have more than 2000 blocks all within a single day. Hence their level proof likely has blocks both before and after the previous pruning point.

The bug is mostly in the case of intermediate levels. It requires that such a block is mined in the anticone of the pruning point (so the bug is likely to occur ~once in a few days, and requires that someone synced a new node exactly at that day).

The logic missing: we did not update relations per level for the pruning point anticone (aka "trusted" blocks).

Scenario: a "lucky" intermediate-level block R happened to be in the pruning point anticone. A future block B on that level is likely to have R as its only parent (for that level). Since level relations are not updated for R, B computes Ghostdag for that level over ORIGIN -- essentially starting from blue score 0 instead of building over the previous 2000 blocks. When building the new proof, this segment of blocks (S = B + its future) is topologically disconnected from block N which is the block at depth 1000 of the next level (which is likely below the current new pruning point). So N becomes root, and the traversal starting from it never reaches S. Leading to a possible prune of blocks in S (some may be preserved via other levels)

Result: following the prune, the proof rebuild fails bcs of blocks missing from S. This is only a symptom of the bug, the real problem is that the level proof is literally incomplete.

Fix: update relations per level also for trusted blocks

[tiram88]

I reviewed the last 3 commits.

My knowledge of the pruning system is not deep enough to bring a valuable review on this part of the PR.

[coderofstuff]

The fix is mainly the change in header_processor. There are 3 other sections unrelated to the pruning proof bug. Approving for these sections

• pruning proof bug fix
• kip9 change
• enable-mainnet-mining change

Please address Tiram's comments for the UTXO tracking section.