Rollback for older node versions that have followed a legacy chain

[teor2345]

Motivation

If an old version of Zebra isn't upgraded before a network upgrade, we need to rollback or delete the database.

Solution

Let's do this work in a few separate parts.

• Mandatory checkpoint refactor Mandatory checkpoint refactor #2305
• Acceptance test refactor Acceptance test refactor #2306
• Move Block in State Checks to Downloaders Move Block in State Checks to Downloaders #2307
• Legacy Chain Check Legacy Chain Check  #2308

Alternatives

We don't need to do the legacy chain check until after the highest checkpoint in Zebra's checkpoint list. (Regardless of Zebra's checkpoint config.) But it's easier to test if we do it as early as possible.

We could:

• automatically delete the database (like Detect when an older state version is present, and delete it #1213)
• drop existing Zebra services and create new ones
• automatically shutdown and restart any running Zebra services
• exec zebrad using Unix or Windows process APIs

This might not be possible on Windows, it could leave some resources open, and we'd need to guard against multiple restarts.

Related Work

This change is part of ZIP-200 and #337.

These tickets might also be related:

• Design and implement graceful shutdown for Zebra Design and implement graceful shutdown for Zebra #1678
• Interrupt handler does not work when a blocking task is running Interrupt handler does not work when a blocking task is running #1351

[teor2345]

Re-adding this to NU5, because the longer NU5 timelines make it more likely that outdated deployed Zebra nodes will pass NU5 activation, but not have the activation heights set.

[teor2345]

This was closed by ticket #2308 and PR #2366.