Generate stateful disks containing lightwalletd-supporting zebra-state cache

[dconnolly]

Motivation

We currently regenerate disk images with zebra-state synced up to Sapling activation for use in our sync_past_mandatory_checkpoint test, so that we don't have to spend hours syncing zebrad to exercise the relevant code paths of the test. We should do similar for our lightwalletd tests, any integration test that requires synced, processed data to be available to a zebra-state/zebrad instance/test harness, so we don't need to spend extra time syncing.

Design

Since this is for general testing and not a particular NU activation height, we should aim for something closer to the most recent NU activation on mainnet.

When syncing, ensure that all build features required for lightwalletd support are enabled, and that the config for zebrad has all the lightwalletd functionality enabled to populate the state with the tx index, the compactblock cache, etc.

This should run automatically. Currently we regenerate the Sapling checkpoint test state on changes to zebra-state format, but that may not be the entire set of changes to trigger on to kick off a new state disk regeneration.

https://docs.google.com/document/d/1HrDs0eWZs3s8C5rpwCR5-obCO0nNOXtlG5w4jXD9MJM/edit

Tasks

- [ ] Create a test similar to sync_to_mandatory_checkpoint_* that syncs to a blockheight (NU5 activation?) and stops This exists as the full sync test to tip.

• Create a new / updated GitHub Actions workflow to generate the stateful disk in a similar manner as https://github.com/ZcashFoundation/zebra/blob/main/.github/workflows/regenerate-stateful-test-disks.yml, using the above test, with the appropriate features, config files, etc
• Make sure the triggers for the workflow fire on changes to state format, lightwalletd support tweaks, etc

Follow-Up Tasks

• Add lightwalletd full sync integration test #3511
• Create an integration test where we make requests to lightwalletd backed by zebrad #3510

[dconnolly]

Hey team! Please add your planning poker estimate with ZenHub @conradoplg @gustavovalverde @jvff @oxarbitrage @teor2345 @upbqdn

[teor2345]

Does lightwalletd wait until it's reached the tip to use some RPCs?
Do we want to always sync to tip for the lightwalletd cached state?