test(rpc): Add Rust tests for lightwalletd sync from Zebra

[teor2345]

Motivation

We want to run full sync and quick sync integration tests for lightwalletd.

We want to merge PR #4068 first, to unblock some other work.

Depends-On: #4068

CI Commands

You'll need to update the cache paths to match the CI paths.

To test full sync:

export ZEBRA_CACHED_STATE_PATH="$HOME/.cache/zebra"
export RUST_LOG="info"

cargo test -- --ignored --nocapture  lightwalletd_full_sync

To test quick update sync:

export ZEBRA_CACHED_STATE_PATH="$HOME/.cache/zebra"
export LIGHTWALLETD_DATA_DIR="$HOME/.cache/lightwalletd"
export RUST_LOG="info"

cargo test -- --ignored --nocapture  lightwalletd_update_sync

Manual Testing

To manually test all the lightwalletd integration tests in series (which avoids state conflicts), use:

export ZEBRA_CACHED_STATE_PATH="$HOME/.cache/zebra"
export LIGHTWALLETD_DATA_DIR="$HOME/.cache/lightwalletd"
export RUST_LOG="info"

cargo test -- --ignored --nocapture  lightwalletd_test_suite

Solution

• Create a lightwalletd full sync test
• Create a lightwalletd cached state update test

Details:

• Create a LightwalletdTestType enum
• Make the lightwalletd integration test take a test type
• Configure lightwalletd tests based on the test type
• Add checks for each LightwalletdTestType
• Create test functions that run each lightwalletd test type
• Fail if the state doesn't match the test type

Related Cleanups:

• Remove obsolete kill_on_error() in the lightwalletd test
• Add a sub-directory for lightwalletd test state
• Move failure messages into their own module

Closes #3511
Closes #4166

Review

@jvff is probably be the best person to review this PR.

@gustavovalverde this PR should work in CI if you use the commands above.
(It's draft because I haven't finished testing it yet.)

Reviewer Checklist

• Code implements Specs and Designs
• Tests for Expected Behaviour
• Tests for Errors

Follow Up Work

• ci: run a full lightwalletd sync from a close-to-tip zebrad #3926
• Store lightwalletd state after lightwalletd full sync test #4165
• ci: call lightwalletd quick tip sync test  #4167

[teor2345]

@Mergifyio update

[mergify]

update

✅ Branch has been successfully updated

[jvff]

Looks great! There's just one minor tweak, to avoid having duplicate environment variables with different names after resolving any merge conflicts.

[teor2345]

When rebasing on main, I removed a bunch of duplicate code.

The LightwalletdTestType enum names are still a bit messy, but I'd like to fix that later.

[teor2345]

Failed due to #4234:

Error: buildx failed with: error: failed to solve: failed to fetch oauth token: unexpected status: 401 Unauthorized

[oxarbitrage]

I think it is ok to merge this now as it will conflict with some other code i am working on. The comment from @jvff was addressed and it also looks good to me.