feat(state): add transparent address indexes to the non-finalized state

[teor2345]

Motivation

As part of the address index RPCs, we need to track address balances, UTXOs, and transactions in the non-finalized state.

This PR creates the address indexes in the non-finalized state, but it doesn't connect them to state requests or RPCs yet.

Specifications

See tickets:

• getaddresstxids JSON-RPC method #3147
• getaddressbalance JSON-RPC method #3157
• getaddressutxos JSON-RPC method #3158

Designs

The TransparentTransfers struct holds the address indexes for a single address.

Index updates use UpdateWith<T> for TransparentTransfers, where T is a tuple of arguments for:

• created UTXOs, or
• spent UTXOs

Spent UTXOs are a bit tricky, because we need to add their values to the ContextuallyValidBlock.

Solution

• Create a TransparentTransfers address index type
• Implement balance, UTXO, and transaction ID updates for created UTXOs
• Implement balance, UTXO, and transaction ID updates for spent UTXOs
• Remove empty TransparentTransfers after a spend revert
• Add address transfers index accessor methods, which return UTXOs and transactions in chain order
• Make accessor methods provide all the information required for each query

Related changes:

• Add spent UTXO values to ContextuallyValidBlock
• Use OrderedUtxos so that spent UTXOs can be stored in chain order
• Switch to height-based TransactionLocations as part of merging PR 3. change(db): Store UTXOs by transaction location rather than transaction hash #3978

Related bug fixes:

• Stop indexing each transparent output multiple times

Previously, the chain code was indexing each transparent output in the block multiple times (once per transaction). Because we were using HashMaps for chain indexes, this bug did not cause any problems. But in the address indexes, it made balances overflow.

(This should also improve non-finalized state performance for large blocks.)

Review

@conradoplg reviewed a draft of this PR.

We still need to connect these indexes to state requests, but the changes in this PR are enough to get us started.

Reviewer Checklist

• Code implements Specs and Designs
• Tests for Expected Behaviour
• Tests for Errors

Follow Up Work

In other PRs:

• Implement the address RPCs and state requests
  • Add unit tests
  • Add RPC snapshot tests

[teor2345]

There's something wrong with the balance calculations in this PR:

The application panicked (crashed).
Message: called Result::unwrap() on an Err value: Constraint { value: 2100025173264129, range: -2100000000000000..=2100000000000000 }
Location: zebra-state/src/service/non_finalized_state/chain/index.rs:85

[conradoplg]

I keep reviewing this PR but I can't seem to find the bug 😞 (or any other issue). I'll leave it for someone else to review.

[teor2345]

I keep reviewing this PR but I can't seem to find the bug 😞 (or any other issue). I'll leave it for someone else to review.

I think it's probably a mismatch between the created and spent UTXOs. Probably some missing spent UTXOs, because it's an overflow.

I'll make this part of the code easier to get right.

[teor2345]

@Mergifyio update

[mergify]

update

✅ Branch has been successfully updated

[teor2345]

@Mergifyio update

[mergify]

update

✅ Branch has been successfully updated

[oxarbitrage]

Re-approving.