Draft design for lightwalletd database support

[teor2345]

Motivation

To support lightwalletd, Zebra needs to:

• add a transparent output address index, and
• make accessing individual transactions more efficient.

This design change helps us estimate those changes, and open tickets.

This is unexpected work in Sprint 24.

Specifications

List of lightwalletd RPC methods:

• https://github.com/zcash/lightwalletd/blob/master/README.md#zcashd

Impacted RPCs:

• https://zcash.github.io/rpc/getrawtransaction.html
• https://zcash.github.io/rpc/getaddresstxids.html

Solution

• Split transaction data from block data in the on-disk database
• Add transparent address indexes to the on-disk database
• Support transparent address balances and address UTXOs
• Make keys smaller, and make them use chain order, to improve database performance

Review

Anyone can review this database design change.

Reviewer Checklist

• Design makes sense
• RPCs could be implemented efficiently using this design

Follow Up Work

• open tickets for each change in this design

[dconnolly]

👍

[codecov]

Codecov Report

Merging #3137 (a6b3ef1) into main (31b083f) will increase coverage by 0.73%.
The diff coverage is n/a.

❗ Current head a6b3ef1 differs from pull request most recent head 2d8cac2. Consider uploading reports for the commit 2d8cac2 to get more accurate results

@@            Coverage Diff             @@
##             main    #3137      +/-   ##
==========================================
+ Coverage   77.03%   77.77%   +0.73%     
==========================================
  Files         265      264       -1     
  Lines       31267    31013     -254     
==========================================
+ Hits        24088    24121      +33     
+ Misses       7179     6892     -287