Murmel BIP-157/158 Gap Analysis

[notmandatory]

This is an overview issue for discussion about how to make murmel a fully functional BIP-157/158 client. I'd like to make PRs or additional issues for each missing feature once we have consensus on the right approach.

The below BIP-157/158 protocol features are based on the BIP-157 Client Operation section. Please suggest additions and corrections in the comments.

Missing protocol features for Murmel are indicated below with an unchecked box [ ], completed features with a checked box [x].

Terminology

• peer: bitcoin full node peer
• block header: bitcoin standard block header
• filter header: BIP-157 compact block filter header
• block filter: BIP-157/158 compact block filter

Peer Discovery

1. • Allow user to specify peer addresses
2. • Receive peer addresses via [addr] messages from connected peers
3. • Store new peer addresses
4. • If no provided or previously stored peers, find seed peer addresses via DNS

Connect to Peers

1. • Allow user to specify number of peers to connect to
2. • Allow user to specify minimum number of BIP-157 peers to connect to
3. • Connect to peers and request signalling bits via [version] message
4. • Keep track of peers signalling service bit NODE_COMPACT_FILTERS
5. • Connect to specified number of peers
6. • Connect to minimum number of BIP-157 peers

Request Block Headers

1. • Request block headers via [getheaders] message to connected peers
2. • Request block headers from genesis to most work block, 1 to 2000 blocks per request
3. • Optional: Request block headers from stored checkpoint block instead of genesis block

Receive Block Headers

1. • Receive block headers [headers] from peers
2. • Build chain of block headers from stored and received new block headers
3. • Store new block headers and new block header tip
4. • Disconnect peers whos longest chain has significantly less work than known tip
5. • Request block filter headers from all peers for any new block header received

Request Filter Headers

1. • Allow user to specify start block height for block filter sync
2. • Request filter headers via [getcfheaders] message to peers
3. • Request from start block height to end block hash (1-1000 per request)
4. • Request filter headers for the same block from more than one peer
5. • Optional: First request filter headers at intervals of 1000 via [getcfcheckpt] message

Receive Filter Headers

1. • Receive filter headers [cfheaders] from peers
2. • Verify filter headers from multiple peers are the same
3. • If any received filter headers for a block are different,
   1. • request full block and compute correct block filter and filter header
   2. • ban peers that provided incorrect filter header
4. • Build chain of filter headers from stored and received new filter headers
5. • Store verified filter headers, at least to depth of 100
6. • Request block filter for any new verified filter header
7. • Optional: store more than 100 verified filter headers

Request Block Filters

1. • Request block filters after all filter headers are received and verified
2. • Request block filters from start block via [getcfilters] message to peers
3. • Request from start block height to end block hash (1-1000 per request)
4. • Optional: Check if filter header is for empty block filter if so skip request
5. • Optional: Request different block filters in parallel from multiple peers

Receive Block Filters

1. • Receive block filter [cfilter] from peers
2. • Verify block filters sequentially, must link to corresponding filter header
3. • Ban any peer that sends an invalid block filter
4. • Check block filters for relevance
   1. • Previous output scripts for transaction inputs
   2. • ScriptPubKey for transaction outputs

Fetch Full Blocks

1. • If fetching full blocks, download from random peers
2. • Fetch full blocks from peers even if they don’t support compact block filters
3. • Fetch full blocks that correspond to relevant block filters

Other Required Enhancements

1. • Make syscall free similar to LDK
   1. • Traits for storage of peers, block headers, filter headers, block filters, blocks
   2. • Traits for network socket IO
2. • Use latest rust async/.await
   1. • Refactor and remove existing futures alpha lib
   2. • Create tests and example cli based on tokio futures executor lib