Implement SPV with minimal storage requirements and fast block headers sync time

[shamardy]

This is important to enable spv in mobile and web DEXes. Ref. #1471 (comment), #1339 (comment)

[borngraced]

This Pull Request introduces new storage and bandwidth enhancements for SPV in mm2 by allowing custom starting block height and a max number of block headers to keep in storage for verifying transactions.

starting_block_header

Where to start block headers sync from. For BTC and similar coins, this must be a retarget header.

Params - this parameters can be fetched from any block explorer for the coin, but it's good practice to get it from a trusted source

height - block height
hash - block header hash
bits - block header bits
time - time the block was mined in epoch

max_stored_block_headers

Max number of block headers to keep in storage for verification
Notes:
1- For BTC and similar coins, this must be greater than the retarget interval (2016 blocks for BTC).
2- Transactions can’t be verified against a block header if it’s deleted from storage, so this value needs to be big enough to make sure that swaps/lightning transactions can be verified.

Example configuration for spv_conf for BTC with starting_block_header 500, validation_params and without max_stored_block_headers.

{
	"coin": "BTC",
	"asset":"BTC",
	"pubtype": 0,
	"p2shtype": 5,
	"wiftype": 128,
	"segwit": true,
	"bech32_hrp": "bc",
	"txfee": 0,
	"estimate_fee_mode": "ECONOMICAL",
	"required_confirmations": 0,
	"protocol": {
    	"type": "UTXO"
	},
	"spv_conf": {
    	"starting_block_header": {
        	"height": 2016,
        	"hash": "00000000a141216a896c54f211301c436e557a8d55900637bbdce14c6c7bddef",
        	"bits": 486604799,
        	"time": 1233063531,
    	},
    	"validation_params": {
        	"difficulty_check": true,
        	"constant_difficulty": false,
        	"difficulty_algorithm": "Bitcoin Mainnet"
    	}
	}
}

Example configuration for spv_conf for RICK coin with max_stored_block_headers and without validation_params

  "coin": "RICK",
  "asset": "RICK",
  "fname": "RICK (TESTCOIN)",
  "rpcport": 25435,
  "txversion": 4,
  "overwintered": 1,
  "mm2": 1,
  "enable_spv_proof": true,
  "required_confirmations": 1,
  "requires_notarization": false,
  "avg_blocktime": 1,
  "protocol": {
	"type": "UTXO"
  },
"spv_conf": {
   "starting_block_header": {
	"height": 1,
	"hash": "0c714ba4f8d5f2d5c014a08c4e21a5387156e23bcc819c0f9bc536437586cdf5",
	"time": 1564482125,
	"bits": 537857807
},
"max_stored_block_headers": 15
}}

cc. @shamardy

[borngraced]

cc. @smk762

[smk762]

Will this require updating the coins file with static values for all UTXO coins?

[shamardy]

Will this require updating the coins file with static values for all UTXO coins?

Last I checked, only BTC has SPV enabled in coins file. So it will require changing BTC config only.

[shamardy]

Example configuration for spv_conf for BTC with starting_block_header 500, validation_params and without max_stored_block_headers.

@borngraced can u give an example for the genesis block header as the starting_block_header. I believe this is what will be used in config at this stage, we will use a more recent block header when #1644 is merged since we will be able to finally use BTC spv in WebDEX and mobile.

[borngraced]

Example configuration for spv_conf for BTC with starting_block_header 500, validation_params and without max_stored_block_headers.

@borngraced can u give an example for the genesis block header as the starting_block_header. I believe this is what will be used in config at this stage, we will use a more recent block header when #1644 is merged since we will be able to finally use BTC spv in WebDEX and mobile.

ok.

spv_conf with genesis as starting block header

 "spv_conf": {
            "starting_block_header": {
                "height": 0,
                "hash": "000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f",
                "bits": 486604799,
                "time": 1231006505
            },
            "validation_params": {
                "difficulty_check": true,
                "constant_difficulty": false,
                "difficulty_algorithm": "Bitcoin Mainnet"
            }
        }

[shamardy]

I believe this has been implemented, additionally, there is a global issue for tracking SPV implementation #1612 . Closing this one.