p2p changes

[dergoegge]

Service bit

A node should set the NODE_UTREEXO service bit to announce its ability to serve or receive proofs. Bits 24- 31 are reserved for experiments so we should choose one of those for now.

Inventory vector types

https://en.bitcoin.it/wiki/Protocol_documentation#Inventory_Vectors
Add two new types: MSG_BLOCK_UTREEXO, MSG_TX_UTREEXO

Messages

There are multiple options when it comes to how we could add proofs to the protocol.

1. Add a new message (e.g.: proof) that only contains proof data for blocks or transactions. This is not the best idea since transactions and blocks don't make sense to a compact node without proof, so it makes more sense to fuse the two into one message. Requesting individual proofs could still be useful in addition to (2) or (3). A compact node could request proof for a block or transaction that was received by a non-Utreexo node from a Utreexo peer. This could help bridges remain undetected because they would otherwise be the only Utreexo nodes that request regular blocks and transactions.

2. Duplicate existing messages related to block and transaction relay. (e.g.: block -> ublock) The duplicated messages are the same as there original besides the command string and the appended proofs. This has the benefit that non-Utreexo nodes would ignore these messages if they were send to them by accident. Messages that would have to be duplicated: getblocks, block, tx, sendcmpct, cmpctblock, getblocktxn, blocktxn. I personally am a bit on the fence about this approach because a combination of (3) and the added inventory types should be sufficient to avoid sending proofs to non-Utreexo nodes and requires no new messages.

3. Don't add any new messages and append the proofs at the end of messages (block, tx, blocktxn) depending on the type of node that is being served. If a NODE_UTREEXO node receives a message from anotherNODE_UTREEXO node the receiver should expect the message to contain proofs. If a message is send by a NODE_UTREEXO node to another NODE_UTREEXO node the sender should append proofs. This approach should also take the added inventory types into account, inv(MSG_*_UTREEXO) or getdata(MSG_*_UTREEXO) should only be answered with proof appended messages.

We should discuss which of these approaches we want to go with (Maybe there are more approaches?). utcd currently implements (2) although so far only getblocks and block have been duplicated.

[kcalvinalvin]

For the service bit, let's just send out a message on the dev mailing list like the comment suggests when we're ready to actually set up nodes. How about just using bit 24? If there are other experiments using it, we can increment by +1 until we get to a bit that's not being used.

Inventory vectors look good.

For messages, instead of making separate wire messages, how about making a separate encoding like UtreexoEncoding. Segwit is sorta similar in that it requires changes to all those messages. I think it'd be cleaner overall to implement it like how segwit did it. I guess this would be along the lines of option 3.

We can have the UtreexoEncoding require appending the utreexo proof for the given message. For block, you'd append the BatchProof for the entire block. for tx, you'd also append BatchProof but the proof being only for that one tx. I could be missing something but for compact blocks, you'd only need to append proofs for blocktxn right? Have to think about how compact blocks would work..

[dergoegge]

Yeah with compact blocks we only need one batch proof for the transactions in the blocktxn message, so in most cases it should be smaller than the proof for the entire block.

I was also thinking that if a node requests multiple transactions at once (getdata(txid_1, txid_2, ...)) then we could send one batch proof for all their inputs. That would probably need a new message since there is no message that sends multiple transactions (besides blocktxn but that is specific to compact blocks).

(Package relay and Erlay could also benefit from batch proofs for multiple transactions but these are still proposals so no need to think about that now.)

There are probably a hole bunch of ways to reduce bandwidth usage for the proofs during transaction relay.

[kcalvinalvin]

Yeah with compact blocks we only need one batch proof for the transactions in the blocktxn message, so in most cases it should be smaller than the proof for the entire block.

Ok yeah just calculate a batchproof on the fly. Sounds good.

I was also thinking that if a node requests multiple transactions at once (getdata(txid_1, txid_2, ...)) then we could send one batch proof for all their inputs. That would probably need a new message since there is no message that sends multiple transactions (besides blocktxn but that is specific to compact blocks).

This one might be a bit tricky but definitely worth looking into once we have the basic stuff working.

Since compact blocks aren't implemented in btcd, I'll just be getting to messages block and tx. Are we ok with going forward with the UtreexoEncoding method?