Tracking: Document Zcash consensus rules in Zebra codebase

[mpguerra]

Motivation

We want to ensure that all of the Zcash consensus rules are implemented in Zebra. In order to help us do this, we should document where the Zcash consensus rules are implemented in the Zebra codebase.

This will also help when auditing Zebra.

Tasks for each consensus rule:

• check that the rule is implemented in Zebra
• make sure the latest version of the rule is quoted in the docs for the code that implements it
• make sure the code implements the latest version of the rule

Consensus Rules found in the Zcash Protocol Specification

• Document consensus rules from Zcash spec: Section 3.4 Transactions and Treestates #3209
• Document consensus rules from Zcash spec: Section 3.5 JoinSplit Transfers and Descriptions #3210
• Document consensus rules from Zcash spec: Section 3.6 "Spend Transfers, Output Transfers, and their Descriptions" #3211
• Document consensus rules from Zcash spec: Section 3.7 Action Transfers and their Descriptions #3212
• Document consensus rules from Zcash spec: Section 3.8 Note Commitment Trees #3213
• Document consensus rules from Zcash spec: Section 3.9 Nullifier Sets #3214
• Document consensus rules from Zcash spec: Section 4.3 JoinSplit Descriptions #3215
• Document consensus rules from Zcash spec: 4.4 Spend Descriptions #3216
• Document consensus rules from Zcash spec: 4.5 Output Descriptions #3217
• Document consensus rules from Zcash spec: 4.6 Action Descriptions #3218
• Document consensus rules from Zcash spec: 4.12 Balance (Sprout) #3219
• Document consensus rules from Zcash spec: 4.13 Balance and Binding Signature (Sapling) #3220
• Document consensus rules from Zcash spec: 4.14 Balance and Binding Signature (Orchard) #3221
• Document types specified in 7.1 Transaction Encoding and Consensus as part of the consensus rules #3222
  • Document types specified in 7.1: Transaction Header #3420
  • Document types specified in 7.1: Transparent #3421
  • Document types specified in 7.1: Sapling #3422
  • Document types specified in 7.1: Sprout #3423
  • Document types specified in 7.1: Orchard #3424
• Document consensus rules from Zcash spec: 7.1.2 Transaction Consensus Rules #3223
  • Document consensus rules from Zcash spec 7.1.2: Transaction Header #3425
  • Document consensus rules from Zcash spec 7.1.2: Transaction Size #3426
  • Document consensus rules from Zcash spec 7.1.2: Coinbase transactions #3427
  • Document consensus rules from Zcash spec 7.1.2: Shielded pools #3428
• Document consensus rules from Zcash spec: 7.3 Spend Description Encoding and Consensus #3224
• Document consensus rules from Zcash spec: 7.4 Output Description Encoding and Consensus #3225
• Document consensus rules from Zcash spec: 7.5 Action Description Encoding and Consensus #3226
• Document part of consensus rules from Zcash spec: 7.6 Block Header Encoding and Consensus #3227
• Document remaining consensus rules from Zcash spec: 7.6 Block Header Encoding and Consensus #3297
• Document 7.9 Payment of Founders’ Reward consensus rule (Not implemented as we checkpoint on Canopy)
• Document consensus rules from Zcash spec: 7.10 Payment of Funding Streams #3228

Consensus ZIPs

• NU5
  • TODO: list other ZIPs
  • Document ZIP 244: Transaction Identifier Non-Malleability, blocked by:
    • ZIP 244 - Changes to transparent component of signature digest
    • Add a new v5 transaction script verification API to zcash_script and Zebra #3291
• NU4: Canopy
  • Document ZIP 207: Funding Streams
  • Document ZIP 211: Disabling Addition of New Value to the Sprout Value Pool
  • Document ZIP 212: Allow Recipient to Derive Sapling Ephemeral Secret from Note Plaintext
  • Document ZIP 214: Consensus rules for a Zcash Development Fund
  • Document ZIP 215: Explicitly Defining and Modifying Ed25519 Validation Rules
  • Document ZIP 251: Deployment of the Canopy Network Upgrade
• NU3: Heartwood
  • Document ZIP-221: Flyclient
  • Document ZIP-213: Shielded Coinbase
  • Document ZIP 250: Deployment of the Heartwood Network Upgrade
• NU2: Blossom
  • Document ZIP-208: Shorter Block Target Spacing
  • Document ZIP-206: Deployment of the Blossom Network Upgrade
• NU1: Sapling
  • Document ZIP-243: Transaction Signature Validation for Sapling
  • Document ZIP 205: Deployment of the Sapling Network Upgrade
• NU0: Overwinter
  • ZIP-143: Transaction Signature Validation for Overwinter - obsoleted by ZIP-243 (Sapling - Canopy) and ZIP-244 (NU5)
  • ZIP-200: Network Upgrade Mechanism
  • ZIP-201: Network Peer Management for Overwinter
  • ZIP-202: Version 3 Transaction Format for Overwinter
  • ZIP-203: Transaction Expiry
• Sprout
  • ZIP-209: Prohibit Negative Shielded Chain Value Pool Balances - done in Document chain value balances consensus rules with new format #3286
  • [ ]

Related Work

[conradoplg]

Thank you @mpguerra for listing those!

How should we carry out this work?

I think the hardest part is identifying each rule. My suggestion is coming up with a way to create an identifier for each one, and them mention each of them in the code where they are implemented. Then we could maybe create a script or just grep the code to list them all, and compare the list to the full list of rules to make sure we didn't forget anything. Eventually we could reach out to ECC to maybe include the identifier in the spec. As long as we have an identifier for each one of them, it's easy to rename them later if needed.

Does this make sense, or is it too much work? Without this it will be really hard to find where each rule is implemented, and to determine if we missed any rule.

My suggestion would look like:

• Decide on a standard for the rule identifiers (e.g. something like rule-7.3-anchor-sapling-less-qj)
• Go over the spec and fill an e.g. spreadsheet with the (sub)section, the identifier, and the text of the rule (this can probably be automated somehow, since consensus rules are marked with a macro in the latex source)
• Decide on how to mention the identifier in the code to make it easy to find
• For each rule, find where it's implemented and add the mention to the rule
• List all identifiers in the code, compare to the rule list, make sure we didn't forget anything

@dconnolly @teor2345 @upbqdn what do you think?
I feel like @daira and @str4d probably have thought about this in the past and may already have something in mind 😆 In fact I just saw zcash/zips#468 and zcash/zcash#3957. Anything that we should keep in mind when considering this?

[upbqdn]

I like the general idea of having the consensus rules mapped in a one-to-one fashion between the code and the spec by leveraging unique identifiers. I think it might be helpful even if one consensus rule is scattered around multiple places in the codebase.

To me, the consensus rules resemble math definitions, so having a naming system that would allow referring to each rule by its name would be convenient.

[teor2345]

I think the hardest part is identifying each rule. My suggestion is coming up with a way to create an identifier for each one

I don't think we should create our own identifiers, because we would have to maintain them separately with every spec change. But if the zcashd developers can commit to maintaining the identifiers in the spec, in the timeframe we need, we could use those.

Otherwise linking to the sub-section is good enough for now.

[teor2345]

This is done!