From 4cba267e98b48cce9e1d23760b25db282e517935 Mon Sep 17 00:00:00 2001 From: Alfredo Garcia Date: Thu, 11 Mar 2021 14:11:42 -0300 Subject: [PATCH 01/36] propose a v5 transaction rfc --- book/src/dev/rfcs/XXXX-v5-transaction.md | 211 +++++++++++++++++++++++ 1 file changed, 211 insertions(+) create mode 100644 book/src/dev/rfcs/XXXX-v5-transaction.md diff --git a/book/src/dev/rfcs/XXXX-v5-transaction.md b/book/src/dev/rfcs/XXXX-v5-transaction.md new file mode 100644 index 00000000000..9243a839f1c --- /dev/null +++ b/book/src/dev/rfcs/XXXX-v5-transaction.md @@ -0,0 +1,211 @@ +- Feature Name: `v5_transaction` +- Start Date: 2021-03-11 +- Design PR: [ZcashFoundation/zebra#0000](https://github.com/ZcashFoundation/zebra/pull/0000) +- Zebra Issue: [ZcashFoundation/zebra#1863](https://github.com/ZcashFoundation/zebra/issues/1863) + +# Summary +[summary]: #summary + +Network Upgrade number 5(`NU5`) introduces a new transaction type(transaction version 5). This document is a proposed design for implementing such a transaction version. + +# Motivation +[motivation]: #motivation + +The Zebra software wants to be a protocol compatible Zcash implementation One of the tasks to do this includes the support of the new version 5 transactions that will be implemented in the next network upgrade. + +# Definitions +[definitions]: #definitions + +- `NU5` +- `Orchard` +- `Sapling` +- `orchard data` - Data types needed to support orchard transactions. +- `saplig data` - Data types needed to support sapling transactions. +- `orchard transaction` - Transactions that suppoort orchard data. Currently only V5. +- `sapling transaction` - Transactions that support sapling data. Currently V4 and V5 but the data is implemented differently in them. + +# Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + +V5 transactions are described by the protocol in the second table of [Transaction Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#txnencodingandconsensus). + +We need the raw representation to Serialize/Deserialize the transaction however Zebra uses its own representation internally. + +V5 transactions are the only ones that will support orchard transactions with `Orchard` data types. + +V4 and V5 transactions both support sapling data but they are implemented differently. By this reason we need to split sapling data types into V4 and V5. + +The order of some of the fields changed from V4 to V5. For example the `lock_time` and `expiry_height` were moved above the transparent inputs and outputs. + +All of the changes proposed in this document are only to the `zebra-chain` crate. + +To highlight changes all document comments from the code snippets in the [reference section](#reference-level-explanation) were removed. + +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +We start by looking how a V4(already implemented) transaction is represented in Zebra. + +This transaction version is specified by the protocol in the first table of [Transaction Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#txnencodingandconsensus). + + +``` +V4 { + inputs: Vec, + outputs: Vec, + lock_time: LockTime, + expiry_height: block::Height, + value_balance: Amount, + joinsplit_data: Option>, + shielded_data: Option, +} +``` + +Currently the `ShieldedData` type is defined in `zebra-chain/src/transaction/shielded_data.rs` as follows: + +``` +pub struct ShieldedData { + pub first: Either, + pub rest_spends: Vec, + pub rest_outputs: Vec, + pub binding_sig: Signature, +} +``` + +We know by protocol(2nd table of [Transaction Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#txnencodingandconsensus)) that V5 transactions will support sapling data however we also know by protocol that spends([Spend Description Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#spendencodingandconsensus), See †) and outputs([Output Description Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#outputencodingandconsensus), See †) fields change from V4 to V5. + +Here we have the proposed changes for V4 transactions: + +``` +V4 { + inputs: Vec, + outputs: Vec, + lock_time: LockTime, + expiry_height: block::Height, + sapling_value_balance: Amount, + joinsplit_data: Option>, + shielded_data: Option, +} +``` + +The only change above from the current implementation is the `shielded_data` field type, changed from `Option` to `Option`. + +`ShieldedData` is currently defined and implemented in `src/transaction/shielded_data.rs`. We propose to change `ShieldedData` into `SaplingV4ShieldedData` and to add `SaplingV5ShieldedData` into the same file. + +The difference between V4 and V5 shielded data is in the spends and outputs. For this reason we need to add V4 and V5 spend and output versions to our code: + +``` +pub struct SaplingV4ShieldedData { + pub first: Either, + pub rest_spends: Vec, + pub rest_outputs: Vec, + pub binding_sig: Signature, +} +``` + +Proposed `SpendV4` and `OutputV4` is now described, this is the same as the current implementation but just changing the struct names. +Sapling spend code is located at `zebra-chain/src/sapling/spend.rs`: + +``` +pub struct SpendV4 { + pub cv: commitment::ValueCommitment, + pub anchor: tree::Root, + pub nullifier: note::Nullifier, + pub rk: redjubjub::VerificationKeyBytes, + pub zkproof: Groth16Proof, + pub spend_auth_sig: redjubjub::Signature, +} +``` + +The output code is located at `zebra-chain/src/sapling/output.rs`: + +``` +pub struct OutputV4 { + pub cv: commitment::ValueCommitment, + pub cm_u: jubjub::Fq, + pub ephemeral_key: keys::EphemeralPublicKey, + pub enc_ciphertext: note::EncryptedNote, + pub out_ciphertext: note::WrappedNoteKey, + pub zkproof: Groth16Proof, +} +``` + +Now lets see how the v5 transaction is specified in the protocol, this is the second table of [Transaction Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#txnencodingandconsensus). + +We propose the following representation for transaction V5 in Zebra: + +``` +V5 { + lock_time: LockTime, + expiry_height: block::Height, + inputs: Vec, + outputs: Vec, + shielded_data: Option, + value_balance_sapling: Amount, + output_proofs_sapling: Vec + anchor_sapling: tree::Root, + spend_zkproof_sapling: Vec, + spend_auth_sigs_sapling: Vec> + orchard_data: Option, + flags_orchard: u8, + value_balance_orchard: Amount, + anchor_orchard: tree::Root, +} +``` + +`SaplingV5ShieldedData` is different from `SaplingV4ShieldedData` so a new type will be defined and implemented in `zebra-chain/src/transaction/shielded_data.rs`. +Definition will look as follows: + +``` +pub struct SaplingV5ShieldedData { + pub first: Either, + pub rest_spends: Vec, + pub rest_outputs: Vec, + pub binding_sig: Signature, +} +``` + +This leads to new sapling output and spend types. + +V5 sapling spend will be defined and implemented in `zebra-chain/src/sapling/spend.rs`(this is in the same file as `SpendV4`): +``` +pub struct SpendV5 { + pub cv: commitment::ValueCommitment, + pub nullifier: note::Nullifier, + pub rk: redjubjub::VerificationKeyBytes, +} +``` + +V5 sapling output will live in `zebra-chain/src/sapling/output.rs`(this is in the same file as `OutputV4`): + +``` +pub struct OutputV5 { + pub cv: commitment::ValueCommitment, + pub cm_u: jubjub::Fq, + pub ephemeral_key: keys::EphemeralPublicKey, + pub enc_ciphertext: note::EncryptedNote, + pub out_ciphertext: note::WrappedNoteKey, +} +``` + +Finally, the new V5 structure will create a new `OrchardData` type. This new type will be defined in a separated file: `zebra-chain/src/transaction/orchard_data.rs` and it will look as follows: +``` +pub struct OrchardData { + pub cv: commitment::ValueCommitment, + pub nullifier: ..., + pub rk: ..., + pub cmx: ..., + pub ephemeral_key: keys::EphemeralPublicKey, + pub enc_ciphertext: note::EncryptedNote, + pub out_ciphertext: note::WrappedNoteKey, +} +``` + +## Test Plan +[test-plan]: #test-plan + +- All renamed, modified and new types should serialize and deserialize. +- The full V4 and V5 transactions should serialize and deserialize. +- Prop test strategies for v4 and v5 will be updated and created. + + From c304245b891f63b371b456199fe394666e82fa48 Mon Sep 17 00:00:00 2001 From: Alfredo Garcia Date: Thu, 11 Mar 2021 14:16:58 -0300 Subject: [PATCH 02/36] add rfc number, add design pr --- .../dev/rfcs/{XXXX-v5-transaction.md => 0010-v5-transaction.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename book/src/dev/rfcs/{XXXX-v5-transaction.md => 0010-v5-transaction.md} (98%) diff --git a/book/src/dev/rfcs/XXXX-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md similarity index 98% rename from book/src/dev/rfcs/XXXX-v5-transaction.md rename to book/src/dev/rfcs/0010-v5-transaction.md index 9243a839f1c..7ad84527adb 100644 --- a/book/src/dev/rfcs/XXXX-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -1,6 +1,6 @@ - Feature Name: `v5_transaction` - Start Date: 2021-03-11 -- Design PR: [ZcashFoundation/zebra#0000](https://github.com/ZcashFoundation/zebra/pull/0000) +- Design PR: [ZcashFoundation/zebra#1886](https://github.com/ZcashFoundation/zebra/pull/1886) - Zebra Issue: [ZcashFoundation/zebra#1863](https://github.com/ZcashFoundation/zebra/issues/1863) # Summary From f456251e6d98f7abcb5f8532fd0121b06a518fde Mon Sep 17 00:00:00 2001 From: Alfredo Garcia Date: Thu, 11 Mar 2021 20:08:36 -0300 Subject: [PATCH 03/36] Apply suggestions from code review Co-authored-by: teor --- book/src/dev/rfcs/0010-v5-transaction.md | 53 ++++++++++++------------ 1 file changed, 27 insertions(+), 26 deletions(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index 7ad84527adb..5ed675f6b63 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -6,12 +6,12 @@ # Summary [summary]: #summary -Network Upgrade number 5(`NU5`) introduces a new transaction type(transaction version 5). This document is a proposed design for implementing such a transaction version. +Network Upgrade number 5 (`NU5`) introduces a new transaction type (transaction version 5). This document is a proposed design for implementing such a transaction version. # Motivation [motivation]: #motivation -The Zebra software wants to be a protocol compatible Zcash implementation One of the tasks to do this includes the support of the new version 5 transactions that will be implemented in the next network upgrade. +The Zebra software wants to be a protocol compatible Zcash implementation. One of the tasks to do this includes the support of the new version 5 transactions that will be implemented in the next network upgrade. # Definitions [definitions]: #definitions @@ -20,9 +20,9 @@ The Zebra software wants to be a protocol compatible Zcash implementation One of - `Orchard` - `Sapling` - `orchard data` - Data types needed to support orchard transactions. -- `saplig data` - Data types needed to support sapling transactions. -- `orchard transaction` - Transactions that suppoort orchard data. Currently only V5. -- `sapling transaction` - Transactions that support sapling data. Currently V4 and V5 but the data is implemented differently in them. +- `sapling data` - Data types needed to support sapling transactions. +- `orchard transaction version` - Transactions that support orchard data. Currently only V5. +- `sapling transaction version` - Transactions that support sapling data. Currently V4 and V5 but the data is implemented differently in them. # Guide-level explanation [guide-level-explanation]: #guide-level-explanation @@ -44,7 +44,7 @@ To highlight changes all document comments from the code snippets in the [refere # Reference-level explanation [reference-level-explanation]: #reference-level-explanation -We start by looking how a V4(already implemented) transaction is represented in Zebra. +We start by looking how a V4 (already implemented) transaction is represented in Zebra. This transaction version is specified by the protocol in the first table of [Transaction Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#txnencodingandconsensus). @@ -72,19 +72,16 @@ pub struct ShieldedData { } ``` -We know by protocol(2nd table of [Transaction Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#txnencodingandconsensus)) that V5 transactions will support sapling data however we also know by protocol that spends([Spend Description Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#spendencodingandconsensus), See †) and outputs([Output Description Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#outputencodingandconsensus), See †) fields change from V4 to V5. +We know by protocol (2nd table of [Transaction Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#txnencodingandconsensus)) that V5 transactions will support sapling data however we also know by protocol that spends ([Spend Description Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#spendencodingandconsensus), See †) and outputs ([Output Description Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#outputencodingandconsensus), See †) fields change from V4 to V5. Here we have the proposed changes for V4 transactions: ``` V4 { - inputs: Vec, - outputs: Vec, - lock_time: LockTime, - expiry_height: block::Height, + ... sapling_value_balance: Amount, - joinsplit_data: Option>, - shielded_data: Option, + ... + sapling_shielded_data: Option, } ``` @@ -96,6 +93,11 @@ The difference between V4 and V5 shielded data is in the spends and outputs. For ``` pub struct SaplingV4ShieldedData { + /// Either a spend or output description. + /// + /// Storing this separately ensures that it is impossible to construct + /// an invalid `ShieldedData` with no spends or outputs. + /// ... pub first: Either, pub rest_spends: Vec, pub rest_outputs: Vec, @@ -140,16 +142,13 @@ V5 { expiry_height: block::Height, inputs: Vec, outputs: Vec, - shielded_data: Option, - value_balance_sapling: Amount, - output_proofs_sapling: Vec - anchor_sapling: tree::Root, - spend_zkproof_sapling: Vec, - spend_auth_sigs_sapling: Vec> - orchard_data: Option, - flags_orchard: u8, - value_balance_orchard: Amount, - anchor_orchard: tree::Root, + sapling_value_balance: Amount, + sapling_anchor: tree::Root, + sapling_shielded_data: Option, + orchard_flags: OrchardFlags, + orchard_value_balance: Amount, + orchard_anchor: tree::Root, + orchard_shielded_data: Option, } ``` @@ -167,16 +166,18 @@ pub struct SaplingV5ShieldedData { This leads to new sapling output and spend types. -V5 sapling spend will be defined and implemented in `zebra-chain/src/sapling/spend.rs`(this is in the same file as `SpendV4`): +V5 sapling spend will be defined and implemented in `zebra-chain/src/sapling/spend.rs` (this is in the same file as `SpendV4`): ``` pub struct SpendV5 { pub cv: commitment::ValueCommitment, pub nullifier: note::Nullifier, pub rk: redjubjub::VerificationKeyBytes, + pub zkproof: Groth16Proof, + pub spend_auth_sig: redjubjub::Signature, } ``` -V5 sapling output will live in `zebra-chain/src/sapling/output.rs`(this is in the same file as `OutputV4`): +V5 sapling output will live in `zebra-chain/src/sapling/output.rs` (this is in the same file as `OutputV4`): ``` pub struct OutputV5 { @@ -185,6 +186,7 @@ pub struct OutputV5 { pub ephemeral_key: keys::EphemeralPublicKey, pub enc_ciphertext: note::EncryptedNote, pub out_ciphertext: note::WrappedNoteKey, + pub zkproof: Groth16Proof, } ``` @@ -208,4 +210,3 @@ pub struct OrchardData { - The full V4 and V5 transactions should serialize and deserialize. - Prop test strategies for v4 and v5 will be updated and created. - From 91e33789bda0d333e188dda8e6ade7a9680485cf Mon Sep 17 00:00:00 2001 From: Alfredo Garcia Date: Sat, 13 Mar 2021 08:37:50 -0300 Subject: [PATCH 04/36] define orchard flags --- book/src/dev/rfcs/0010-v5-transaction.md | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index 5ed675f6b63..d1cc9815bf8 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -190,7 +190,7 @@ pub struct OutputV5 { } ``` -Finally, the new V5 structure will create a new `OrchardData` type. This new type will be defined in a separated file: `zebra-chain/src/transaction/orchard_data.rs` and it will look as follows: +The new V5 structure will create a new `OrchardData` type. This new type will be defined in a separated file: `zebra-chain/src/transaction/orchard_data.rs` and it will look as follows: ``` pub struct OrchardData { pub cv: commitment::ValueCommitment, @@ -203,6 +203,18 @@ pub struct OrchardData { } ``` +Also in the V5 transaction we have a new `OrchardFlags`. This is a bitfield type defined as: + +``` +bitflags! { + pub struct OrchardFlags: u8 { + const ENABLE_SPENDS = 0; + const ENABLE_OUTPUTS = 1; + // Reserved, zeros (bits 2 .. 7) + } +} +``` + ## Test Plan [test-plan]: #test-plan From 717f40f74434fee0ddda9e2ddb4652e4772b5de4 Mon Sep 17 00:00:00 2001 From: Alfredo Garcia Date: Sat, 13 Mar 2021 09:18:56 -0300 Subject: [PATCH 05/36] change some names for consistency --- book/src/dev/rfcs/0010-v5-transaction.md | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index d1cc9815bf8..0d556f35b4c 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -81,18 +81,16 @@ V4 { ... sapling_value_balance: Amount, ... - sapling_shielded_data: Option, + sapling_shielded_data: Option, } ``` -The only change above from the current implementation is the `shielded_data` field type, changed from `Option` to `Option`. - -`ShieldedData` is currently defined and implemented in `src/transaction/shielded_data.rs`. We propose to change `ShieldedData` into `SaplingV4ShieldedData` and to add `SaplingV5ShieldedData` into the same file. +`ShieldedData` is currently defined and implemented in `src/transaction/shielded_data.rs`. We propose to change `ShieldedData` into `SaplingShieldedDataV4` and to add `SaplingShieldedDataV5` into the same file. The difference between V4 and V5 shielded data is in the spends and outputs. For this reason we need to add V4 and V5 spend and output versions to our code: ``` -pub struct SaplingV4ShieldedData { +pub struct SaplingShieldedDataV4 { /// Either a spend or output description. /// /// Storing this separately ensures that it is impossible to construct @@ -144,7 +142,7 @@ V5 { outputs: Vec, sapling_value_balance: Amount, sapling_anchor: tree::Root, - sapling_shielded_data: Option, + sapling_shielded_data: Option, orchard_flags: OrchardFlags, orchard_value_balance: Amount, orchard_anchor: tree::Root, @@ -152,11 +150,11 @@ V5 { } ``` -`SaplingV5ShieldedData` is different from `SaplingV4ShieldedData` so a new type will be defined and implemented in `zebra-chain/src/transaction/shielded_data.rs`. +`SaplingShieldedDataV5` is different from `SaplingShieldedDataV4` so a new type will be defined and implemented in `zebra-chain/src/transaction/shielded_data.rs`. Definition will look as follows: ``` -pub struct SaplingV5ShieldedData { +pub struct SaplingShieldedDataV5 { pub first: Either, pub rest_spends: Vec, pub rest_outputs: Vec, @@ -196,7 +194,7 @@ pub struct OrchardData { pub cv: commitment::ValueCommitment, pub nullifier: ..., pub rk: ..., - pub cmx: ..., + pub cm_x: ..., pub ephemeral_key: keys::EphemeralPublicKey, pub enc_ciphertext: note::EncryptedNote, pub out_ciphertext: note::WrappedNoteKey, From 239a8dff51d265729388adf3115e8ef230da1bf6 Mon Sep 17 00:00:00 2001 From: Alfredo Garcia Date: Sat, 13 Mar 2021 10:15:06 -0300 Subject: [PATCH 06/36] update orchard stuff --- book/src/dev/rfcs/0010-v5-transaction.md | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index 0d556f35b4c..38a333a5987 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -188,20 +188,18 @@ pub struct OutputV5 { } ``` -The new V5 structure will create a new `OrchardData` type. This new type will be defined in a separated file: `zebra-chain/src/transaction/orchard_data.rs` and it will look as follows: +The new V5 structure will create a new `OrchardShieldedData` type. This new type will be defined in a separated file: `zebra-chain/src/transaction/orchard_shielded_data.rs` and it will look as follows: ``` -pub struct OrchardData { - pub cv: commitment::ValueCommitment, - pub nullifier: ..., - pub rk: ..., - pub cm_x: ..., - pub ephemeral_key: keys::EphemeralPublicKey, - pub enc_ciphertext: note::EncryptedNote, - pub out_ciphertext: note::WrappedNoteKey, +pub struct OrchardShieldedData { + pub first: Action, + pub rest_spends: Vec, + pub binding_sig: Signature, } ``` -Also in the V5 transaction we have a new `OrchardFlags`. This is a bitfield type defined as: +Where `Action` is defined as [Action definition](https://github.com/ZcashFoundation/zebra/blob/68c12d045b63ed49dd1963dd2dc22eb991f3998c/zebra-chain/src/orchard/action.rs#L18-L41). + +Finally, in the V5 transaction we have a new `OrchardFlags`. This is a bitfield type defined as: ``` bitflags! { From e0501761bed954a22a8a0f91e59b24d3c079a308 Mon Sep 17 00:00:00 2001 From: Alfredo Garcia Date: Mon, 15 Mar 2021 09:23:44 -0300 Subject: [PATCH 07/36] Apply suggestions from code review Co-authored-by: teor --- book/src/dev/rfcs/0010-v5-transaction.md | 25 +++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index 38a333a5987..0b5f59b72ad 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -16,9 +16,9 @@ The Zebra software wants to be a protocol compatible Zcash implementation. One o # Definitions [definitions]: #definitions -- `NU5` -- `Orchard` -- `Sapling` +- `NU5` - the 5th Zcash network upgrade, counting from the `Overwinter` upgrade as upgrade zero. +- `Orchard` - a new shielded pool introduced in `NU5`. +- `Sapling` - a new shielded pool introduced in the 1st network upgrade. (`Sapling` is also the name of that network upgrade, but this RFC is focused on the `Sapling` shielded pool.) - `orchard data` - Data types needed to support orchard transactions. - `sapling data` - Data types needed to support sapling transactions. - `orchard transaction version` - Transactions that support orchard data. Currently only V5. @@ -191,8 +191,12 @@ pub struct OutputV5 { The new V5 structure will create a new `OrchardShieldedData` type. This new type will be defined in a separated file: `zebra-chain/src/transaction/orchard_shielded_data.rs` and it will look as follows: ``` pub struct OrchardShieldedData { + /// An action description. + /// + /// Storing this separately ensures that it is impossible to construct + /// an invalid `OrchardShieldedData` with no actions. pub first: Action, - pub rest_spends: Vec, + pub rest: Vec, pub binding_sig: Signature, } ``` @@ -204,8 +208,8 @@ Finally, in the V5 transaction we have a new `OrchardFlags`. This is a bitfield ``` bitflags! { pub struct OrchardFlags: u8 { - const ENABLE_SPENDS = 0; - const ENABLE_OUTPUTS = 1; + const ENABLE_SPENDS = 0b00000001; + const ENABLE_OUTPUTS = 0b00000010; // Reserved, zeros (bits 2 .. 7) } } @@ -217,4 +221,11 @@ bitflags! { - All renamed, modified and new types should serialize and deserialize. - The full V4 and V5 transactions should serialize and deserialize. - Prop test strategies for v4 and v5 will be updated and created. - +- Before NU5 activation on testnet, test on the following test vectors: + - Hand-crafted Orchard, Orchard/Sapling, Orchard/Sprout, and Orchard/Sapling/Sprout transactions based on the spec + - Converted Sapling and Sapling/Sprout transactions in the existing test vectors from `V4` to `V5` format + - Any available `zcashd` test vectors +- After NU5 activation on testnet: + - Add test vectors using the testnet activation block and 2 more post-activation blocks +- After NU5 activation on mainnet: + - Add test vectors using the mainnet activation block and 2 more post-activation blocks From aee84278e0ba7530f9d95a6aaf8d4e5a144ec615 Mon Sep 17 00:00:00 2001 From: Alfredo Garcia Date: Mon, 15 Mar 2021 09:43:18 -0300 Subject: [PATCH 08/36] rename file --- book/src/dev/rfcs/0010-v5-transaction.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index 0b5f59b72ad..e98dcefd9f4 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -85,7 +85,7 @@ V4 { } ``` -`ShieldedData` is currently defined and implemented in `src/transaction/shielded_data.rs`. We propose to change `ShieldedData` into `SaplingShieldedDataV4` and to add `SaplingShieldedDataV5` into the same file. +`ShieldedData` is currently defined and implemented in `src/transaction/shielded_data.rs`. We propose to rename this file to `src/transaction/sapling_shielded_data.rs`. Inside we will change `ShieldedData` into `SaplingShieldedDataV4` and add the new version `SaplingShieldedDataV5`. The difference between V4 and V5 shielded data is in the spends and outputs. For this reason we need to add V4 and V5 spend and output versions to our code: @@ -150,7 +150,7 @@ V5 { } ``` -`SaplingShieldedDataV5` is different from `SaplingShieldedDataV4` so a new type will be defined and implemented in `zebra-chain/src/transaction/shielded_data.rs`. +`SaplingShieldedDataV5` is different from `SaplingShieldedDataV4` so a new type will be defined and implemented in `zebra-chain/src/transaction/sapling_shielded_data.rs`. Definition will look as follows: ``` From 84cfc444336b1ee7435a83c26b1342043f0ad99f Mon Sep 17 00:00:00 2001 From: Alfredo Garcia Date: Mon, 15 Mar 2021 19:43:36 -0300 Subject: [PATCH 09/36] Apply suggestions from code review Co-authored-by: teor --- book/src/dev/rfcs/0010-v5-transaction.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index e98dcefd9f4..edee192c86c 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -175,7 +175,9 @@ pub struct SpendV5 { } ``` -V5 sapling output will live in `zebra-chain/src/sapling/output.rs` (this is in the same file as `OutputV4`): +V5 sapling output will live in `zebra-chain/src/sapling/output.rs` (this is in the same file as `OutputV4`). + +Note: the V4 and V5 sapling outputs currently have identical fields. We use different types to consistently distinguish V4 and V5 transaction data. ``` pub struct OutputV5 { @@ -187,7 +189,11 @@ pub struct OutputV5 { pub zkproof: Groth16Proof, } ``` +To abstract over the `V4` and `V5` `Transaction` enum variants, we will implement `Transaction` methods to access sapling and orchard data. + +To abstract over the `SaplingShieldedDataV4`/`SaplingShieldedDataV5`, `SpendV4`/`SpendV5`, and `OutputV4`/`OutputV5` structs, we will implement `SaplingShieldedData`, `Spend`, and `Output` traits to access sapling data. +The methods in these traits will be implemented in a similar way to the existing `Transaction` methods. The new V5 structure will create a new `OrchardShieldedData` type. This new type will be defined in a separated file: `zebra-chain/src/transaction/orchard_shielded_data.rs` and it will look as follows: ``` pub struct OrchardShieldedData { From f14ac65e76f04b87ec4e7590899c50b58ee554e5 Mon Sep 17 00:00:00 2001 From: Alfredo Garcia Date: Wed, 17 Mar 2021 11:29:03 -0300 Subject: [PATCH 10/36] Update test sections Co-authored-by: teor --- book/src/dev/rfcs/0010-v5-transaction.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index edee192c86c..1f5746ed357 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -228,8 +228,11 @@ bitflags! { - The full V4 and V5 transactions should serialize and deserialize. - Prop test strategies for v4 and v5 will be updated and created. - Before NU5 activation on testnet, test on the following test vectors: - - Hand-crafted Orchard, Orchard/Sapling, Orchard/Sprout, and Orchard/Sapling/Sprout transactions based on the spec - - Converted Sapling and Sapling/Sprout transactions in the existing test vectors from `V4` to `V5` format + - Hand-crafted Orchard-only, Orchard/Sapling, Orchard/Transparent, and Orchard/Sapling/Transparent transactions based on the spec + - "Fake" Sapling-only and Sapling/Transparent transactions based on the existing test vectors, converted from `V4` to `V5` format + - We can write a test utility function to automatically do these conversions + - An empty transaction, with no Orchard, Sapling, or Transparent data + - Any available `zcashd` test vectors - After NU5 activation on testnet: - Add test vectors using the testnet activation block and 2 more post-activation blocks From 95cc691a73ca03e13bd0a334b0afe8228adb4cdd Mon Sep 17 00:00:00 2001 From: Alfredo Garcia Date: Thu, 18 Mar 2021 09:41:34 -0300 Subject: [PATCH 11/36] Add security section Co-authored-by: teor --- book/src/dev/rfcs/0010-v5-transaction.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index 1f5746ed357..363966d2e16 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -221,6 +221,17 @@ bitflags! { } ``` +## Security + +To avoid parsing memory exhaustion attacks, we will make the following changes across all `Transaction`, `ShieldedData`, `Spend` and `Output` variants, `V1` through to `V5`: +- `Box` large `Optional` data structures, to defer allocation until the variant or option is actually used +- Statically assert that these data structures remain as small as possible +- Check cardinality consensus rules at parse time, before deserializing any `Vec`s + - In general, Zcash requires that each transaction has at least one Transparent/Sprout/Sapling/Orchard transfer, this rule is not currently encoded in our data structures +- Stop parsing as soon as the first error is detected + +These changes should be made in a later pull request, see [#1917](https://github.com/ZcashFoundation/zebra/issues/1917) for details. + ## Test Plan [test-plan]: #test-plan From 447403cbd6109a5f0b125fe5b20e33eab376749a Mon Sep 17 00:00:00 2001 From: Alfredo Garcia Date: Thu, 18 Mar 2021 15:50:19 -0300 Subject: [PATCH 12/36] change design to use enums --- book/src/dev/rfcs/0010-v5-transaction.md | 123 +++++++++-------------- 1 file changed, 48 insertions(+), 75 deletions(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index 363966d2e16..1d13d96a158 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -33,7 +33,7 @@ We need the raw representation to Serialize/Deserialize the transaction however V5 transactions are the only ones that will support orchard transactions with `Orchard` data types. -V4 and V5 transactions both support sapling data but they are implemented differently. By this reason we need to split sapling data types into V4 and V5. +V4 and V5 transactions both support sapling data but they are implemented differently. By this reason we need to split sapling data types into V4 and V5. We do this by using `enum` data types. The order of some of the fields changed from V4 to V5. For example the `lock_time` and `expiry_height` were moved above the transparent inputs and outputs. @@ -81,46 +81,63 @@ V4 { ... sapling_value_balance: Amount, ... - sapling_shielded_data: Option, + sapling_shielded_data: Option, } ``` -`ShieldedData` is currently defined and implemented in `src/transaction/shielded_data.rs`. We propose to rename this file to `src/transaction/sapling_shielded_data.rs`. Inside we will change `ShieldedData` into `SaplingShieldedDataV4` and add the new version `SaplingShieldedDataV5`. +In order to mantain the order of the fields that we will need for Serialization/Deserialization we keep the `value_balance` in the transaction for V4 but we rename it to `sapling_value_balance` for clarity. -The difference between V4 and V5 shielded data is in the spends and outputs. For this reason we need to add V4 and V5 spend and output versions to our code: +`ShieldedData` is currently defined and implemented in `src/transaction/shielded_data.rs`. We propose to rename this file to `src/transaction/sapling_shielded_data.rs`. Inside we will change `ShieldedData` into a `SaplingShieldedData` enum with `V4` and `V5` variants. ``` -pub struct SaplingShieldedDataV4 { +pub enum SaplingShieldedData { /// Either a spend or output description. /// /// Storing this separately ensures that it is impossible to construct - /// an invalid `ShieldedData` with no spends or outputs. + /// an invalid `SaplingShieldedData` with no spends or outputs. /// ... - pub first: Either, - pub rest_spends: Vec, - pub rest_outputs: Vec, - pub binding_sig: Signature, + V4 { + pub first: Either, + pub rest_spends: Vec, + pub rest_outputs: Vec, + pub binding_sig: Signature, + }, + V5 { + pub first: Either, + pub rest_spends: Vec, + pub rest_outputs: Vec, + pub sapling_value_balance: Amount, + pub anchor: tree::Root, + pub binding_sig: Signature, + } } ``` -Proposed `SpendV4` and `OutputV4` is now described, this is the same as the current implementation but just changing the struct names. -Sapling spend code is located at `zebra-chain/src/sapling/spend.rs`: - -``` -pub struct SpendV4 { - pub cv: commitment::ValueCommitment, - pub anchor: tree::Root, - pub nullifier: note::Nullifier, - pub rk: redjubjub::VerificationKeyBytes, - pub zkproof: Groth16Proof, - pub spend_auth_sig: redjubjub::Signature, +Proposed `SaplingSpend` is now defined as an enum with `V4` and `V5` variants. Sapling spend code is located at `zebra-chain/src/sapling/spend.rs`. Notable difference here is that the `anchor` in `V4` is needed for every spend of the transaction while in `V5` the anchor is a single one defined in `SaplingShieldedData`: + +``` +pub enum SaplingSpend { + V4 { + pub cv: commitment::ValueCommitment, + pub anchor: tree::Root, + pub nullifier: note::Nullifier, + pub rk: redjubjub::VerificationKeyBytes, + pub zkproof: Groth16Proof, + pub spend_auth_sig: redjubjub::Signature, + }, + V5 { + pub cv: commitment::ValueCommitment, + pub nullifier: note::Nullifier, + pub rk: redjubjub::VerificationKeyBytes, + pub zkproof: Groth16Proof, + pub spend_auth_sig: redjubjub::Signature, + } } ``` - -The output code is located at `zebra-chain/src/sapling/output.rs`: +In Zebra the output representations are the same for V4 and V5 so no variants are needed. The output code is located at `zebra-chain/src/sapling/output.rs`: ``` -pub struct OutputV4 { +pub struct SaplingOutput { pub cv: commitment::ValueCommitment, pub cm_u: jubjub::Fq, pub ephemeral_key: keys::EphemeralPublicKey, @@ -130,7 +147,7 @@ pub struct OutputV4 { } ``` -Now lets see how the v5 transaction is specified in the protocol, this is the second table of [Transaction Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#txnencodingandconsensus). +Now lets see how the V5 transaction is specified in the protocol, this is the second table of [Transaction Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#txnencodingandconsensus) and how are we going to represent it based in the above changes for Sapling fields and the new Orchard fields. We propose the following representation for transaction V5 in Zebra: @@ -140,60 +157,13 @@ V5 { expiry_height: block::Height, inputs: Vec, outputs: Vec, - sapling_value_balance: Amount, - sapling_anchor: tree::Root, - sapling_shielded_data: Option, - orchard_flags: OrchardFlags, - orchard_value_balance: Amount, - orchard_anchor: tree::Root, + sapling_shielded_data: Option, orchard_shielded_data: Option, } ``` -`SaplingShieldedDataV5` is different from `SaplingShieldedDataV4` so a new type will be defined and implemented in `zebra-chain/src/transaction/sapling_shielded_data.rs`. -Definition will look as follows: +`sapling_shielded_data` will now use `SaplingShieldedData::V5` variant located at `zebra-chain/src/transaction/sapling_shielded_data.rs` with the corresponding `SaplingSpend::V5` for the spends. -``` -pub struct SaplingShieldedDataV5 { - pub first: Either, - pub rest_spends: Vec, - pub rest_outputs: Vec, - pub binding_sig: Signature, -} -``` - -This leads to new sapling output and spend types. - -V5 sapling spend will be defined and implemented in `zebra-chain/src/sapling/spend.rs` (this is in the same file as `SpendV4`): -``` -pub struct SpendV5 { - pub cv: commitment::ValueCommitment, - pub nullifier: note::Nullifier, - pub rk: redjubjub::VerificationKeyBytes, - pub zkproof: Groth16Proof, - pub spend_auth_sig: redjubjub::Signature, -} -``` - -V5 sapling output will live in `zebra-chain/src/sapling/output.rs` (this is in the same file as `OutputV4`). - -Note: the V4 and V5 sapling outputs currently have identical fields. We use different types to consistently distinguish V4 and V5 transaction data. - -``` -pub struct OutputV5 { - pub cv: commitment::ValueCommitment, - pub cm_u: jubjub::Fq, - pub ephemeral_key: keys::EphemeralPublicKey, - pub enc_ciphertext: note::EncryptedNote, - pub out_ciphertext: note::WrappedNoteKey, - pub zkproof: Groth16Proof, -} -``` -To abstract over the `V4` and `V5` `Transaction` enum variants, we will implement `Transaction` methods to access sapling and orchard data. - -To abstract over the `SaplingShieldedDataV4`/`SaplingShieldedDataV5`, `SpendV4`/`SpendV5`, and `OutputV4`/`OutputV5` structs, we will implement `SaplingShieldedData`, `Spend`, and `Output` traits to access sapling data. - -The methods in these traits will be implemented in a similar way to the existing `Transaction` methods. The new V5 structure will create a new `OrchardShieldedData` type. This new type will be defined in a separated file: `zebra-chain/src/transaction/orchard_shielded_data.rs` and it will look as follows: ``` pub struct OrchardShieldedData { @@ -203,7 +173,10 @@ pub struct OrchardShieldedData { /// an invalid `OrchardShieldedData` with no actions. pub first: Action, pub rest: Vec, - pub binding_sig: Signature, + pub flags: OrchardFlags, + pub orchard_value_balance: Amount, + pub anchor: tree::Root, + pub binding_sig: redpallas::Signature, } ``` From efcc65b89e2ff704b88a4acb974d246e678156bc Mon Sep 17 00:00:00 2001 From: Alfredo Garcia Date: Thu, 18 Mar 2021 18:54:00 -0300 Subject: [PATCH 13/36] move some code into sapling and orchard crates, do renames --- book/src/dev/rfcs/0010-v5-transaction.md | 26 ++++++++++++------------ 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index 1d13d96a158..da81e26402d 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -87,7 +87,7 @@ V4 { In order to mantain the order of the fields that we will need for Serialization/Deserialization we keep the `value_balance` in the transaction for V4 but we rename it to `sapling_value_balance` for clarity. -`ShieldedData` is currently defined and implemented in `src/transaction/shielded_data.rs`. We propose to rename this file to `src/transaction/sapling_shielded_data.rs`. Inside we will change `ShieldedData` into a `SaplingShieldedData` enum with `V4` and `V5` variants. +`ShieldedData` is currently defined and implemented in `zebra-chain/src/transaction/shielded_data.rs`. As this is Sapling specific we propose to move this file to `zebra-chain/src/sapling/shielded_data.rs`. Inside we will change `ShieldedData` into `SaplingShieldedData` and implement it as an enum with `V4` and `V5` variants. ``` pub enum SaplingShieldedData { @@ -97,15 +97,15 @@ pub enum SaplingShieldedData { /// an invalid `SaplingShieldedData` with no spends or outputs. /// ... V4 { - pub first: Either, - pub rest_spends: Vec, - pub rest_outputs: Vec, + pub first: Either, + pub rest_spends: Vec, + pub rest_outputs: Vec, pub binding_sig: Signature, }, V5 { - pub first: Either, - pub rest_spends: Vec, - pub rest_outputs: Vec, + pub first: Either, + pub rest_spends: Vec, + pub rest_outputs: Vec, pub sapling_value_balance: Amount, pub anchor: tree::Root, pub binding_sig: Signature, @@ -113,10 +113,10 @@ pub enum SaplingShieldedData { } ``` -Proposed `SaplingSpend` is now defined as an enum with `V4` and `V5` variants. Sapling spend code is located at `zebra-chain/src/sapling/spend.rs`. Notable difference here is that the `anchor` in `V4` is needed for every spend of the transaction while in `V5` the anchor is a single one defined in `SaplingShieldedData`: +Proposed `Spend` is now defined as an enum with `V4` and `V5` variants. Sapling spend code is located at `zebra-chain/src/sapling/spend.rs`. Notable difference here is that the `anchor` in `V4` is needed for every spend of the transaction while in `V5` the anchor is a single one defined in `SaplingShieldedData`: ``` -pub enum SaplingSpend { +pub enum Spend { V4 { pub cv: commitment::ValueCommitment, pub anchor: tree::Root, @@ -137,7 +137,7 @@ pub enum SaplingSpend { In Zebra the output representations are the same for V4 and V5 so no variants are needed. The output code is located at `zebra-chain/src/sapling/output.rs`: ``` -pub struct SaplingOutput { +pub struct Output { pub cv: commitment::ValueCommitment, pub cm_u: jubjub::Fq, pub ephemeral_key: keys::EphemeralPublicKey, @@ -162,9 +162,9 @@ V5 { } ``` -`sapling_shielded_data` will now use `SaplingShieldedData::V5` variant located at `zebra-chain/src/transaction/sapling_shielded_data.rs` with the corresponding `SaplingSpend::V5` for the spends. +`sapling_shielded_data` will now use `SaplingShieldedData::V5` variant located at `zebra-chain/src/transaction/sapling_shielded_data.rs` with the corresponding `Spend::V5` for the spends. -The new V5 structure will create a new `OrchardShieldedData` type. This new type will be defined in a separated file: `zebra-chain/src/transaction/orchard_shielded_data.rs` and it will look as follows: +The new V5 structure will create a new `OrchardShieldedData` type. This new type will be defined in a separated file: `zebra-chain/src/orchard/shielded_data.rs` and it will look as follows: ``` pub struct OrchardShieldedData { /// An action description. @@ -182,7 +182,7 @@ pub struct OrchardShieldedData { Where `Action` is defined as [Action definition](https://github.com/ZcashFoundation/zebra/blob/68c12d045b63ed49dd1963dd2dc22eb991f3998c/zebra-chain/src/orchard/action.rs#L18-L41). -Finally, in the V5 transaction we have a new `OrchardFlags`. This is a bitfield type defined as: +Finally, in the V5 transaction we have a new `OrchardFlags` type. This is a bitfield type defined as: ``` bitflags! { From 066ae597b4ae17cba4b2b6f9a75f6e79a1478d5d Mon Sep 17 00:00:00 2001 From: teor Date: Fri, 19 Mar 2021 15:42:51 +1000 Subject: [PATCH 14/36] Move a comment --- book/src/dev/rfcs/0010-v5-transaction.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index da81e26402d..db61f24ff37 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -91,12 +91,12 @@ In order to mantain the order of the fields that we will need for Serialization/ ``` pub enum SaplingShieldedData { - /// Either a spend or output description. - /// - /// Storing this separately ensures that it is impossible to construct - /// an invalid `SaplingShieldedData` with no spends or outputs. - /// ... V4 { + /// Either a spend or output description. + /// + /// Storing this separately ensures that it is impossible to construct + /// an invalid `SaplingShieldedData` with no spends or outputs. + /// ... pub first: Either, pub rest_spends: Vec, pub rest_outputs: Vec, From 356881aa97e2944ba9ffdbedde38867b9364bd3a Mon Sep 17 00:00:00 2001 From: teor Date: Fri, 19 Mar 2021 15:43:10 +1000 Subject: [PATCH 15/36] Delete unnecessary security fixes --- book/src/dev/rfcs/0010-v5-transaction.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index db61f24ff37..1a86e372c88 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -197,8 +197,6 @@ bitflags! { ## Security To avoid parsing memory exhaustion attacks, we will make the following changes across all `Transaction`, `ShieldedData`, `Spend` and `Output` variants, `V1` through to `V5`: -- `Box` large `Optional` data structures, to defer allocation until the variant or option is actually used -- Statically assert that these data structures remain as small as possible - Check cardinality consensus rules at parse time, before deserializing any `Vec`s - In general, Zcash requires that each transaction has at least one Transparent/Sprout/Sapling/Orchard transfer, this rule is not currently encoded in our data structures - Stop parsing as soon as the first error is detected From dbcf972b0f0e5417f929162366ecf19c6385ebf5 Mon Sep 17 00:00:00 2001 From: teor Date: Fri, 19 Mar 2021 15:43:24 +1000 Subject: [PATCH 16/36] Fix whitespace --- book/src/dev/rfcs/0010-v5-transaction.md | 1 - 1 file changed, 1 deletion(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index 1a86e372c88..e0fcee8888d 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -214,7 +214,6 @@ These changes should be made in a later pull request, see [#1917](https://github - "Fake" Sapling-only and Sapling/Transparent transactions based on the existing test vectors, converted from `V4` to `V5` format - We can write a test utility function to automatically do these conversions - An empty transaction, with no Orchard, Sapling, or Transparent data - - Any available `zcashd` test vectors - After NU5 activation on testnet: - Add test vectors using the testnet activation block and 2 more post-activation blocks From 9ff6db140408eca7657695916479309bba41ece8 Mon Sep 17 00:00:00 2001 From: teor Date: Fri, 19 Mar 2021 15:45:44 +1000 Subject: [PATCH 17/36] Move sapling value balance into shielded data --- book/src/dev/rfcs/0010-v5-transaction.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index e0fcee8888d..f61c9c6efce 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -78,14 +78,11 @@ Here we have the proposed changes for V4 transactions: ``` V4 { - ... - sapling_value_balance: Amount, ... sapling_shielded_data: Option, } ``` -In order to mantain the order of the fields that we will need for Serialization/Deserialization we keep the `value_balance` in the transaction for V4 but we rename it to `sapling_value_balance` for clarity. `ShieldedData` is currently defined and implemented in `zebra-chain/src/transaction/shielded_data.rs`. As this is Sapling specific we propose to move this file to `zebra-chain/src/sapling/shielded_data.rs`. Inside we will change `ShieldedData` into `SaplingShieldedData` and implement it as an enum with `V4` and `V5` variants. @@ -100,6 +97,7 @@ pub enum SaplingShieldedData { pub first: Either, pub rest_spends: Vec, pub rest_outputs: Vec, + pub sapling_value_balance: Amount, pub binding_sig: Signature, }, V5 { From 4b476688dcaa1f8cb2de731d0300b65211d2d3e8 Mon Sep 17 00:00:00 2001 From: teor Date: Fri, 19 Mar 2021 15:49:18 +1000 Subject: [PATCH 18/36] Add AuthorizedAction type --- book/src/dev/rfcs/0010-v5-transaction.md | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index f61c9c6efce..bd2ee77a1e3 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -165,12 +165,12 @@ V5 { The new V5 structure will create a new `OrchardShieldedData` type. This new type will be defined in a separated file: `zebra-chain/src/orchard/shielded_data.rs` and it will look as follows: ``` pub struct OrchardShieldedData { - /// An action description. + /// An authorized action description. /// /// Storing this separately ensures that it is impossible to construct /// an invalid `OrchardShieldedData` with no actions. - pub first: Action, - pub rest: Vec, + pub first: AuthorizedAction, + pub rest: Vec, pub flags: OrchardFlags, pub orchard_value_balance: Amount, pub anchor: tree::Root, @@ -178,6 +178,12 @@ pub struct OrchardShieldedData { } ``` +In `V5` transactions, there is one `SpendAuth` signature for every `Action`. To ensure that this structural rule is followed, we create an `AuthorizedAction` type: +AuthorizedAction { + action: Action, + spend_auth_sig: redpallas::Signature, +} + Where `Action` is defined as [Action definition](https://github.com/ZcashFoundation/zebra/blob/68c12d045b63ed49dd1963dd2dc22eb991f3998c/zebra-chain/src/orchard/action.rs#L18-L41). Finally, in the V5 transaction we have a new `OrchardFlags` type. This is a bitfield type defined as: From 40933f1bca5eada5dc9f457a82c3408c698ae11d Mon Sep 17 00:00:00 2001 From: teor Date: Fri, 19 Mar 2021 16:13:46 +1000 Subject: [PATCH 19/36] Improve formatting and consistency --- book/src/dev/rfcs/0010-v5-transaction.md | 125 +++++++++++++++-------- 1 file changed, 84 insertions(+), 41 deletions(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index bd2ee77a1e3..18872a99442 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -33,23 +33,26 @@ We need the raw representation to Serialize/Deserialize the transaction however V5 transactions are the only ones that will support orchard transactions with `Orchard` data types. -V4 and V5 transactions both support sapling data but they are implemented differently. By this reason we need to split sapling data types into V4 and V5. We do this by using `enum` data types. +V4 and V5 transactions both support sapling data but they are implemented differently. By this reason we need to split sapling data types into V4 and V5. The order of some of the fields changed from V4 to V5. For example the `lock_time` and `expiry_height` were moved above the transparent inputs and outputs. All of the changes proposed in this document are only to the `zebra-chain` crate. -To highlight changes all document comments from the code snippets in the [reference section](#reference-level-explanation) were removed. +To highlight changes most of the document comments from the code snippets in the [reference section](#reference-level-explanation) were removed. # Reference-level explanation [reference-level-explanation]: #reference-level-explanation +## Current V4 Transactions +[current-v4-transactions]: #current-v4-transactions + We start by looking how a V4 (already implemented) transaction is represented in Zebra. This transaction version is specified by the protocol in the first table of [Transaction Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#txnencodingandconsensus). -``` +```rust V4 { inputs: Vec, outputs: Vec, @@ -63,7 +66,7 @@ V4 { Currently the `ShieldedData` type is defined in `zebra-chain/src/transaction/shielded_data.rs` as follows: -``` +```rust pub struct ShieldedData { pub first: Either, pub rest_spends: Vec, @@ -74,46 +77,60 @@ pub struct ShieldedData { We know by protocol (2nd table of [Transaction Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#txnencodingandconsensus)) that V5 transactions will support sapling data however we also know by protocol that spends ([Spend Description Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#spendencodingandconsensus), See †) and outputs ([Output Description Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#outputencodingandconsensus), See †) fields change from V4 to V5. -Here we have the proposed changes for V4 transactions: +## Sapling Changes +[sapling-changes]: #sapling-changes -``` +### Changes to V4 Transactions +[changes-to-v4-transactions]: #changes-to-v4-transactions + +Here we have the proposed changes for `Transaction::V4`: +* make `sapling_shielded_data` use the `V4` shielded data type (**TODO: how?**) +* rename `shielded_data` to `sapling_shielded_data` +* move `value_balance` into the `sapling::ShieldedData` type + +```rust V4 { ... - sapling_shielded_data: Option, + sapling_shielded_data: Option, // Note: enum variants can't be generic parameters in Rust } ``` +### Changes to Sapling ShieldedData +[changes-to-sapling-shieldeddata]: #changes-to-sapling-shieldeddata -`ShieldedData` is currently defined and implemented in `zebra-chain/src/transaction/shielded_data.rs`. As this is Sapling specific we propose to move this file to `zebra-chain/src/sapling/shielded_data.rs`. Inside we will change `ShieldedData` into `SaplingShieldedData` and implement it as an enum with `V4` and `V5` variants. +`ShieldedData` is currently defined and implemented in `zebra-chain/src/transaction/shielded_data.rs`. As this is Sapling specific we propose to move this file to `zebra-chain/src/sapling/shielded_data.rs`. We will also change `ShieldedData` into an enum with `V4` and `V5` variants. -``` -pub enum SaplingShieldedData { +```rust +pub enum sapling::ShieldedData { V4 { /// Either a spend or output description. /// /// Storing this separately ensures that it is impossible to construct - /// an invalid `SaplingShieldedData` with no spends or outputs. + /// an invalid `ShieldedData` with no spends or outputs. /// ... - pub first: Either, - pub rest_spends: Vec, + pub first: Either, // Note: enum variants can't be generic parameters in Rust + pub rest_spends: Vec, // Note: enum variants can't be generic parameters in Rust pub rest_outputs: Vec, - pub sapling_value_balance: Amount, + pub value_balance: Amount, pub binding_sig: Signature, }, V5 { - pub first: Either, - pub rest_spends: Vec, + pub first: Either, // Note: enum variants can't be generic parameters in Rust + pub rest_spends: Vec, // Note: enum variants can't be generic parameters in Rust pub rest_outputs: Vec, - pub sapling_value_balance: Amount, + pub value_balance: Amount, pub anchor: tree::Root, pub binding_sig: Signature, } } ``` +### Adding V5 Sapling Spend +[adding-v5-sapling-spend-and-output]: #adding-v5-sapling-spend-and-output + Proposed `Spend` is now defined as an enum with `V4` and `V5` variants. Sapling spend code is located at `zebra-chain/src/sapling/spend.rs`. Notable difference here is that the `anchor` in `V4` is needed for every spend of the transaction while in `V5` the anchor is a single one defined in `SaplingShieldedData`: -``` +```rust pub enum Spend { V4 { pub cv: commitment::ValueCommitment, @@ -132,9 +149,13 @@ pub enum Spend { } } ``` -In Zebra the output representations are the same for V4 and V5 so no variants are needed. The output code is located at `zebra-chain/src/sapling/output.rs`: -``` +### No Changes to Sapling Output +[no-changes-to-sapling-output]: #no-changes-to-sapling-output + +In Zcash the Sapling output representations are the same for V4 and V5 transactions, so no variants are needed. The output code is located at `zebra-chain/src/sapling/output.rs`: + +```rust pub struct Output { pub cv: commitment::ValueCommitment, pub cm_u: jubjub::Fq, @@ -145,50 +166,72 @@ pub struct Output { } ``` +## Orchard Additions +[orchard-additions]: #orchard-additions + +### Adding V5 Transactions +[adding-v5-transactions]: #adding-v5-transactions + Now lets see how the V5 transaction is specified in the protocol, this is the second table of [Transaction Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#txnencodingandconsensus) and how are we going to represent it based in the above changes for Sapling fields and the new Orchard fields. We propose the following representation for transaction V5 in Zebra: -``` +```rust V5 { lock_time: LockTime, expiry_height: block::Height, inputs: Vec, outputs: Vec, - sapling_shielded_data: Option, - orchard_shielded_data: Option, + sapling_shielded_data: Option, // Note: enum variants can't be generic parameters in Rust + orchard_shielded_data: Option, } ``` -`sapling_shielded_data` will now use `SaplingShieldedData::V5` variant located at `zebra-chain/src/transaction/sapling_shielded_data.rs` with the corresponding `Spend::V5` for the spends. +`sapling_shielded_data` will now use `SaplingShieldedData::V5` variant located at `zebra-chain/src/transaction/sapling_shielded_data.rs` with the corresponding `Spend::V5` for the spends. (**TODO: how?**) -The new V5 structure will create a new `OrchardShieldedData` type. This new type will be defined in a separated file: `zebra-chain/src/orchard/shielded_data.rs` and it will look as follows: -``` -pub struct OrchardShieldedData { +### Adding Orchard ShieldedData +[adding-orchard-shieldeddata]: #adding-orchard-shieldeddata + +The new V5 structure will create a new `orchard::ShieldedData` type. This new type will be defined in a separated file: `zebra-chain/src/orchard/shielded_data.rs` and it will look as follows: + +```rust +pub struct orchard::ShieldedData { /// An authorized action description. /// /// Storing this separately ensures that it is impossible to construct - /// an invalid `OrchardShieldedData` with no actions. + /// an invalid `ShieldedData` with no actions. pub first: AuthorizedAction, pub rest: Vec, - pub flags: OrchardFlags, + pub flags: Flags, pub orchard_value_balance: Amount, pub anchor: tree::Root, pub binding_sig: redpallas::Signature, } ``` +### Adding Orchard AuthorizedAction +[adding-orchard-authorizedaction]: #adding-orchard-authorizedaction + In `V5` transactions, there is one `SpendAuth` signature for every `Action`. To ensure that this structural rule is followed, we create an `AuthorizedAction` type: -AuthorizedAction { + +```rust +/// An authorized action description. +/// +/// Every authorized Orchard `Action` must have a corresponding `SpendAuth` signature. +pub struct orchard::AuthorizedAction { action: Action, spend_auth_sig: redpallas::Signature, } +``` Where `Action` is defined as [Action definition](https://github.com/ZcashFoundation/zebra/blob/68c12d045b63ed49dd1963dd2dc22eb991f3998c/zebra-chain/src/orchard/action.rs#L18-L41). -Finally, in the V5 transaction we have a new `OrchardFlags` type. This is a bitfield type defined as: +### Adding Orchard Flags +[adding-orchard-flags]: #adding-orchard-flags -``` +Finally, in the V5 transaction we have a new `orchard::Flags` type. This is a bitfield type defined as: + +```rust bitflags! { pub struct OrchardFlags: u8 { const ENABLE_SPENDS = 0b00000001; @@ -198,15 +241,6 @@ bitflags! { } ``` -## Security - -To avoid parsing memory exhaustion attacks, we will make the following changes across all `Transaction`, `ShieldedData`, `Spend` and `Output` variants, `V1` through to `V5`: -- Check cardinality consensus rules at parse time, before deserializing any `Vec`s - - In general, Zcash requires that each transaction has at least one Transparent/Sprout/Sapling/Orchard transfer, this rule is not currently encoded in our data structures -- Stop parsing as soon as the first error is detected - -These changes should be made in a later pull request, see [#1917](https://github.com/ZcashFoundation/zebra/issues/1917) for details. - ## Test Plan [test-plan]: #test-plan @@ -223,3 +257,12 @@ These changes should be made in a later pull request, see [#1917](https://github - Add test vectors using the testnet activation block and 2 more post-activation blocks - After NU5 activation on mainnet: - Add test vectors using the mainnet activation block and 2 more post-activation blocks + +# Security + +To avoid parsing memory exhaustion attacks, we will make the following changes across all `Transaction`, `ShieldedData`, `Spend` and `Output` variants, `V1` through to `V5`: +- Check cardinality consensus rules at parse time, before deserializing any `Vec`s + - In general, Zcash requires that each transaction has at least one Transparent/Sprout/Sapling/Orchard transfer, this rule is not currently encoded in our data structures +- Stop parsing as soon as the first error is detected + +These changes should be made in a later pull request, see [#1917](https://github.com/ZcashFoundation/zebra/issues/1917) for details. From ae3b51284db966155c009c58e456ece3cff9c686 Mon Sep 17 00:00:00 2001 From: teor Date: Fri, 19 Mar 2021 16:18:33 +1000 Subject: [PATCH 20/36] More module structure edits --- book/src/dev/rfcs/0010-v5-transaction.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index 18872a99442..fde403c36b7 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -128,7 +128,7 @@ pub enum sapling::ShieldedData { ### Adding V5 Sapling Spend [adding-v5-sapling-spend-and-output]: #adding-v5-sapling-spend-and-output -Proposed `Spend` is now defined as an enum with `V4` and `V5` variants. Sapling spend code is located at `zebra-chain/src/sapling/spend.rs`. Notable difference here is that the `anchor` in `V4` is needed for every spend of the transaction while in `V5` the anchor is a single one defined in `SaplingShieldedData`: +Proposed `Spend` is now defined as an enum with `V4` and `V5` variants. Sapling spend code is located at `zebra-chain/src/sapling/spend.rs`. Notable difference here is that the `anchor` in `V4` is needed for every spend of the transaction while in `V5` the anchor is a single one defined in `sapling::ShieldedData`: ```rust pub enum Spend { @@ -187,7 +187,7 @@ V5 { } ``` -`sapling_shielded_data` will now use `SaplingShieldedData::V5` variant located at `zebra-chain/src/transaction/sapling_shielded_data.rs` with the corresponding `Spend::V5` for the spends. (**TODO: how?**) +`sapling_shielded_data` will now use `sapling::ShieldedData::V5` variant located at `zebra-chain/src/transaction/sapling/shielded_data.rs` with the corresponding `Spend::V5` for the spends. (**TODO: how?**) ### Adding Orchard ShieldedData [adding-orchard-shieldeddata]: #adding-orchard-shieldeddata @@ -203,7 +203,7 @@ pub struct orchard::ShieldedData { pub first: AuthorizedAction, pub rest: Vec, pub flags: Flags, - pub orchard_value_balance: Amount, + pub value_balance: Amount, pub anchor: tree::Root, pub binding_sig: redpallas::Signature, } @@ -233,7 +233,7 @@ Finally, in the V5 transaction we have a new `orchard::Flags` type. This is a bi ```rust bitflags! { - pub struct OrchardFlags: u8 { + pub struct orchard::Flags: u8 { const ENABLE_SPENDS = 0b00000001; const ENABLE_OUTPUTS = 0b00000010; // Reserved, zeros (bits 2 .. 7) From bf84852c2c22759937f7590a63429f15246e1b33 Mon Sep 17 00:00:00 2001 From: teor Date: Fri, 19 Mar 2021 16:22:18 +1000 Subject: [PATCH 21/36] Make enum variants more explicit --- book/src/dev/rfcs/0010-v5-transaction.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index fde403c36b7..038567cd096 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -53,7 +53,7 @@ This transaction version is specified by the protocol in the first table of [Tra ```rust -V4 { +pub enum Transaction::V4 { inputs: Vec, outputs: Vec, lock_time: LockTime, @@ -177,7 +177,7 @@ Now lets see how the V5 transaction is specified in the protocol, this is the se We propose the following representation for transaction V5 in Zebra: ```rust -V5 { +pub enum Transaction::V5 { lock_time: LockTime, expiry_height: block::Height, inputs: Vec, From dc81896b7b7ad1031fb47c98e13cba8dae9ca484 Mon Sep 17 00:00:00 2001 From: teor Date: Fri, 19 Mar 2021 16:23:31 +1000 Subject: [PATCH 22/36] Make enum variants more explicit again --- book/src/dev/rfcs/0010-v5-transaction.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index 038567cd096..cd232bcd77f 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -89,7 +89,7 @@ Here we have the proposed changes for `Transaction::V4`: * move `value_balance` into the `sapling::ShieldedData` type ```rust -V4 { +pub enum Transaction::V4 { ... sapling_shielded_data: Option, // Note: enum variants can't be generic parameters in Rust } From b5af73d6d2a5e2dd3e3382197d024b694827433a Mon Sep 17 00:00:00 2001 From: teor Date: Fri, 19 Mar 2021 16:26:42 +1000 Subject: [PATCH 23/36] Delete current code to avoid confusion --- book/src/dev/rfcs/0010-v5-transaction.md | 41 ++++-------------------- 1 file changed, 7 insertions(+), 34 deletions(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index cd232bcd77f..4a5a159ce38 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -44,42 +44,11 @@ To highlight changes most of the document comments from the code snippets in the # Reference-level explanation [reference-level-explanation]: #reference-level-explanation -## Current V4 Transactions -[current-v4-transactions]: #current-v4-transactions - -We start by looking how a V4 (already implemented) transaction is represented in Zebra. - -This transaction version is specified by the protocol in the first table of [Transaction Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#txnencodingandconsensus). - - -```rust -pub enum Transaction::V4 { - inputs: Vec, - outputs: Vec, - lock_time: LockTime, - expiry_height: block::Height, - value_balance: Amount, - joinsplit_data: Option>, - shielded_data: Option, -} -``` - -Currently the `ShieldedData` type is defined in `zebra-chain/src/transaction/shielded_data.rs` as follows: - -```rust -pub struct ShieldedData { - pub first: Either, - pub rest_spends: Vec, - pub rest_outputs: Vec, - pub binding_sig: Signature, -} -``` - -We know by protocol (2nd table of [Transaction Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#txnencodingandconsensus)) that V5 transactions will support sapling data however we also know by protocol that spends ([Spend Description Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#spendencodingandconsensus), See †) and outputs ([Output Description Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#outputencodingandconsensus), See †) fields change from V4 to V5. - ## Sapling Changes [sapling-changes]: #sapling-changes +We know by protocol (2nd table of [Transaction Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#txnencodingandconsensus)) that V5 transactions will support sapling data however we also know by protocol that spends ([Spend Description Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#spendencodingandconsensus), See †) and outputs ([Output Description Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#outputencodingandconsensus), See †) fields change from V4 to V5. + ### Changes to V4 Transactions [changes-to-v4-transactions]: #changes-to-v4-transactions @@ -90,7 +59,11 @@ Here we have the proposed changes for `Transaction::V4`: ```rust pub enum Transaction::V4 { - ... + inputs: Vec, + outputs: Vec, + lock_time: LockTime, + expiry_height: block::Height, + joinsplit_data: Option>, sapling_shielded_data: Option, // Note: enum variants can't be generic parameters in Rust } ``` From 6a5724b7af16f0143809bd455f2507df407aadae Mon Sep 17 00:00:00 2001 From: teor Date: Fri, 19 Mar 2021 16:37:23 +1000 Subject: [PATCH 24/36] Order fields based on last serialized data --- book/src/dev/rfcs/0010-v5-transaction.md | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index 4a5a159ce38..c5aa202e381 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -56,6 +56,7 @@ Here we have the proposed changes for `Transaction::V4`: * make `sapling_shielded_data` use the `V4` shielded data type (**TODO: how?**) * rename `shielded_data` to `sapling_shielded_data` * move `value_balance` into the `sapling::ShieldedData` type +* order fields based on the **last** data deserialized for each field ```rust pub enum Transaction::V4 { @@ -76,6 +77,7 @@ pub enum Transaction::V4 { ```rust pub enum sapling::ShieldedData { V4 { + pub value_balance: Amount, /// Either a spend or output description. /// /// Storing this separately ensures that it is impossible to construct @@ -84,15 +86,14 @@ pub enum sapling::ShieldedData { pub first: Either, // Note: enum variants can't be generic parameters in Rust pub rest_spends: Vec, // Note: enum variants can't be generic parameters in Rust pub rest_outputs: Vec, - pub value_balance: Amount, pub binding_sig: Signature, }, V5 { + pub value_balance: Amount, + pub anchor: tree::Root, pub first: Either, // Note: enum variants can't be generic parameters in Rust pub rest_spends: Vec, // Note: enum variants can't be generic parameters in Rust pub rest_outputs: Vec, - pub value_balance: Amount, - pub anchor: tree::Root, pub binding_sig: Signature, } } @@ -169,19 +170,21 @@ The new V5 structure will create a new `orchard::ShieldedData` type. This new ty ```rust pub struct orchard::ShieldedData { + pub flags: Flags, + pub value_balance: Amount, + pub anchor: tree::Root, /// An authorized action description. /// /// Storing this separately ensures that it is impossible to construct /// an invalid `ShieldedData` with no actions. pub first: AuthorizedAction, pub rest: Vec, - pub flags: Flags, - pub value_balance: Amount, - pub anchor: tree::Root, pub binding_sig: redpallas::Signature, } ``` +The fields are ordered based on the **last** data deserialized for each field. + ### Adding Orchard AuthorizedAction [adding-orchard-authorizedaction]: #adding-orchard-authorizedaction From 98ab9ed5caff80318952446935bccba601f06e78 Mon Sep 17 00:00:00 2001 From: teor Date: Fri, 19 Mar 2021 16:37:45 +1000 Subject: [PATCH 25/36] Fix crossref name --- book/src/dev/rfcs/0010-v5-transaction.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index c5aa202e381..2d82ce62003 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -100,7 +100,7 @@ pub enum sapling::ShieldedData { ``` ### Adding V5 Sapling Spend -[adding-v5-sapling-spend-and-output]: #adding-v5-sapling-spend-and-output +[adding-v5-sapling-spend]: #adding-v5-sapling-spend Proposed `Spend` is now defined as an enum with `V4` and `V5` variants. Sapling spend code is located at `zebra-chain/src/sapling/spend.rs`. Notable difference here is that the `anchor` in `V4` is needed for every spend of the transaction while in `V5` the anchor is a single one defined in `sapling::ShieldedData`: From cc5d360f2f5a5740af65c8d99dbeef8a8797af99 Mon Sep 17 00:00:00 2001 From: teor Date: Fri, 19 Mar 2021 16:44:54 +1000 Subject: [PATCH 26/36] Add the proofsOrchard field --- book/src/dev/rfcs/0010-v5-transaction.md | 1 + 1 file changed, 1 insertion(+) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index 2d82ce62003..6760894e7aa 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -173,6 +173,7 @@ pub struct orchard::ShieldedData { pub flags: Flags, pub value_balance: Amount, pub anchor: tree::Root, + combined_proof: Halo2Proof, /// An authorized action description. /// /// Storing this separately ensures that it is impossible to construct From 884a074d395c8b0dacd399b15f562a4138faa511 Mon Sep 17 00:00:00 2001 From: teor Date: Fri, 19 Mar 2021 16:50:16 +1000 Subject: [PATCH 27/36] Fix visibility --- book/src/dev/rfcs/0010-v5-transaction.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index 6760894e7aa..15551b7d97d 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -173,7 +173,7 @@ pub struct orchard::ShieldedData { pub flags: Flags, pub value_balance: Amount, pub anchor: tree::Root, - combined_proof: Halo2Proof, + pub combined_proof: Halo2Proof, /// An authorized action description. /// /// Storing this separately ensures that it is impossible to construct From f2a7ca4e4b9f02031ed880ded26054e70ad13f1c Mon Sep 17 00:00:00 2001 From: teor Date: Fri, 19 Mar 2021 16:53:31 +1000 Subject: [PATCH 28/36] Delete visibility modifiers for readability All those `pub`s make the design harder to read. --- book/src/dev/rfcs/0010-v5-transaction.md | 86 ++++++++++++------------ 1 file changed, 43 insertions(+), 43 deletions(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index 15551b7d97d..7dc4d2cc2f1 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -59,7 +59,7 @@ Here we have the proposed changes for `Transaction::V4`: * order fields based on the **last** data deserialized for each field ```rust -pub enum Transaction::V4 { +enum Transaction::V4 { inputs: Vec, outputs: Vec, lock_time: LockTime, @@ -75,26 +75,26 @@ pub enum Transaction::V4 { `ShieldedData` is currently defined and implemented in `zebra-chain/src/transaction/shielded_data.rs`. As this is Sapling specific we propose to move this file to `zebra-chain/src/sapling/shielded_data.rs`. We will also change `ShieldedData` into an enum with `V4` and `V5` variants. ```rust -pub enum sapling::ShieldedData { +enum sapling::ShieldedData { V4 { - pub value_balance: Amount, + value_balance: Amount, /// Either a spend or output description. /// /// Storing this separately ensures that it is impossible to construct /// an invalid `ShieldedData` with no spends or outputs. /// ... - pub first: Either, // Note: enum variants can't be generic parameters in Rust - pub rest_spends: Vec, // Note: enum variants can't be generic parameters in Rust - pub rest_outputs: Vec, - pub binding_sig: Signature, + first: Either, // Note: enum variants can't be generic parameters in Rust + rest_spends: Vec, // Note: enum variants can't be generic parameters in Rust + rest_outputs: Vec, + binding_sig: Signature, }, V5 { - pub value_balance: Amount, - pub anchor: tree::Root, - pub first: Either, // Note: enum variants can't be generic parameters in Rust - pub rest_spends: Vec, // Note: enum variants can't be generic parameters in Rust - pub rest_outputs: Vec, - pub binding_sig: Signature, + value_balance: Amount, + anchor: tree::Root, + first: Either, // Note: enum variants can't be generic parameters in Rust + rest_spends: Vec, // Note: enum variants can't be generic parameters in Rust + rest_outputs: Vec, + binding_sig: Signature, } } ``` @@ -105,21 +105,21 @@ pub enum sapling::ShieldedData { Proposed `Spend` is now defined as an enum with `V4` and `V5` variants. Sapling spend code is located at `zebra-chain/src/sapling/spend.rs`. Notable difference here is that the `anchor` in `V4` is needed for every spend of the transaction while in `V5` the anchor is a single one defined in `sapling::ShieldedData`: ```rust -pub enum Spend { +enum Spend { V4 { - pub cv: commitment::ValueCommitment, - pub anchor: tree::Root, - pub nullifier: note::Nullifier, - pub rk: redjubjub::VerificationKeyBytes, - pub zkproof: Groth16Proof, - pub spend_auth_sig: redjubjub::Signature, + cv: commitment::ValueCommitment, + anchor: tree::Root, + nullifier: note::Nullifier, + rk: redjubjub::VerificationKeyBytes, + zkproof: Groth16Proof, + spend_auth_sig: redjubjub::Signature, }, V5 { - pub cv: commitment::ValueCommitment, - pub nullifier: note::Nullifier, - pub rk: redjubjub::VerificationKeyBytes, - pub zkproof: Groth16Proof, - pub spend_auth_sig: redjubjub::Signature, + cv: commitment::ValueCommitment, + nullifier: note::Nullifier, + rk: redjubjub::VerificationKeyBytes, + zkproof: Groth16Proof, + spend_auth_sig: redjubjub::Signature, } } ``` @@ -130,13 +130,13 @@ pub enum Spend { In Zcash the Sapling output representations are the same for V4 and V5 transactions, so no variants are needed. The output code is located at `zebra-chain/src/sapling/output.rs`: ```rust -pub struct Output { - pub cv: commitment::ValueCommitment, - pub cm_u: jubjub::Fq, - pub ephemeral_key: keys::EphemeralPublicKey, - pub enc_ciphertext: note::EncryptedNote, - pub out_ciphertext: note::WrappedNoteKey, - pub zkproof: Groth16Proof, +struct Output { + cv: commitment::ValueCommitment, + cm_u: jubjub::Fq, + ephemeral_key: keys::EphemeralPublicKey, + enc_ciphertext: note::EncryptedNote, + out_ciphertext: note::WrappedNoteKey, + zkproof: Groth16Proof, } ``` @@ -151,7 +151,7 @@ Now lets see how the V5 transaction is specified in the protocol, this is the se We propose the following representation for transaction V5 in Zebra: ```rust -pub enum Transaction::V5 { +enum Transaction::V5 { lock_time: LockTime, expiry_height: block::Height, inputs: Vec, @@ -169,18 +169,18 @@ pub enum Transaction::V5 { The new V5 structure will create a new `orchard::ShieldedData` type. This new type will be defined in a separated file: `zebra-chain/src/orchard/shielded_data.rs` and it will look as follows: ```rust -pub struct orchard::ShieldedData { - pub flags: Flags, - pub value_balance: Amount, - pub anchor: tree::Root, - pub combined_proof: Halo2Proof, +struct orchard::ShieldedData { + flags: Flags, + value_balance: Amount, + anchor: tree::Root, + combined_proof: Halo2Proof, /// An authorized action description. /// /// Storing this separately ensures that it is impossible to construct /// an invalid `ShieldedData` with no actions. - pub first: AuthorizedAction, - pub rest: Vec, - pub binding_sig: redpallas::Signature, + first: AuthorizedAction, + rest: Vec, + binding_sig: redpallas::Signature, } ``` @@ -195,7 +195,7 @@ In `V5` transactions, there is one `SpendAuth` signature for every `Action`. To /// An authorized action description. /// /// Every authorized Orchard `Action` must have a corresponding `SpendAuth` signature. -pub struct orchard::AuthorizedAction { +struct orchard::AuthorizedAction { action: Action, spend_auth_sig: redpallas::Signature, } @@ -210,7 +210,7 @@ Finally, in the V5 transaction we have a new `orchard::Flags` type. This is a bi ```rust bitflags! { - pub struct orchard::Flags: u8 { + struct orchard::Flags: u8 { const ENABLE_SPENDS = 0b00000001; const ENABLE_OUTPUTS = 0b00000010; // Reserved, zeros (bits 2 .. 7) From e7653681c6b63f3b26cfcbd2df7cfc26e573f5c8 Mon Sep 17 00:00:00 2001 From: teor Date: Fri, 19 Mar 2021 19:31:39 +1000 Subject: [PATCH 29/36] Model anchor variants as generic trait types --- book/src/dev/rfcs/0010-v5-transaction.md | 92 ++++++++++++------------ 1 file changed, 47 insertions(+), 45 deletions(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index 7dc4d2cc2f1..32e0e71f60c 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -33,7 +33,7 @@ We need the raw representation to Serialize/Deserialize the transaction however V5 transactions are the only ones that will support orchard transactions with `Orchard` data types. -V4 and V5 transactions both support sapling data but they are implemented differently. By this reason we need to split sapling data types into V4 and V5. +V4 and V5 transactions both support sapling, but the underying data structures are different. So need to make the sapling data types generic over the V4 and V5 structures. The order of some of the fields changed from V4 to V5. For example the `lock_time` and `expiry_height` were moved above the transparent inputs and outputs. @@ -52,8 +52,8 @@ We know by protocol (2nd table of [Transaction Encoding and Consensus](https://z ### Changes to V4 Transactions [changes-to-v4-transactions]: #changes-to-v4-transactions -Here we have the proposed changes for `Transaction::V4`: -* make `sapling_shielded_data` use the `V4` shielded data type (**TODO: how?**) +Here we have the proposed changes for V4 transactions: +* make `sapling_shielded_data` use the `PerSpendAnchor` anchor variant * rename `shielded_data` to `sapling_shielded_data` * move `value_balance` into the `sapling::ShieldedData` type * order fields based on the **last** data deserialized for each field @@ -65,62 +65,64 @@ enum Transaction::V4 { lock_time: LockTime, expiry_height: block::Height, joinsplit_data: Option>, - sapling_shielded_data: Option, // Note: enum variants can't be generic parameters in Rust + sapling_shielded_data: Option>, +} +``` + +### Anchor Variants +[anchor-variants]: #anchor-variants + +We implement an `AnchorVariant` generic type trait, because V4 transactions have a per-`Spend` anchor, but V5 transactions have a shared anchor: + +```rust +struct PerSpendAnchor {} +struct SharedAnchor {} + +impl AnchorVariant for PerSpendAnchor { + type Shared = (); + type PerSpend = tree::Root; +} + +impl AnchorVariant for SharedAnchor { + type Shared = tree::Root; + type PerSpend = (); +} + +trait AnchorVariant { + type Shared; + type PerSpend; } ``` ### Changes to Sapling ShieldedData [changes-to-sapling-shieldeddata]: #changes-to-sapling-shieldeddata -`ShieldedData` is currently defined and implemented in `zebra-chain/src/transaction/shielded_data.rs`. As this is Sapling specific we propose to move this file to `zebra-chain/src/sapling/shielded_data.rs`. We will also change `ShieldedData` into an enum with `V4` and `V5` variants. +`ShieldedData` is currently defined and implemented in `zebra-chain/src/transaction/shielded_data.rs`. As this is Sapling specific we propose to move this file to `zebra-chain/src/sapling/shielded_data.rs`. We use `AnchorVariant` to model the anchor differences between V4 and V5: ```rust -enum sapling::ShieldedData { - V4 { - value_balance: Amount, - /// Either a spend or output description. - /// - /// Storing this separately ensures that it is impossible to construct - /// an invalid `ShieldedData` with no spends or outputs. - /// ... - first: Either, // Note: enum variants can't be generic parameters in Rust - rest_spends: Vec, // Note: enum variants can't be generic parameters in Rust - rest_outputs: Vec, - binding_sig: Signature, - }, - V5 { - value_balance: Amount, - anchor: tree::Root, - first: Either, // Note: enum variants can't be generic parameters in Rust - rest_spends: Vec, // Note: enum variants can't be generic parameters in Rust - rest_outputs: Vec, - binding_sig: Signature, - } +struct sapling::ShieldedData { + value_balance: Amount, + anchor: Anchor::Shared, + first: Either, Output>, + rest_spends: Vec>, + rest_outputs: Vec, + binding_sig: Signature, } ``` ### Adding V5 Sapling Spend [adding-v5-sapling-spend]: #adding-v5-sapling-spend -Proposed `Spend` is now defined as an enum with `V4` and `V5` variants. Sapling spend code is located at `zebra-chain/src/sapling/spend.rs`. Notable difference here is that the `anchor` in `V4` is needed for every spend of the transaction while in `V5` the anchor is a single one defined in `sapling::ShieldedData`: +Sapling spend code is located at `zebra-chain/src/sapling/spend.rs`. We use `AnchorVariant` to model the anchor differences between V4 and V5: ```rust -enum Spend { - V4 { - cv: commitment::ValueCommitment, - anchor: tree::Root, - nullifier: note::Nullifier, - rk: redjubjub::VerificationKeyBytes, - zkproof: Groth16Proof, - spend_auth_sig: redjubjub::Signature, - }, - V5 { - cv: commitment::ValueCommitment, - nullifier: note::Nullifier, - rk: redjubjub::VerificationKeyBytes, - zkproof: Groth16Proof, - spend_auth_sig: redjubjub::Signature, - } +struct Spend { + cv: commitment::ValueCommitment, + anchor: Anchor::PerSpend, + nullifier: note::Nullifier, + rk: redjubjub::VerificationKeyBytes, + zkproof: Groth16Proof, + spend_auth_sig: redjubjub::Signature, } ``` @@ -156,12 +158,12 @@ enum Transaction::V5 { expiry_height: block::Height, inputs: Vec, outputs: Vec, - sapling_shielded_data: Option, // Note: enum variants can't be generic parameters in Rust + sapling_shielded_data: Option>, orchard_shielded_data: Option, } ``` -`sapling_shielded_data` will now use `sapling::ShieldedData::V5` variant located at `zebra-chain/src/transaction/sapling/shielded_data.rs` with the corresponding `Spend::V5` for the spends. (**TODO: how?**) +To model the V5 anchor type, `sapling_shielded_data` uses the `SharedAnchor` variant located at `zebra-chain/src/transaction/sapling/shielded_data.rs`. ### Adding Orchard ShieldedData [adding-orchard-shieldeddata]: #adding-orchard-shieldeddata From 2a12990a92cd455bbace506141ad0acf6b603002 Mon Sep 17 00:00:00 2001 From: teor Date: Fri, 19 Mar 2021 19:37:14 +1000 Subject: [PATCH 30/36] Minor cleanups --- book/src/dev/rfcs/0010-v5-transaction.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index 32e0e71f60c..a7dab1b2dd1 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -175,7 +175,7 @@ struct orchard::ShieldedData { flags: Flags, value_balance: Amount, anchor: tree::Root, - combined_proof: Halo2Proof, + proof: Halo2Proof, /// An authorized action description. /// /// Storing this separately ensures that it is impossible to construct @@ -223,12 +223,12 @@ bitflags! { ## Test Plan [test-plan]: #test-plan -- All renamed, modified and new types should serialize and deserialize. +- All renamed, modified and new types should serialize and deserialize. - The full V4 and V5 transactions should serialize and deserialize. -- Prop test strategies for v4 and v5 will be updated and created. +- Prop test strategies for V4 and V5 will be updated and created. - Before NU5 activation on testnet, test on the following test vectors: - Hand-crafted Orchard-only, Orchard/Sapling, Orchard/Transparent, and Orchard/Sapling/Transparent transactions based on the spec - - "Fake" Sapling-only and Sapling/Transparent transactions based on the existing test vectors, converted from `V4` to `V5` format + - "Fake" Sapling-only and Sapling/Transparent transactions based on the existing test vectors, converted from V4 to V5 format - We can write a test utility function to automatically do these conversions - An empty transaction, with no Orchard, Sapling, or Transparent data - Any available `zcashd` test vectors @@ -239,9 +239,9 @@ bitflags! { # Security -To avoid parsing memory exhaustion attacks, we will make the following changes across all `Transaction`, `ShieldedData`, `Spend` and `Output` variants, `V1` through to `V5`: +To avoid parsing memory exhaustion attacks, we will make the following changes across all `Transaction`, `ShieldedData`, `Spend` and `Output` variants, V1 through to V5: - Check cardinality consensus rules at parse time, before deserializing any `Vec`s - - In general, Zcash requires that each transaction has at least one Transparent/Sprout/Sapling/Orchard transfer, this rule is not currently encoded in our data structures + - In general, Zcash requires that each transaction has at least one Transparent/Sprout/Sapling/Orchard transfer, this rule is not currently encoded in our data structures (it is only checked during semantic verification) - Stop parsing as soon as the first error is detected These changes should be made in a later pull request, see [#1917](https://github.com/ZcashFoundation/zebra/issues/1917) for details. From 708d15f273767bd615f54a5758a9ae8aeddb3db9 Mon Sep 17 00:00:00 2001 From: teor Date: Fri, 19 Mar 2021 19:52:52 +1000 Subject: [PATCH 31/36] Be specific about the network upgrade --- book/src/dev/rfcs/0010-v5-transaction.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index a7dab1b2dd1..0febc916b6c 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -11,7 +11,7 @@ Network Upgrade number 5 (`NU5`) introduces a new transaction type (transaction # Motivation [motivation]: #motivation -The Zebra software wants to be a protocol compatible Zcash implementation. One of the tasks to do this includes the support of the new version 5 transactions that will be implemented in the next network upgrade. +The Zebra software wants to be a protocol compatible Zcash implementation. One of the tasks to do this includes the support of the new version 5 transactions that will be implemented in Network Upgrade 5 (NU5). # Definitions [definitions]: #definitions From c41d1fcb50045fb3eac643ec1ae8f596689645e5 Mon Sep 17 00:00:00 2001 From: teor Date: Mon, 22 Mar 2021 08:30:47 +1000 Subject: [PATCH 32/36] Rewrite guide-level explanation --- book/src/dev/rfcs/0010-v5-transaction.md | 25 +++++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index 0febc916b6c..ee95a0c795a 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -29,17 +29,32 @@ The Zebra software wants to be a protocol compatible Zcash implementation. One o V5 transactions are described by the protocol in the second table of [Transaction Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#txnencodingandconsensus). -We need the raw representation to Serialize/Deserialize the transaction however Zebra uses its own representation internally. +All of the changes proposed in this document are only to the `zebra-chain` crate. + +To highlight changes most of the document comments from the code snippets in the [reference section](#reference-level-explanation) were removed. + +## Sapling Changes Overview +[sapling-changes-overview]: #sapling-changes-overview + +V4 and V5 transactions both support sapling, but the underlying data structures are different. So need to make the sapling data types generic over the V4 and V5 structures. + +In V4, anchors are per-spend, but in V5, they are per-transaction. + +For consistency, also we move some fields into the `ShieldedData` type, and rename some fields and types. + +## Orchard Additions Overview +[orchard-additions-overview]: #orchard-additions-overview V5 transactions are the only ones that will support orchard transactions with `Orchard` data types. -V4 and V5 transactions both support sapling, but the underying data structures are different. So need to make the sapling data types generic over the V4 and V5 structures. +Orchard uses `Halo2Proof`s with corresponding signature type changes. Each Orchard `Action` contains a spend and an output. Placeholder values are substituted for unused spends and outputs. -The order of some of the fields changed from V4 to V5. For example the `lock_time` and `expiry_height` were moved above the transparent inputs and outputs. +## Other Transaction V5 Changes +[other-transaction-v5-changes]: #other-transaction-v5-changes -All of the changes proposed in this document are only to the `zebra-chain` crate. +The order of some of the fields changed from V4 to V5. For example the `lock_time` and `expiry_height` were moved above the transparent inputs and outputs. -To highlight changes most of the document comments from the code snippets in the [reference section](#reference-level-explanation) were removed. +Zebra enums and structs put fields in serialized order. Composite fields are ordered based on **last** data deserialized for each field. # Reference-level explanation [reference-level-explanation]: #reference-level-explanation From d973f5b425ff680f8c5073c2ed41dc226e64c192 Mon Sep 17 00:00:00 2001 From: teor Date: Mon, 22 Mar 2021 08:37:30 +1000 Subject: [PATCH 33/36] Tweak the AnchorVariant generic name --- book/src/dev/rfcs/0010-v5-transaction.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index ee95a0c795a..b3548e9ad84 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -115,11 +115,11 @@ trait AnchorVariant { `ShieldedData` is currently defined and implemented in `zebra-chain/src/transaction/shielded_data.rs`. As this is Sapling specific we propose to move this file to `zebra-chain/src/sapling/shielded_data.rs`. We use `AnchorVariant` to model the anchor differences between V4 and V5: ```rust -struct sapling::ShieldedData { +struct sapling::ShieldedData { value_balance: Amount, - anchor: Anchor::Shared, - first: Either, Output>, - rest_spends: Vec>, + anchor: AnchorV::Shared, + first: Either, Output>, + rest_spends: Vec>, rest_outputs: Vec, binding_sig: Signature, } @@ -131,9 +131,9 @@ struct sapling::ShieldedData { Sapling spend code is located at `zebra-chain/src/sapling/spend.rs`. We use `AnchorVariant` to model the anchor differences between V4 and V5: ```rust -struct Spend { +struct Spend { cv: commitment::ValueCommitment, - anchor: Anchor::PerSpend, + anchor: AnchorV::PerSpend, nullifier: note::Nullifier, rk: redjubjub::VerificationKeyBytes, zkproof: Groth16Proof, From 8d7ea8071a2d397dd80813bf0fb56092ea37d532 Mon Sep 17 00:00:00 2001 From: teor Date: Tue, 23 Mar 2021 09:03:44 +1000 Subject: [PATCH 34/36] Specify a file for each new type --- book/src/dev/rfcs/0010-v5-transaction.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index b3548e9ad84..429f9d50e77 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -64,6 +64,8 @@ Zebra enums and structs put fields in serialized order. Composite fields are ord We know by protocol (2nd table of [Transaction Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#txnencodingandconsensus)) that V5 transactions will support sapling data however we also know by protocol that spends ([Spend Description Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#spendencodingandconsensus), See †) and outputs ([Output Description Encoding and Consensus](https://zips.z.cash/protocol/nu5.pdf#outputencodingandconsensus), See †) fields change from V4 to V5. +`ShieldedData` is currently defined and implemented in `zebra-chain/src/transaction/shielded_data.rs`. As this is Sapling specific we propose to move this file to `zebra-chain/src/sapling/shielded_data.rs`. + ### Changes to V4 Transactions [changes-to-v4-transactions]: #changes-to-v4-transactions @@ -87,7 +89,7 @@ enum Transaction::V4 { ### Anchor Variants [anchor-variants]: #anchor-variants -We implement an `AnchorVariant` generic type trait, because V4 transactions have a per-`Spend` anchor, but V5 transactions have a shared anchor: +We add an `AnchorVariant` generic type trait, because V4 transactions have a per-`Spend` anchor, but V5 transactions have a shared anchor. This trait can be added to `sapling/shielded_data.rs`: ```rust struct PerSpendAnchor {} @@ -112,7 +114,7 @@ trait AnchorVariant { ### Changes to Sapling ShieldedData [changes-to-sapling-shieldeddata]: #changes-to-sapling-shieldeddata -`ShieldedData` is currently defined and implemented in `zebra-chain/src/transaction/shielded_data.rs`. As this is Sapling specific we propose to move this file to `zebra-chain/src/sapling/shielded_data.rs`. We use `AnchorVariant` to model the anchor differences between V4 and V5: +We use `AnchorVariant` in `ShieldedData` to model the anchor differences between V4 and V5: ```rust struct sapling::ShieldedData { @@ -183,7 +185,7 @@ To model the V5 anchor type, `sapling_shielded_data` uses the `SharedAnchor` var ### Adding Orchard ShieldedData [adding-orchard-shieldeddata]: #adding-orchard-shieldeddata -The new V5 structure will create a new `orchard::ShieldedData` type. This new type will be defined in a separated file: `zebra-chain/src/orchard/shielded_data.rs` and it will look as follows: +The new V5 structure will create a new `orchard::ShieldedData` type. This new type will be defined in a new `zebra-chain/src/orchard/shielded_data.rs` file: ```rust struct orchard::ShieldedData { @@ -206,7 +208,7 @@ The fields are ordered based on the **last** data deserialized for each field. ### Adding Orchard AuthorizedAction [adding-orchard-authorizedaction]: #adding-orchard-authorizedaction -In `V5` transactions, there is one `SpendAuth` signature for every `Action`. To ensure that this structural rule is followed, we create an `AuthorizedAction` type: +In `V5` transactions, there is one `SpendAuth` signature for every `Action`. To ensure that this structural rule is followed, we create an `AuthorizedAction` type in `orchard/shielded_data.rs`: ```rust /// An authorized action description. @@ -235,6 +237,8 @@ bitflags! { } ``` +This type is also defined in `orchard/shielded_data.rs`. + ## Test Plan [test-plan]: #test-plan From f7a8c02d13966bc8f8a879c12454edfb097ca514 Mon Sep 17 00:00:00 2001 From: teor Date: Wed, 24 Mar 2021 09:21:55 +1000 Subject: [PATCH 35/36] Document how the Orchard flags are used --- book/src/dev/rfcs/0010-v5-transaction.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index 429f9d50e77..47e7bca201d 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -229,8 +229,16 @@ Finally, in the V5 transaction we have a new `orchard::Flags` type. This is a bi ```rust bitflags! { + /// Per-Transaction flags for Orchard. + /// + /// The spend and output flags are passed to the `Halo2Proof` verifier, which implements + /// the relevant note spending and creation consensus rules. struct orchard::Flags: u8 { + /// Enable spending non-zero valued Orchard notes. + /// + /// "the `enableSpendsOrchard` flag, if present, MUST be 0 for coinbase transactions" const ENABLE_SPENDS = 0b00000001; + /// Enable creating new non-zero valued Orchard notes. const ENABLE_OUTPUTS = 0b00000010; // Reserved, zeros (bits 2 .. 7) } From 867c1cb3c39f9de6bd66b57fdc328a67be4bceff Mon Sep 17 00:00:00 2001 From: teor Date: Wed, 24 Mar 2021 09:32:19 +1000 Subject: [PATCH 36/36] Fix: the proof verifies rules --- book/src/dev/rfcs/0010-v5-transaction.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/book/src/dev/rfcs/0010-v5-transaction.md b/book/src/dev/rfcs/0010-v5-transaction.md index 47e7bca201d..e7b2df80ce4 100644 --- a/book/src/dev/rfcs/0010-v5-transaction.md +++ b/book/src/dev/rfcs/0010-v5-transaction.md @@ -231,7 +231,7 @@ Finally, in the V5 transaction we have a new `orchard::Flags` type. This is a bi bitflags! { /// Per-Transaction flags for Orchard. /// - /// The spend and output flags are passed to the `Halo2Proof` verifier, which implements + /// The spend and output flags are passed to the `Halo2Proof` verifier, which verifies /// the relevant note spending and creation consensus rules. struct orchard::Flags: u8 { /// Enable spending non-zero valued Orchard notes.