Merge branch 'main' into non-finalized-state-finalization

README.md

@@ -6,22 +6,93 @@
 [![codecov](https://codecov.io/gh/ZcashFoundation/zebra/branch/main/graph/badge.svg)](https://codecov.io/gh/ZcashFoundation/zebra)
 ![License](https://img.shields.io/badge/license-MIT%2FApache--2.0-blue.svg)
 
-Hello! I am Zebra, an ongoing Rust implementation of a Zcash node.
+### 🚧 UNDER CONSTRUCTION 🚧
 
-Zebra is a work in progress.  It is developed as a collection of `zebra-*`
-libraries implementing the different components of a Zcash node (networking,
-chain structures, consensus rules, etc), and a `zebrad` binary which uses them.
+[Zebra](https://zebra.zfnd.org/) is the Zcash Foundation's independent,
+consensus-compatible implementation of the Zcash protocol, currently under
+development.  Please [join us on Discord](https://discord.gg/na6QZNd) if you'd
+like to find out more or get involved!
 
-Most of our work so far has gone into `zebra-network`, building a new
-networking stack for Zcash, `zebra-chain`, building foundational data
-structures, `zebra-consensus`, implementing consensus rules, and
-`zebra-state`, providing chain state.
+Unlike `zcashd`, which originated as a Bitcoin Core fork and inherited its
+monolithic architecture, Zebra has a modular, library-first design, with the
+intent that each component can be independently reused outside of the `zebrad`
+fullnode.  For instance, the `zebra-network` crate containing the network stack
+can also be used to implement anonymous transaction relay, network crawlers, or
+other functionality, without requiring a full node.
 
-[Zebra Website](https://zebra.zfnd.org).
+Our first goal is to be able to participate in the network and replicate the
+Zcash chain state, and we intend to ship an alpha before the end of 2020 with
+this functionality.  In 2021, we intend to add RPC support and wallet
+integration.  This phased approach allows us to test the independent
+implementation of the consensus rules before asking users to entrust it with
+their funds.
 
-[Rendered docs from the `main` branch](https://doc.zebra.zfnd.org).
+At a high level, the fullnode functionality required by `zebrad` is factored
+into several components:
 
-[Join us on Discord](https://discord.gg/na6QZNd).
+- [`zebra-chain`](https://doc.zebra.zfnd.org/zebra_chain/index.html), providing
+  definitions of core data structures for Zcash, such as blocks, transactions,
+  addresses, etc., and related functionality.  It also contains the
+  implementation of the consensus-critical serialization formats used in Zcash.
+  The data structures in `zebra-chain` are defined to enforce [*structural
+  validity*](https://zebra.zfnd.org/dev/rfcs/0002-parallel-verification.html#verification-stages) by making invalid states unrepresentable.  For instance, the
+  `Transaction` enum has variants for each transaction version, and it's
+  impossible to construct a transaction with, e.g., spend or output
+  descriptions but no binding signature, or, e.g., a version 2 (Sprout)
+  transaction with Sapling proofs.  Currently, `zebra-chain` is oriented
+  towards verifying transactions, but will be extended to support creating them
+  in the future.
+
+- [`zebra-network`](https://doc.zebra.zfnd.org/zebra_network/index.html),
+  providing an asynchronous, multithreaded implementation of the Zcash network
+  protocol inherited from Bitcoin.  In contrast to `zcashd`, each peer
+  connection has a separate state machine, and the crate translates the
+  external network protocol into a stateless, request/response-oriented
+  protocol for internal use.  The crate provides two interfaces: an
+  auto-managed connection pool that load-balances requests over available
+  peers, and a `connect_isolated` method that produces a peer connection
+  completely isolated from all other node state.  This can be used, for
+  instance, to safely relay data over Tor, without revealing distinguishing
+  information.
+
+- [`zebra-script`](https://doc.zebra.zfnd.org/zebra_script/index.html) provides
+  script validation.  Currently, this is implemented by linking to the C++
+  script verification code from `zcashd`, but in the future we may implement a
+  pure-Rust script implementation.
+
+- [`zebra-consensus`](https://doc.zebra.zfnd.org/zebra_consensus/index.html)
+  performs [*semantic validation*](https://zebra.zfnd.org/dev/rfcs/0002-parallel-verification.html#verification-stages) of blocks and transactions: all consensus
+  rules that can be checked independently of the chain state, such as
+  verification of signatures, proofs, and scripts.  Internally, the library
+  uses [`tower-batch`](https://doc.zebra.zfnd.org/tower_batch/index.html) to
+  perform automatic, transparent batch processing of contemporaneous
+  verification requests.
+
+- [`zebra-state`](https://doc.zebra.zfnd.org/zebra_state/index.html) is
+  responsible for storing, updating, and querying the chain state.  The state
+  service is responsible for [*contextual verification*](https://zebra.zfnd.org/dev/rfcs/0002-parallel-verification.html#verification-stages): all consensus rules
+  that check whether a new block is a valid extension of an existing chain,
+  such as updating the nullifier set or checking that transaction inputs remain
+  unspent.
+
+- [`zebrad`](https://doc.zebra.zfnd.org/zebrad/index.html) contains the full
+  node, which connects these components together and implements logic to handle
+  inbound requests from peers and the chain sync process.
+
+- `zebra-rpc` and `zebra-client` will eventually contain the RPC and wallet
+  functionality, but as mentioned above, our goal is to implement replication
+  of chain state first before asking users to entrust Zebra with their funds.
+
+All of these components can be reused as independent libraries, and all
+communication between stateful components is handled internally by
+[internal asynchronous RPC abstraction](https://docs.rs/tower/) ("microservices in one process").
+
+The [Zebra website](https://zebra.zfnd.org/) contains user documentation, such
+as how to run or configure Zebra, set up metrics integrations, etc., as well as
+developer documentation, such as design documents.  We also render [API
+documentation](https://doc.zebra.zfnd.org) for the external API of our crates,
+as well as [internal documentation](https://doc-internal.zebra.zfnd.org) for
+private APIs.
 
 ## License
 

tower-batch/Cargo.toml

@@ -12,7 +12,7 @@ futures-core = "0.3.6"
 pin-project = "0.4.26"
 tracing = "0.1.21"
 tracing-futures = "0.2.4"
-futures = "0.3.5"
+futures = "0.3.6"
 
 [dev-dependencies]
 ed25519-zebra = "2.1.0"

zebra-chain/src/sapling/commitment.rs

@@ -135,6 +135,29 @@ impl NoteCommitment {
 #[derive(Clone, Copy, Deserialize, PartialEq, Serialize)]
 pub struct ValueCommitment(#[serde(with = "serde_helpers::AffinePoint")] pub jubjub::AffinePoint);
 
+impl<'a> std::ops::Add<&'a ValueCommitment> for ValueCommitment {
+    type Output = Self;
+
+    fn add(self, rhs: &'a ValueCommitment) -> Self::Output {
+        self + *rhs
+    }
+}
+
+impl std::ops::Add<ValueCommitment> for ValueCommitment {
+    type Output = Self;
+
+    fn add(self, rhs: ValueCommitment) -> Self::Output {
+        let value = self.0.to_extended() + rhs.0.to_extended();
+        ValueCommitment(value.into())
+    }
+}
+
+impl std::ops::AddAssign<ValueCommitment> for ValueCommitment {
+    fn add_assign(&mut self, rhs: ValueCommitment) {
+        *self = *self + rhs
+    }
+}
+
 impl fmt::Debug for ValueCommitment {
     fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
         f.debug_struct("ValueCommitment")
@@ -162,6 +185,40 @@ impl From<ValueCommitment> for [u8; 32] {
     }
 }
 
+impl<'a> std::ops::Sub<&'a ValueCommitment> for ValueCommitment {
+    type Output = Self;
+
+    fn sub(self, rhs: &'a ValueCommitment) -> Self::Output {
+        self - *rhs
+    }
+}
+
+impl std::ops::Sub<ValueCommitment> for ValueCommitment {
+    type Output = Self;
+
+    fn sub(self, rhs: ValueCommitment) -> Self::Output {
+        ValueCommitment((self.0.to_extended() - rhs.0.to_extended()).into())
+    }
+}
+
+impl std::ops::SubAssign<ValueCommitment> for ValueCommitment {
+    fn sub_assign(&mut self, rhs: ValueCommitment) {
+        *self = *self - rhs;
+    }
+}
+
+impl std::iter::Sum for ValueCommitment {
+    fn sum<I>(iter: I) -> Self
+    where
+        I: Iterator<Item = Self>,
+    {
+        iter.fold(
+            ValueCommitment(jubjub::AffinePoint::identity()),
+            std::ops::Add::add,
+        )
+    }
+}
+
 /// LEBS2OSP256(repr_J(cv))
 ///
 /// https://zips.z.cash/protocol/protocol.pdf#spendencoding
@@ -197,14 +254,22 @@ impl ValueCommitment {
     /// Generate a new _ValueCommitment_.
     ///
     /// https://zips.z.cash/protocol/protocol.pdf#concretehomomorphiccommit
-    #[allow(non_snake_case)]
-    pub fn new<T>(csprng: &mut T, value: Amount<NonNegative>) -> Self
+    pub fn randomized<T>(csprng: &mut T, value: Amount) -> Self
     where
         T: RngCore + CryptoRng,
     {
-        let v = jubjub::Fr::from(value);
         let rcv = generate_trapdoor(csprng);
 
+        Self::new(rcv, value)
+    }
+
+    /// Generate a new _ValueCommitment_ from an existing _rcv_ on a _value_.
+    ///
+    /// https://zips.z.cash/protocol/protocol.pdf#concretehomomorphiccommit
+    #[allow(non_snake_case)]
+    pub fn new(rcv: jubjub::Fr, value: Amount) -> Self {
+        let v = jubjub::Fr::from(value);
+
         // TODO: These generator points can be generated once somewhere else to
         // avoid having to recompute them on every new commitment.
         let V = find_group_hash(*b"Zcash_cv", b"v");
@@ -217,6 +282,8 @@ impl ValueCommitment {
 #[cfg(test)]
 mod tests {
 
+    use std::ops::Neg;
+
     use super::*;
 
     #[test]
@@ -232,4 +299,129 @@ mod tests {
             assert_eq!(result, test_vector.output_point);
         }
     }
+
+    #[test]
+    fn add() {
+        let identity = ValueCommitment(jubjub::AffinePoint::identity());
+
+        let g = ValueCommitment(jubjub::AffinePoint::from_raw_unchecked(
+            jubjub::Fq::from_raw([
+                0xe4b3_d35d_f1a7_adfe,
+                0xcaf5_5d1b_29bf_81af,
+                0x8b0f_03dd_d60a_8187,
+                0x62ed_cbb8_bf37_87c8,
+            ]),
+            jubjub::Fq::from_raw([
+                0x0000_0000_0000_000b,
+                0x0000_0000_0000_0000,
+                0x0000_0000_0000_0000,
+                0x0000_0000_0000_0000,
+            ]),
+        ));
+
+        assert_eq!(identity + g, g);
+    }
+
+    #[test]
+    fn add_assign() {
+        let mut identity = ValueCommitment(jubjub::AffinePoint::identity());
+
+        let g = ValueCommitment(jubjub::AffinePoint::from_raw_unchecked(
+            jubjub::Fq::from_raw([
+                0xe4b3_d35d_f1a7_adfe,
+                0xcaf5_5d1b_29bf_81af,
+                0x8b0f_03dd_d60a_8187,
+                0x62ed_cbb8_bf37_87c8,
+            ]),
+            jubjub::Fq::from_raw([
+                0x0000_0000_0000_000b,
+                0x0000_0000_0000_0000,
+                0x0000_0000_0000_0000,
+                0x0000_0000_0000_0000,
+            ]),
+        ));
+
+        identity += g;
+        let new_g = identity;
+
+        assert_eq!(new_g, g);
+    }
+
+    #[test]
+    fn sub() {
+        let g_point = jubjub::AffinePoint::from_raw_unchecked(
+            jubjub::Fq::from_raw([
+                0xe4b3_d35d_f1a7_adfe,
+                0xcaf5_5d1b_29bf_81af,
+                0x8b0f_03dd_d60a_8187,
+                0x62ed_cbb8_bf37_87c8,
+            ]),
+            jubjub::Fq::from_raw([
+                0x0000_0000_0000_000b,
+                0x0000_0000_0000_0000,
+                0x0000_0000_0000_0000,
+                0x0000_0000_0000_0000,
+            ]),
+        );
+
+        let identity = ValueCommitment(jubjub::AffinePoint::identity());
+
+        let g = ValueCommitment(g_point);
+
+        assert_eq!(identity - g, ValueCommitment(g_point.neg()));
+    }
+
+    #[test]
+    fn sub_assign() {
+        let g_point = jubjub::AffinePoint::from_raw_unchecked(
+            jubjub::Fq::from_raw([
+                0xe4b3_d35d_f1a7_adfe,
+                0xcaf5_5d1b_29bf_81af,
+                0x8b0f_03dd_d60a_8187,
+                0x62ed_cbb8_bf37_87c8,
+            ]),
+            jubjub::Fq::from_raw([
+                0x0000_0000_0000_000b,
+                0x0000_0000_0000_0000,
+                0x0000_0000_0000_0000,
+                0x0000_0000_0000_0000,
+            ]),
+        );
+
+        let mut identity = ValueCommitment(jubjub::AffinePoint::identity());
+
+        let g = ValueCommitment(g_point);
+
+        identity -= g;
+        let new_g = identity;
+
+        assert_eq!(new_g, ValueCommitment(g_point.neg()));
+    }
+
+    #[test]
+    fn sum() {
+        let g_point = jubjub::AffinePoint::from_raw_unchecked(
+            jubjub::Fq::from_raw([
+                0xe4b3_d35d_f1a7_adfe,
+                0xcaf5_5d1b_29bf_81af,
+                0x8b0f_03dd_d60a_8187,
+                0x62ed_cbb8_bf37_87c8,
+            ]),
+            jubjub::Fq::from_raw([
+                0x0000_0000_0000_000b,
+                0x0000_0000_0000_0000,
+                0x0000_0000_0000_0000,
+                0x0000_0000_0000_0000,
+            ]),
+        );
+
+        let g = ValueCommitment(g_point);
+        let other_g = ValueCommitment(g_point);
+
+        let sum: ValueCommitment = vec![g, other_g].into_iter().sum();
+
+        let doubled_g = ValueCommitment(g_point.to_extended().double().into());
+
+        assert_eq!(sum, doubled_g);
+    }
 }