Generate test chains that pass basic chain consistency tests #2221

teor2345 · 2021-05-28T05:04:11Z

Motivation

In #2185, we want to generate random block chains that pass basic chain consistency tests, like:

starting with the genesis height and previous block hash
adding 1 to the previous height
making the previous block hash match the previous block's header

Solution

Redesign how we generate arbitrary blocks and chains
Allow partial chains to start at genesis, rather than block 1
Generate partial chains with correct previous block hashes
Provide the network as part of the prepared chain value

The code in this pull request has:

Documentation Comments
New Property Tests

Review

@oxarbitrage can review this PR for #2185

And cleanup the ledger strategy interface.

teor2345 · 2021-05-28T05:27:57Z

zebra-chain/src/block/arbitrary.rs

+        // after the vec strategy generates blocks, update the previous block hashes
+        vec.prop_map(|mut vec| {
+            let mut previous_block_hash = None;
+            for block in vec.iter_mut() {
+                if let Some(previous_block_hash) = previous_block_hash {
+                    block.header.previous_block_hash = previous_block_hash;
+                }
+                previous_block_hash = Some(block.hash());
+            }
+            vec.into_iter().map(Arc::new).collect()
+        })
+        .boxed()


This change fixes up the previous block hashes

teor2345 · 2021-05-28T05:28:23Z

zebra-chain/src/block/arbitrary.rs

+/// Overrides for arbitrary [`LedgerState`]s.
+#[derive(Debug, Clone, Copy)]
+pub struct LedgerStateOverride {
+    /// Regardless of tip height and network, every block has features from this
+    /// network upgrade.
+    pub network_upgrade_override: Option<NetworkUpgrade>,
+
+    /// Every block has exactly one coinbase transaction.
+    /// Transactions are always coinbase transactions.
+    pub always_has_coinbase: bool,
+
+    /// Every chain starts at this block. Single blocks have this height.
+    pub height_override: Option<Height>,
+
+    /// Every chain starts with a block with this previous block hash.
+    /// Single blocks have this previous block hash.
+    pub previous_block_hash_override: Option<block::Hash>,
 }


This change makes it easier to generate different kinds of chains.

teor2345 · 2021-05-28T05:28:50Z

zebra-chain/src/block/arbitrary.rs

+                    if let Some(previous_block_hash_override) =
+                        ledger_state.previous_block_hash_override
+                    {
+                        previous_block_hash = previous_block_hash_override;
+                    } else if ledger_state.height == Height(0) {
+                        previous_block_hash = GENESIS_PREVIOUS_BLOCK_HASH;
+                    }


This code move makes header generation consistent with block generation

teor2345 · 2021-05-28T05:29:19Z

zebra-chain/src/block/tests/prop.rs

+/// Make sure our genesis strategy generates blocks with the correct coinbase
+/// height and previous block hash.
+#[test]
+fn block_genesis_strategy() -> Result<()> {
+    zebra_test::init();
+
+    let strategy = LedgerState::genesis_strategy(None).prop_flat_map(Block::arbitrary_with);
+
+    proptest!(|(block in strategy)| {
+        prop_assert_eq!(block.coinbase_height(), Some(Height(0)));
+        prop_assert_eq!(block.header.previous_block_hash, GENESIS_PREVIOUS_BLOCK_HASH);
+    });
+
+    Ok(())
+}
+
+/// Make sure our partial chain strategy generates a chain with the correct coinbase
+/// heights and previous block hashes.
+#[test]
+fn partial_chain_strategy() -> Result<()> {
+    zebra_test::init();
+
+    let strategy = LedgerState::genesis_strategy(None)
+        .prop_flat_map(|init| Block::partial_chain_strategy(init, 3));
+
+    proptest!(|(chain in strategy)| {
+        let mut height = Height(0);
+        let mut previous_block_hash = GENESIS_PREVIOUS_BLOCK_HASH;
+        for block in chain {
+            prop_assert_eq!(block.coinbase_height(), Some(height));
+            prop_assert_eq!(block.header.previous_block_hash, previous_block_hash);
+            height = Height(height.0 + 1);
+            previous_block_hash = block.hash();
+        }
+    });
+
+    Ok(())
+}


These tests make sure the new test code actually works

teor2345 · 2021-05-28T05:29:35Z

zebra-chain/src/transparent/arbitrary.rs

-            let height = block::Height(ledger_state.tip_height.0 + 1);
-            Self::arbitrary_with(Some(height))
+            Self::arbitrary_with(Some(ledger_state.height))


This change lets us start generating blocks at genesis, rather than block 1

teor2345 · 2021-05-28T05:30:03Z

zebra-state/src/service/non_finalized_state/arbitrary.rs

@@ -12,19 +12,24 @@ use crate::tests::Prepare;

 use super::*;

-const MAX_PARTIAL_CHAIN_BLOCKS: usize = 100;
+const MAX_PARTIAL_CHAIN_BLOCKS: usize = 102;


This change makes sure we go past the non-finalized state limit

teor2345 · 2021-05-28T05:30:32Z

zebra-state/src/service/non_finalized_state/arbitrary.rs

+    type Value = (
+        Arc<Vec<PreparedBlock>>,
+        <BinarySearch as ValueTree>::Value,
+        Network,
+    );


This change lets us use the right network to initialize the finalized state

teor2345 · 2021-05-28T05:31:00Z

zebra-state/src/service/non_finalized_state/arbitrary.rs

+            // Disable NU5 for now
+            // `genesis_strategy(None)` re-enables the default Nu5 override
+            let ledger_strategy = LedgerState::genesis_strategy(Canopy);


@oxarbitrage this is the NU5 change you'll need to add in your PR after you merge

oxarbitrage

This looks great, I am merging as is so we can continue the work in #2185.
Will make modifications in another PR if needed.

teor2345 added 3 commits May 28, 2021 14:02

Set the tip height and previous hash for arbitrary genesis blocks

3bcaa5d

And cleanup the ledger strategy interface.

Generate partial chains with correct previous block hashes

7ac42b1

Provide the network value from the PreparedChain strategy

ace8b2c

teor2345 added A-rust Area: Updates to Rust code C-enhancement Category: This is an improvement P-Medium labels May 28, 2021

teor2345 added this to the 2021 Sprint 10 milestone May 28, 2021

teor2345 requested a review from oxarbitrage May 28, 2021 05:04

teor2345 self-assigned this May 28, 2021

This was referenced May 28, 2021

Store orchard nullifiers into the state #2185

Merged

Make arbitrary block chains pass some genesis checks #2208

Merged

teor2345 commented May 28, 2021

View reviewed changes

oxarbitrage approved these changes May 28, 2021

View reviewed changes

oxarbitrage merged commit 0b611eb into main May 28, 2021

oxarbitrage deleted the genesis-proptest-height branch May 28, 2021 12:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generate test chains that pass basic chain consistency tests #2221

Generate test chains that pass basic chain consistency tests #2221

teor2345 commented May 28, 2021

teor2345 May 28, 2021

teor2345 May 28, 2021

teor2345 May 28, 2021

teor2345 May 28, 2021

teor2345 May 28, 2021

teor2345 May 28, 2021

teor2345 May 28, 2021

teor2345 May 28, 2021

oxarbitrage left a comment

Generate test chains that pass basic chain consistency tests #2221

Generate test chains that pass basic chain consistency tests #2221

Conversation

teor2345 commented May 28, 2021

Motivation

Solution

Review

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oxarbitrage left a comment

Choose a reason for hiding this comment