[System] Support rebuilding AccumulatorInfo and provides repair tools for reverting transactions

[baichuan3]

Summary

1. Support rebuilding AccumulatorInfo, At the same time, ensure the compatibility of new and old data
2. Provides repair tools for reverting transactions

• Closes [system] Should we put the full AccumulatorInfo to TransactionSequeceInfo #2289

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
rooch-portal	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Aug 2, 2024 0:14am

1 Skipped Deployment

Name	Status	Preview	Comments	Updated (UTC)
rooch	⬜️ Ignored (Inspect)	Visit Preview		Aug 2, 2024 0:14am

[github-actions]

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

OpenSSF Scorecard

Package	Version	Score	Details

Scanned Manifest Files

[baichuan3]

I need some help when ensure the compatibility of new and old data, after add new fields in the Struct @jolestar

the key problem is that the old data does not have the Option flag in the bcs serialization. When parsing by field, the old data returns empty instead of None, causing the deserialization to fail.

1. Solution A encountered an exception: unexpected end of input.
   The reason is that in the bcs standard format, Option will be serialized as vec![0], and the old data does not have the 0 flag, resulting in deserialization failure, even though the newly added field is defined as #[serde(default)]

2. The problem of Solution B : It is expected that in visit_seq, the three new fields tx_accumulator_frozen_subtree_roots, tx_accumulator_num_leaves, and tx_accumulator_num_nodes will be returned as None when deserializing old data. In fact, next_element() returns a value, but the value is empty, resulting in deserialization failure. The corresponding code is let tx_accumulator_frozen_subtree_roots: Vec<H256> = seq.next_element().unwrap_or(vec![]);

PS1. The solution A:

• Based on TransactionSequenceInfo Wrapper, Deserialize and Serialize of TransactionSequenceInfo are re-implemented
• AccumulatorInfo is defined as Option, allowing it to be absent (like in old data) or present with a value (like in new data).
• Use #[serde(default)] to provide a default value when deserializing older data that doesn't include AccumulatorInfo.
• Serde's Option handling ensures compatibility between old and new formats seamlessly.

///`TransactionSequenceInfo` represents the result of sequence a transaction.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct TransactionSequenceInfo {
    /// The tx order
    pub tx_order: u64,
    /// The tx order signature, it is the signature of the sequencer to commit the tx order.
    pub tx_order_signature: Vec<u8>,
    /// The tx accumulator root after the tx is append to the accumulator.
    pub tx_accumulator_root: H256,
    /// The tx accumulator info after the tx is append to the accumulator.
    // pub tx_accumulator_info: Option<AccumulatorInfo>,
    /// The timestamp of the sequencer when the tx is sequenced, in millisecond.
    pub tx_timestamp: u64,

    /// Frozen subtree roots of the accumulator.
    // #[serde(default)]
    pub tx_accumulator_frozen_subtree_roots: Vec<H256>,
    /// The total number of leaves in the accumulator.
    // #[serde(default)]
    pub tx_accumulator_num_leaves: u64,
    /// The total number of nodes in the accumulator.
    // #[serde(default)]
    pub tx_accumulator_num_nodes: u64,
}

#[derive(Clone, Debug, Serialize, Deserialize)]
struct TransactionSequenceInfoWrapper {
    tx_order: u64,
    tx_order_signature: Vec<u8>,
    tx_accumulator_root: H256,
    tx_timestamp: u64,

    #[serde(default)]
    tx_accumulator_info: Option<AccumulatorInfo>,
}

impl Serialize for TransactionSequenceInfo {
    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
    where
        S: Serializer,
    {
        let tx_accumulator_info = AccumulatorInfo::new(
            self.tx_accumulator_root,
            self.tx_accumulator_frozen_subtree_roots.clone(),
            self.tx_accumulator_num_leaves,
            self.tx_accumulator_num_nodes,
        );
        let wrapper = TransactionSequenceInfoWrapper {
            tx_order: self.tx_order,
            tx_order_signature: self.tx_order_signature.clone(),
            tx_accumulator_root: self.tx_accumulator_root,
            tx_timestamp: self.tx_timestamp,
            tx_accumulator_info: Some(tx_accumulator_info),
        };

        wrapper.serialize(serializer)
    }
}

impl<'de> Deserialize<'de> for TransactionSequenceInfo {
    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
    where
        D: Deserializer<'de>,
    {
        println!("[Debug] Deserialize deserialize start");
        let wrapper = TransactionSequenceInfoWrapper::deserialize(deserializer)?;
        println!("[Debug] Deserialize deserialize 01");
        let tx_accumulator_info = wrapper.tx_accumulator_info.unwrap_or_default();
        Ok(TransactionSequenceInfo {
            tx_order: wrapper.tx_order,
            tx_order_signature: wrapper.tx_order_signature,
            tx_accumulator_root: wrapper.tx_accumulator_root,
            tx_timestamp: wrapper.tx_timestamp,
            tx_accumulator_frozen_subtree_roots: tx_accumulator_info.frozen_subtree_roots,
            tx_accumulator_num_leaves: tx_accumulator_info.num_leaves,
            tx_accumulator_num_nodes: tx_accumulator_info.num_nodes,
        })
    }
}

PS2. The solution B:

• Deserialize of TransactionSequenceInfo are re-implemented
• The default serializer format of TransactionSequenceInfo is serialize.serialize_struct, so use deserializer.deserialize_struct to customize deserialization

///`TransactionSequenceInfo` represents the result of sequence a transaction.
#[derive(Clone, Debug, Eq, PartialEq, Serialize)]
pub struct TransactionSequenceInfo {
    /// The tx order
    pub tx_order: u64,
    /// The tx order signature, it is the signature of the sequencer to commit the tx order.
    pub tx_order_signature: Vec<u8>,
    /// The tx accumulator root after the tx is append to the accumulator.
    pub tx_accumulator_root: H256,
    /// The tx accumulator info after the tx is append to the accumulator.
    // pub tx_accumulator_info: Option<AccumulatorInfo>,
    /// The timestamp of the sequencer when the tx is sequenced, in millisecond.
    pub tx_timestamp: u64,

    /// Frozen subtree roots of the accumulator.
    // #[serde(default)]
    pub tx_accumulator_frozen_subtree_roots: Vec<H256>,
    /// The total number of leaves in the accumulator.
    // #[serde(default)]
    pub tx_accumulator_num_leaves: u64,
    /// The total number of nodes in the accumulator.
    // #[serde(default)]
    pub tx_accumulator_num_nodes: u64,
}

// Implement custom Deserialize for TransactionSequenceInfo
impl<'de> Deserialize<'de> for TransactionSequenceInfo {
    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
    where
        D: Deserializer<'de>,
    {
        struct TransactionSequenceInfoVisitor;

        impl<'de> Visitor<'de> for TransactionSequenceInfoVisitor {
            type Value = TransactionSequenceInfo;

            fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
                formatter.write_str("Expect TransactionSequenceInfo Struct")
            }

            fn visit_seq<V>(self, mut seq: V) -> Result<TransactionSequenceInfo, V::Error>
            where
                V: SeqAccess<'de>,
            {
                let size_hint = seq.size_hint();
                let tx_order: u64 = seq.next_element()?.ok_or_else(|| serde::de::Error::custom("Missing or invalid tx_order field when deserialize TransactionSequenceInfo"))?;
                let tx_accumulator_root: H256 = seq.next_element()?.ok_or_else(|| serde::de::Error::custom("Missing or invalid tx_accumulator_root field when deserialize TransactionSequenceInfo"))?;
                let tx_timestamp: u64 = seq.next_element()?.ok_or_else(|| serde::de::Error::custom("Missing or invalid tx_timestamp field when deserialize TransactionSequenceInfo"))?;
                let tx_accumulator_frozen_subtree_roots = if seq.size_hint().unwrap_or(0) > 0 {
                    let tx_accumulator_frozen_subtree_roots: Vec<H256> = seq.next_element().unwrap_or(vec![]);
                        tx_accumulator_frozen_subtree_roots
                } else {
                    vec![]
                };
                let tx_accumulator_num_leaves = if seq.size_hint().unwrap_or(0) > 0 {
                    seq.next_element()?.ok_or_else(|| serde::de::Error::custom("Missing or invalid tx_accumulator_num_leaves field when deserialize TransactionSequenceInfo"))?
                } else {
                    0u64
                };
                let tx_accumulator_num_nodes = if seq.size_hint().unwrap_or(0) > 0 {
                    seq.next_element()?.ok_or_else(|| serde::de::Error::custom("Missing or invalid tx_accumulator_num_nodes field when deserialize TransactionSequenceInfo"))?
                } else {
                    0u64
                };

                Ok(TransactionSequenceInfo {
                    tx_order,
                    tx_order_signature,
                    tx_accumulator_root,
                    tx_timestamp,
                    tx_accumulator_frozen_subtree_roots,
                    tx_accumulator_num_leaves,
                    tx_accumulator_num_nodes,
                })
            }
        }

        deserializer.deserialize_struct(
            TRANSACTION_SEQUENCE_INFO_STR,
            TRANSACTION_SEQUENCE_INFO_FIELDS,
            TransactionSequenceInfoVisitor,
        )
    }
}