feat: global state root 🔥

[EvolveArt]

Pull Request type

Please add the labels corresponding to the type of changes your PR introduces:

• Bugfix
  X Feature
• Code style update (formatting, renaming)
• Refactoring (no functional changes, no API changes)
• Build-related changes
• Documentation content changes
• Testing
• Other (please describe):

What is the current behavior?

Resolves: #166

What is the new behavior?

• Adds SCALE encoding for the StateCommitmentTree structure
• Adds State struct in runtime which contains the two StateCommitmentTree : classes/contracts trees.
• Upgrades a few dependencies to latest version scale-info, scale-coded and bitvec
• Modify BlockifierStateAdapter such that it updates the state trees accordingly.
• Implements IntermediateStateRoot to compute the global state root from the intermediate trees.

Does this introduce a breaking change?

Yes

Other information

• A few things still need to be investigated :

1. Is it fine to store these trees within the runtime ? Discussed with Lucas and spent a few hours thinking about # approaches but it seems unavoidable.
2. When should the tree updates happen ? Incrementally like in this implementation or all at the same time, If we take into account the Storage Overlay (caching layer of substrate) then reading through all the state and building the trees shouldn't be that expensive ?
3. How can we SCALE encode RefCell ? Does it even make sense or should we replace by Box ?
4. BitVec has no MaxEncodedLen which makes sense but prevents from storing it within the runtime, so we can either create some BoundedBitVec wrapper or switch to BoundedVec<bool> for instance.

[tdelabro]

Does computing it in a single go produces the exact same result that tax by tx?

I can't think of any drawbacks and see a lot of pros as it allows us to skip storing all the intermediate states.

[tdelabro]

Ideally (in a few months) this should be computed outside of the runtime, and maybe for block b-1.

So I would advise for the solution that is as decoupled as possible from substrate logic.

[tdelabro]

You can disable the scale max length constraint. Either for one single storage or for the whole pallet.

It should not cause any trouble, especially if we kill the storage at the end of the block execution

[EvolveArt]

Does computing it in a single go produces the exact same result that tax by tx?

I can't think of any drawbacks and see a lot of pros as it allows us to skip storing all the intermediate states.

I mean remember what we we did originally to compute the storage we read everything from storage at each block and that was slowing down a lot.

What do we need here to compute tu global state root ?

• read all the storage keys/values
• read all contract classes along with their class hashes
• read all the nonces along with their contract addresses

Having an iterative process sounds more performant than this as reads are O(log n) without cache

[tdelabro]

The problem was we were reading everything every tx. Now we read only what we need, but at each tx.

If we do this at the end in onfinalize rather than along the execution, it will be exactly the same amount of computing. Won't it ?
And we will spare the burden of storing/loading/making compatible the each transitive state.

[EvolveArt]

Following some discussion with @tdelabro and Mikro from Equilibrium who wrote the initial MPT implementation, I decided to replace RefCell by Box within the MPT implementation. Thus, it is SCALE encodable!

Second thing, I had to do a PR to scale-info repo in order to add a TypeInfo implementation for Rc, while the PR is not merged we can use my fork.

With these two things in place the trees are now persisted through time and we can update them iteratively, this sounds like the best approach until we adopt a client-first approach in the next few months.

[EvolveArt]

.

I'm the boss