Implements concurrent Smt::compute_mutations

[krushimir]

This PR introduces a concurrent implementation of Smt::compute_mutations, leveraging an approach similar to the existing parallel construction logic.

Benchmark results were collected on a 64-core (128-thread) AMD EPYC 7662 processor, with Rayon’s thread pool explicitly limited to the specified thread counts.

For context, construction benchmarks are also included for performance comparison.

1. Construction Benchmark

10k key-value pairs

Threads	Parallel Time (s)	Sequential Time (s)	Speedup
16	0.5	5.7	11.11x
32	0.4	5.7	15.22x
64	0.3	5.7	17.35x
128	0.4	5.7	16.90x

• Optimal performance was achieved with 64 threads.
• Diminishing returns were observed with 128 threads

---

2. Batched Insertion Benchmark

10k key-value pairs

Threads	Parallel Time (ms)	Sequential Time (ms)	Speedup	Avg Insert Time (μs)
16	517.0	6308.7	12.20x	52
32	395.8	6334.5	16.00x	40
64	333.0	6321.6	18.98x	33
128	383.7	6300.7	16.42x	38

• 64 threads offered the best performance, reducing average insertion time to 33 μs.
• Scaling beyond 64 threads led to slight performance degradation.

---

3. Batched Update Benchmark

10k key-value pairs

Threads	Parallel Time (ms)	Sequential Time (ms)	Speedup	Avg Update Time (μs)
16	482.7	6369.8	13.20x	48
32	357.7	6351.5	17.76x	36
64	304.7	6378.5	20.93x	30
128	273.5	6418.8	23.47x	27

• Batched updates scaled better with increased threads.
• 128 threads achieved the fastest update speed, reducing average time to 27 μs.

[PhilippGackstatter]

Looks great to me! I think the logic itself looks good. My comments are mostly about naming, docs and deduplication. I might have to take another look anyway, since I first had to understand how the Smt is implemented in sequential code 😅, so I'll just comment for now.

In general, I think adding comments to code parts that are not easy to understand would improve readability and understandability.

Regarding the approach, please correct me if I have misunderstandings, but my understanding of the approach is the following.

Assuming a tree of depth 64 with subtrees of depth 8 and mutations of just two (for example's sake) leaves at indices 0 and 65536, compute_mutations would do this, on a high-level and making some simple assumptions about how rayon assigns threads:

1. Compute subtrees that were modified. This happens in sorted_pairs_to_mutated_leaves. This would yield two subtrees, covering the column ranges 0..256 and 65536..65792.
2. Then in build_subtree_mutations, the subtrees are updated in parallel.
   • 1st iteration:
     • Thread 0: Compute updates for leaves with indices 0..256 at depth 64. Then updates for leaves at depth 63 within this subtree, and so on, until it eventually results in new root at depth 56, column 0.
     • Thread 1: Compute updates for leaves with indices 65536..65792 at depth 64. Then updates for leaves at depth 63 within this subtree, and so on, until it eventually results in new root at depth 56, column 256 (= 65536 >> 8).
   • 2nd iteration:
     • Thread 0: Compute updates for leaves with indices 0..256 at depth 56 (only root 0 has changed). Eventually this results in a new root at depth 48, column 0.
     • Thread 1: Compute updates for leaves with indices 256..512 at depth 56 (only root 256 has changed). Eventually this results in a new root at depth 48, column 1.
   • 3rd iteration:
     • Thread 0: Compute updates for leaves with indices 0..256 at depth 48 (only root 0 has changed). Eventually this results in a new root at depth 40, column 0.
   • More iterations like the 3rd until the root at depth 0 has been reached.

Is this accurate? Would it make sense to add something like this as a doc comment to compute_mutations_subtree (with corrections if it's inaccurate)?

[krushimir]

10M entries tree.

batch insertions (10k inserts):
without smt_hashmaps: 383.3 ms (~38 μs per insert)
with smt_hashmaps: 281.9 ms (~28 μs per insert)
~26% faster
concurrent vs. sequential: 17.7x faster
concurrent with smt_hashmaps vs. sequential: 24.1x faster

batch updates (10k updates):
without smt_hashmaps: 287.9 ms (~29 μs per update)
without smt_hashmaps: 265.5 ms (~27 μs per update)
~8% faster
concurrent vs. sequential: 23.6x faster
concurrent with smt_hashmaps vs. sequential: 25.6x faster

[PhilippGackstatter]

Hey @krushimir, quick question: Is this still Work-In-Progress or can it be marked as ready for review?

[krushimir]

Hi @PhilippGackstatter, I'll push some more changes today and then I'll mark it ready.

[PhilippGackstatter]

Looks good to me!

[polydez]

Should we use parallel sorting here? par_sort_unstable_by_key

[krushimir]

Thanks for the tip. I did some benchmarking on EPYC 7662 (128 threads) and M1 Pro (10 threads), and for a batch of 10K elements, the benefits are unclear. Nevertheless, I pushed the change.

Also, I've implemented the same improvement into parallel tree construction - currently
tracked here.

[bobbinth]

Looks good! Thank you! I left a couple of comments inline. The main one is about code organization - i.e., potentially moving the parallel mutation functions to the Smt struct.

[bobbinth]

One thing I'm wondering (and maybe this is not a good idea): should we move all the concurrent mutation logic into Smt struct? The thinking is that it applies primarily to Smt use case and while we do support it for SimpleSmts with depth of multiples of 8, arguably, such support is more confusing than helpful.

On the other hand, if we do move the logic to the Smt struct, it will make code comprehension a bit easier. Specifically, compute_mutations_concurrent(), sorted_pairs_to_mutated_subtree_leaves(), build_subtree_mutations(), and fetch_sibling_pair() would move there. The compute_mutations() method in this module would look like:

fn compute_mutations(
    &self,
    kv_pairs: impl IntoIterator<Item = (Self::Key, Self::Value)>,
) -> MutationSet<DEPTH, Self::Key, Self::Value>
where
    Self: Sized + Sync,
    Self::Key: Send + Sync,
    Self::Value: Send + Sync,
{
    self.compute_mutations_sequential(kv_pairs)
}

And then in the Smt struct we'd have the conditional compilation based on the feature flag. We could maybe encapsulate all methods related to concurrency in a separate impl block - e.g.,:

#[cfg(feature = "concurrent")]
impl Smt {
    fn compute_mutations_concurrent(
        &self,
        kv_pairs: impl IntoIterator<Item = (Self::Key, Self::Value)>,
    ) -> MutationSet<DEPTH, Self::Key, Self::Value>
    where
        Self: Sized + Sync,
        Self::Key: Send + Sync,
        Self::Value: Send + Sync,
    {
        ...
    }

    fn sorted_pairs_to_mutated_subtree_leaves(
        ...
    }
}

[krushimir]

I agree with your suggestion. Moving the concurrent mutation logic to the Smt struct makes sense if we're okay with the 8-depth limitation for now.
Would you like me to do the same for parallel construction methods, or leave that for another PR?

[bobbinth]

If it's simple enough, we could move them in this PR as well.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud