Added more efficient unionWith, eliminate transformation Map to List in Foldable{WithIndex} instances.

[xgrommx]

Here benchmark results

Map
===
size
---------------
size: singleton map
mean   = 1.19 μs
stddev = 2.68 μs
min    = 913.00 ns
max    = 71.86 μs
size: small map (100)
mean   = 6.88 μs
stddev = 34.59 μs
min    = 2.28 μs
max    = 717.62 μs
size: midsize map (10000)
mean   = 211.21 μs
stddev = 137.90 μs
min    = 180.70 μs
max    = 1.52 ms
size: big map (1000000)
mean   = 28.68 ms
stddev = 3.98 ms
min    = 24.48 ms
max    = 34.28 ms

fromFoldable
------------
fromFoldable (10000)
mean   = 29.06 ms
stddev = 7.18 ms
min    = 20.62 ms
max    = 59.13 ms
fromFoldable (100000)
mean   = 383.63 ms
stddev = 26.92 ms
min    = 361.95 ms
max    = 451.72 ms

Foldable
---------------
OldMap.foldr big map (1000000)
mean   = 2.74 s
stddev = 583.45 ms
min    = 2.09 s
max    = 3.85 s
M.foldr big map (1000000)
mean   = 136.92 ms
stddev = 8.55 ms
min    = 128.06 ms
max    = 158.92 ms
OldMap.foldl big map (1000000)
mean   = 3.00 s
stddev = 366.36 ms
min    = 2.30 s
max    = 3.68 s
M.foldl big map (1000000)
mean   = 123.67 ms
stddev = 6.86 ms
min    = 116.96 ms
max    = 140.11 ms
OldMap.foldMap big map (1000000)
mean   = 3.32 s
stddev = 353.55 ms
min    = 2.90 s
max    = 3.96 s
M.foldMap big map (1000000)
mean   = 159.57 ms
stddev = 34.20 ms
min    = 116.21 ms
max    = 211.96 ms
OldMap.foldrWithIndex big map (1000000)
mean   = 988.81 ms
stddev = 162.38 ms
min    = 787.27 ms
max    = 1.34 s
M.foldrWithIndex big map (1000000)
mean   = 112.77 ms
stddev = 28.10 ms
min    = 82.08 ms
max    = 158.50 ms
OldMap.foldlWithIndex big map (1000000)
mean   = 810.71 ms
stddev = 120.55 ms
min    = 718.71 ms
max    = 1.13 s
M.foldlWithIndex big map (1000000)
mean   = 82.17 ms
stddev = 4.74 ms
min    = 77.97 ms
max    = 92.23 ms
OldMap.foldMapWithIndex big map (1000000)
mean   = 711.59 ms
stddev = 62.09 ms
min    = 617.76 ms
max    = 820.32 ms
M.foldMapWithIndex big map (1000000)
mean   = 250.51 ms
stddev = 32.33 ms
min    = 198.14 ms
max    = 282.25 ms

union
---------------
OldMap.union: big map (1000000)
mean   = 5.24 s
stddev = 526.87 ms
min    = 4.74 s
max    = 6.54 s
M.union: big map (1000000)
mean   = 4.62 s
stddev = 284.55 ms
min    = 4.28 s
max    = 5.04 s

[JordanMartinez]

Could you check what the benchmarks are if the Foldable instances are updated?

[JordanMartinez]

With the Foldable instances fixed, here are the latest benchmarks.

Map
===
size
---------------
size: singleton map
mean   = 906.63 ns
stddev = 2.65 μs
min    = 698.00 ns
max    = 82.88 μs
size: small map (100)
mean   = 3.17 μs
stddev = 2.01 μs
min    = 2.83 μs
max    = 43.50 μs
size: midsize map (10000)
mean   = 250.72 μs
stddev = 27.18 μs
min    = 239.16 μs
max    = 447.05 μs
size: big map (1000000)
mean   = 34.57 ms
stddev = 3.69 ms
min    = 32.48 ms
max    = 44.56 ms

fromFoldable
------------
fromFoldable (10000)
mean   = 33.48 ms
stddev = 2.92 ms
min    = 30.36 ms
max    = 60.19 ms
fromFoldable (100000)
mean   = 420.50 ms
stddev = 5.78 ms
min    = 408.24 ms
max    = 426.58 ms

Foldable
---------------
Map2a0bff.foldr big map (1000000)
mean   = 3.17 s
stddev = 372.89 ms
min    = 2.80 s
max    = 3.87 s
M.foldr big map (1000000)
mean   = 111.59 ms
stddev = 11.98 ms
min    = 99.89 ms
max    = 132.69 ms
Map2a0bff.foldl big map (1000000)
mean   = 3.37 s
stddev = 486.64 ms
min    = 2.62 s
max    = 4.10 s
M.foldl big map (1000000)
mean   = 151.15 ms
stddev = 45.32 ms
min    = 105.59 ms
max    = 255.58 ms
Map2a0bff.foldMap big map (1000000)
mean   = 3.39 s
stddev = 652.27 ms
min    = 2.65 s
max    = 4.49 s
M.foldMap big map (1000000)
mean   = 152.01 ms
stddev = 15.99 ms
min    = 140.95 ms
max    = 187.79 ms
Map2a0bff.foldrWithIndex big map (1000000)
mean   = 1.19 s
stddev = 293.98 ms
min    = 1.01 s
max    = 1.82 s
M.foldrWithIndex big map (1000000)
mean   = 106.70 ms
stddev = 7.35 ms
min    = 98.46 ms
max    = 123.15 ms
Map2a0bff.foldlWithIndex big map (1000000)
mean   = 979.66 ms
stddev = 194.60 ms
min    = 860.84 ms
max    = 1.50 s
M.foldlWithIndex big map (1000000)
mean   = 138.90 ms
stddev = 23.14 ms
min    = 118.54 ms
max    = 181.73 ms
Map2a0bff.foldMapWithIndex big map (1000000)
mean   = 1.07 s
stddev = 221.14 ms
min    = 896.33 ms
max    = 1.66 s
M.foldMapWithIndex big map (1000000)
mean   = 254.60 ms
stddev = 44.10 ms
min    = 212.17 ms
max    = 346.70 ms

union
---------------
Map2a0bff.union: big map (1000000)
mean   = 7.70 s
stddev = 722.97 ms
min    = 6.69 s
max    = 9.17 s
M.union: big map (1000000)
mean   = 6.62 s
stddev = 720.08 ms
min    = 5.54 s
max    = 8.25 s

[JordanMartinez]

Thanks @xgrommx.

Would you mind pushing a changelog entry stating that the speed was improved?

[JordanMartinez]

🏓 @thomashoneyman

[natefaubion]

Should we implement values in terms of foldr? Seems like values being accidentally quadratic is a huge problem.

[xgrommx]

what do u mean?

[JordanMartinez]

He means the values function. Since it produces a List, and each list part is concatenated via <>, which traverses over the list on the left side again (e.g. 1 : 2 : 3 : Nil <> 4 : Nil), then there's room for optimization here.

[JordanMartinez]

In other words, rather than building multiple lists that get appended together, which requires iterating through list parts a second time, we could just build the final list in one go, no traversals needed:

values :: forall k v. Map k v -> List v
values Leaf = Nil
-values (Two left _ v right) = values left <> pure v <> values right
+values (Two left _ v right) = foldr Cons (Cons v (foldr Cons Nil right)) left
-values (Three left _ v1 mid _ v2 right) =
-  values left <> pure v1 <> values mid <> pure v2 <> values right
+values (Three left _ v1 mid _ v2 right) =
+  foldr Cons (Cons v1 (foldr Cons (Cons v2 (foldr Cons Nil right) mid))) left

[JordanMartinez]

Ha! I'm dumb. Actually, values is just foldr Cons Nil.

values :: forall k v. Map k v -> List v
values = foldr Cons Nil

[JordanMartinez]

How do you like them apples?

Map
===
values
---------------
Map2a0bff.values: big map (1000000)
mean   = 2.89 s
stddev = 302.43 ms
min    = 2.52 s
max    = 3.43 s
M.values: big map (1000000)
mean   = 278.44 ms
stddev = 4.57 ms
min    = 271.75 ms
max    = 287.72 ms

[JordanMartinez]

See #61

[natefaubion]

This is really great work! Thanks!

[thomashoneyman]

Fantastic work! Thank you!