Slotted array vectorized

[Scooletz]

This PR changes the way SlottedArray uses vectorized search.

Previously, it was using the Span.IndexOf over a set of slots. This resulted in 2x more data being searched through, with potential (but quite unlikely) false positive hits when the Slot.Raw was equal to the hash. Additionally, when hash collision occurred, it requires multiple calls to the Span.IndexOf to move the search across the map.

This PR makes a change by aligning SlottedArray to the biggest vector allowed on the given platform, Vector256 for x64 and Vector128 for ARM. It does it by introducing chunks of size of the vector that describe the hashes and the raw part providing the markers and the pointer in page. For x64, this requires to allocate 32 bytes for hashes + 32 bytes for raw = 64 bytes of chunk. There's no other overhead beside some slots potentially not being used (max 60 bytes). This approach allows to scan hashes in a nicely vectorized way without dealing with false positives and checks whether a given value is at a hash offset or not. Then, if the search is not done, jump over the vector of Slots to the next one.

The advantages are the following:

1. 2x less comparisons
2. better hash collision handling (if they occur in the same vector)
3. no reminder handling as everything is vector aligned
4. a simple and tight loop

The vector alignment has been tested on x64, but showed no positive impact in benchmarks

TODO List

• support Vector128 for ARM and others (throws on scalar)
• fix the hidden bug
• add benchmarks
  • hash collision
  • odd nibble path (no impact on construction, rather on Prepare/Unprepare)
  • path longer than 4 so that the NibblePath comparison happens (this will compare equals only)
  • restore defragmentation benchmark (not needed)
  • restore iteration benchmark
• consider materializing the NibblePath in TryFind or before so that the comparison above works as expected (left for future work)
• amend design doc

Benchmarks

The new benchmarks, based on aligned memory, to make them truly comparable show great results. Especially, if the search is not found in the initial keys but requires more iterations.

TryGet

Before

Method	sliceFrom	length	index	odd	Mean	Error	StdDev	Code Size
TryGet	?	?	1	False	7.180 ns	0.1448 ns	0.1355 ns	4,068 B
TryGet	?	?	15	False	7.644 ns	0.1158 ns	0.0967 ns	4,033 B
TryGet	?	?	16	False	8.136 ns	0.1283 ns	0.1200 ns	4,033 B
TryGet	?	?	31	False	8.841 ns	0.1750 ns	0.1637 ns	4,098 B
TryGet	?	?	32	False	9.113 ns	0.0999 ns	0.0934 ns	4,079 B
TryGet	?	?	47	False	9.284 ns	0.0931 ns	0.0871 ns	4,057 B
TryGet	?	?	48	False	10.058 ns	0.0464 ns	0.0412 ns	4,100 B
TryGet	?	?	63	False	10.383 ns	0.1024 ns	0.0957 ns	4,035 B
TryGet	?	?	64	False	10.979 ns	0.0954 ns	0.0893 ns	4,036 B
TryGet	?	?	95	False	12.936 ns	0.1134 ns	0.1006 ns	4,083 B
TryGet	?	?	96	False	11.370 ns	0.1728 ns	0.1616 ns	4,096 B

After

Method	sliceFrom	length	index	odd	Mean	Error	StdDev	Code Size
TryGet	?	?	1	False	7.512 ns	0.1016 ns	0.0951 ns	3,310 B
TryGet	?	?	15	False	7.455 ns	0.0670 ns	0.0627 ns	3,266 B
TryGet	?	?	16	False	7.903 ns	0.0643 ns	0.0601 ns	3,253 B
TryGet	?	?	31	False	7.816 ns	0.0808 ns	0.0756 ns	3,253 B
TryGet	?	?	32	False	8.174 ns	0.0827 ns	0.0774 ns	3,240 B
TryGet	?	?	47	False	8.147 ns	0.0778 ns	0.0650 ns	3,220 B
TryGet	?	?	48	False	8.464 ns	0.1305 ns	0.1157 ns	3,229 B
TryGet	?	?	63	False	8.487 ns	0.1186 ns	0.1110 ns	3,186 B
TryGet	?	?	64	False	8.922 ns	0.1604 ns	0.1501 ns	3,217 B
TryGet	?	?	95	False	9.197 ns	0.1391 ns	0.1233 ns	3,171 B
TryGet	?	?	96	False	10.232 ns	0.1698 ns	0.1588 ns	3,175 B

TryGet_With_Hash_Collisions

Before

Method	index	Mean	Error	StdDev	Code Size
TryGet_With_Hash_Collisions	1	23.79 ns	0.168 ns	0.157 ns	4,458 B
TryGet_With_Hash_Collisions	2	14.18 ns	0.172 ns	0.161 ns	4,338 B
TryGet_With_Hash_Collisions	3	23.95 ns	0.096 ns	0.090 ns	4,497 B
TryGet_With_Hash_Collisions	4	14.22 ns	0.120 ns	0.112 ns	4,345 B
TryGet_With_Hash_Collisions	30	15.72 ns	0.237 ns	0.222 ns	4,316 B
TryGet_With_Hash_Collisions	31	23.88 ns	0.193 ns	0.180 ns	4,496 B

After

Method	index	Mean	Error	StdDev	Code Size
TryGet_With_Hash_Collisions	1	21.75 ns	0.171 ns	0.151 ns	5,598 B
TryGet_With_Hash_Collisions	2	14.82 ns	0.297 ns	0.278 ns	5,551 B
TryGet_With_Hash_Collisions	3	21.62 ns	0.232 ns	0.205 ns	5,649 B
TryGet_With_Hash_Collisions	4	14.73 ns	0.094 ns	0.088 ns	5,553 B
TryGet_With_Hash_Collisions	30	15.31 ns	0.204 ns	0.191 ns	5,528 B
TryGet_With_Hash_Collisions	31	22.44 ns	0.212 ns	0.188 ns	5,606 B

SlottedArray upgraded design

                                                                   
 ┌─────────────────┬────────────────────┬─────────────────────────┐  
 │ HEADER          │ VECTOR of Hashes   │ VECTOR of Slots         │  
 │                 │                    │                         │  
 │                 │                    │                         │  
 │                 │                    │                         │  
 ├─────────────────┴──────┬─────────────┴───────┬─────────────────┤  
 │ VECTOR or Hashes       │ Vector of Slots     │                 │  
 │                        │                     │                 │  
 │                        │                     │                 │  
 │                        │                     │                 │  
 ├────────────────────────┴─────────────────────┘                 │  
 │                                                                │  
 │                                                                │  
 │                                                                │  
 │                                                                │  
 │                                                                │  
 │                                                                │  
 │                                                                │  
 │                            ┌─────────┬─────────────────────────┤  
 │                            │         │                         │  
 │                            │         │                         │  
 │                            │  DATA   │                DATA     │  
 │                            │for slot1│              for slot 0 │  
 │                            │         │                         │  
 └────────────────────────────┴─────────┴─────────────────────────┘  
                                                                      

[github-actions]

Package	Line Rate	Branch Rate	Health
Paprika	84%	79%	➖
Summary	84% (4224 / 4999)	79% (1336 / 1701)	➖

Minimum allowed line rate is 75%