perf: speed up fsst decompression#2626
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2626 +/- ##
==========================================
+ Coverage 79.35% 79.39% +0.04%
==========================================
Files 213 213
Lines 62520 62706 +186
Branches 62520 62706 +186
==========================================
+ Hits 49610 49784 +174
- Misses 9996 10005 +9
- Partials 2914 2917 +3
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
westonpace
left a comment
There was a problem hiding this comment.
Sorry for the delay in getting back to this. I don't yet fully understand FSST and so I don't understand exactly this change. However, the tests pass and FSST is still guarded by env variable so lets merge this and I will try and set some time aside next week to really dig through in more detail.
|
ha, sorry that I didn't explain this PR, it is basically a rust translation from the original c++ implementation of decompression. |

before:

after:

to reproduce:
cargo run --release --example benchmarkin
rust/lance-encoding/compression-algo/fsstmachine info:
11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHzLinux 192 5.10.0-28-amd64 #1 SMP Debian 5.10.209-2 (2024-01-31) x86_64 GNU/Linux