Skip to content
This repository has been archived by the owner on Feb 18, 2024. It is now read-only.

Improved performance of utf8 comparison (1.7x-4x) #322

Merged
merged 2 commits into from
Aug 24, 2021
Merged

Conversation

jorgecarleitao
Copy link
Owner

@jorgecarleitao jorgecarleitao commented Aug 23, 2021

utf8 2^20               time:   [10.376 ms 10.395 ms 10.424 ms]                      
                        change: [-43.339% -43.191% -43.033%] (p = 0.00 < 0.05)
utf8 scalar 2^20        time:   [1.5213 ms 1.5243 ms 1.5283 ms]                              
                        change: [-75.416% -75.313% -75.200%] (p = 0.00 < 0.05)

The idea is that we can iterate directly over values (&str) ignoring validities, which is evidently faster than iterating over Option<&str>

@jorgecarleitao jorgecarleitao added the enhancement An improvement to an existing feature label Aug 23, 2021
@jorgecarleitao jorgecarleitao changed the title Improved performance of utf8 comparison (1.5x-4x) Improved performance of utf8 comparison (1.7x-4x) Aug 23, 2021
@codecov
Copy link

codecov bot commented Aug 23, 2021

Codecov Report

Merging #322 (dfe5460) into main (824ad7e) will increase coverage by 0.02%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #322      +/-   ##
==========================================
+ Coverage   80.55%   80.58%   +0.02%     
==========================================
  Files         324      324              
  Lines       21399    21453      +54     
==========================================
+ Hits        17239    17288      +49     
- Misses       4160     4165       +5     
Impacted Files Coverage Δ
src/util/bench_util.rs 0.00% <ø> (ø)
src/compute/comparison/utf8.rs 67.41% <100.00%> (-1.41%) ⬇️
tests/it/io/ipc/common.rs 100.00% <0.00%> (ø)
tests/it/io/ipc/write/stream.rs 100.00% <0.00%> (ø)
src/io/ipc/read/stream.rs 83.76% <0.00%> (+1.48%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 824ad7e...dfe5460. Read the comment docs.

@jorgecarleitao jorgecarleitao merged commit 6cc3937 into main Aug 24, 2021
@jorgecarleitao jorgecarleitao deleted the comp-utf8 branch August 24, 2021 07:59
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement An improvement to an existing feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant