Skip to content
This repository has been archived by the owner on Feb 18, 2024. It is now read-only.

Special-case filter single array record batch in filter_record_batch #235

Merged
merged 3 commits into from
Jul 30, 2021

Conversation

Dandandan
Copy link
Collaborator

@Dandandan Dandandan commented Jul 29, 2021

Gives a speed up for single-column batches.

Closes #234

filter single record batch                                                                            
                        time:   [229.93 us 231.52 us 233.29 us]
                        change: [-54.913% -54.501% -54.106%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  4 (4.00%) high mild
  3 (3.00%) high severe

Copy link
Owner

@jorgecarleitao jorgecarleitao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great 👍

src/compute/filter.rs Outdated Show resolved Hide resolved
@codecov
Copy link

codecov bot commented Jul 29, 2021

Codecov Report

Merging #235 (5001c88) into main (95fe10a) will decrease coverage by 0.01%.
The diff coverage is 0.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #235      +/-   ##
==========================================
- Coverage   76.83%   76.82%   -0.02%     
==========================================
  Files         229      229              
  Lines       19582    19586       +4     
==========================================
  Hits        15046    15046              
- Misses       4536     4540       +4     
Impacted Files Coverage Δ
src/compute/filter.rs 63.19% <0.00%> (-1.59%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 95fe10a...5001c88. Read the comment docs.

@Dandandan
Copy link
Collaborator Author

Dandandan commented Jul 29, 2021

Nice! About a 2x improvement for f32 values

@Dandandan Dandandan changed the title Special-case single array record batch Special-case filter single array record batch in filter_record_batch Jul 29, 2021
@jorgecarleitao jorgecarleitao merged commit 3cd8cff into jorgecarleitao:main Jul 30, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Avoid materialization of indices in filter_record_batch for single arrays
2 participants