Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add parquet page filter #664

Merged
merged 10 commits into from
Jun 5, 2023
Merged

Conversation

jiacai2050
Copy link
Contributor

@jiacai2050 jiacai2050 commented Feb 24, 2023

Rationale

Part of #589

Detailed Changes

  • Introduce PagePruningPredicate when build ParquetRecordBatchStream

Test Plan

@codecov-commenter
Copy link

codecov-commenter commented Feb 24, 2023

Codecov Report

Merging #664 (ca4bd46) into main (9e06dcc) will increase coverage by 0.05%.
The diff coverage is 98.24%.

❗ Current head ca4bd46 differs from pull request most recent head 2ba5512. Consider uploading reports for the commit 2ba5512 to get more accurate results

@@            Coverage Diff             @@
##             main     #664      +/-   ##
==========================================
+ Coverage   67.28%   67.33%   +0.05%     
==========================================
  Files         283      283              
  Lines       44491    44534      +43     
==========================================
+ Hits        29935    29989      +54     
+ Misses      14556    14545      -11     
Impacted Files Coverage Δ
analytic_engine/src/lib.rs 98.76% <ø> (ø)
analytic_engine/src/sst/parquet/async_reader.rs 89.66% <94.11%> (+0.10%) ⬆️
analytic_engine/src/sst/parquet/meta_data.rs 87.35% <100.00%> (+3.73%) ⬆️
components/parquet_ext/src/prune/min_max.rs 94.04% <100.00%> (+1.14%) ⬆️
analytic_engine/src/row_iter/merge.rs 89.18% <0.00%> (+0.17%) ⬆️
...nalytic_engine/src/sst/parquet/row_group_pruner.rs 82.22% <0.00%> (+3.33%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@jiacai2050 jiacai2050 marked this pull request as ready for review June 2, 2023 07:33
Copy link
Member

@ShiKaiWi ShiKaiWi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jiacai2050 jiacai2050 merged commit d5593b7 into apache:main Jun 5, 2023
@jiacai2050 jiacai2050 deleted the feat-page-filter branch June 5, 2023 13:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants