Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: read all bytes out before do async bench #390

Merged
merged 2 commits into from
Nov 14, 2022

Conversation

jiacai2050
Copy link
Contributor

@jiacai2050 jiacai2050 commented Nov 11, 2022

Which issue does this PR close?

Closes #

Rationale for this change

In parquet sync read, file is read out before iterate record batch, this PR update async read bench, to let it have same logic with sync version

What changes are included in this PR?

Are there any user-facing changes?

How does this change test

Not required

A benchmark for parquet file with 103951966 rows

[2022-11-14T09:03:56Z INFO  benchmarks::parquet_bench]                                                                                                               
ParquetBench Async total rows of sst:103951966, total batch num:12690,
                open cost:790.76716ms, filter cost:149.713098ms, iter cost:9.603906449s

[2022-11-14T09:31:39Z INFO  benchmarks::parquet_bench] 
ParquetBench Sync total rows of sst:103951966, total batch num:12690,
                open cost:969.653961ms, filter cost:165.7483ms, iter cost:9.981503914s

Copy link
Member

@ShiKaiWi ShiKaiWi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jiacai2050 jiacai2050 merged commit 4b84ec7 into apache:main Nov 14, 2022
@jiacai2050 jiacai2050 deleted the bench-sync branch November 14, 2022 12:17
chunshao90 pushed a commit to chunshao90/ceresdb that referenced this pull request May 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants