Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create dfbench, split up tpch benchmark runner into modules #7054

Merged
merged 1 commit into from
Jul 24, 2023

Conversation

alamb
Copy link
Contributor

@alamb alamb commented Jul 21, 2023

Note this looks like a big PR but it just moves code around. I will move the other runners into this new format as follow on PRs

Which issue does this PR close?

Part of #7052
related to #6994

Rationale for this change

I want to have a nice home for clickbench tests #6994 but the current situation in the benchmark runners is a mess

What changes are included in this PR?

  1. Create dfbench entrypoint
  2. split up tpch benchmark runner code into moudles

This PR also retain the old entrypoints for backwards compatibility, so the following two commands do the same thing

Old

cargo run  --bin tpch -- benchmark datafusion --iterations 5 -m --format parquet -q 1 --path data/tpch_sf1

New

cargo run  --bin dfbench -- tpch --iterations 5 -m --format parquet -q 1 --path data/tpch_sf1

Are these changes tested?

existing coverage

Are there any user-facing changes?


use arrow::util::pretty::pretty_format_batches;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All this code is moved elsewhere

@alamb alamb marked this pull request as draft July 21, 2023 17:13
@alamb alamb marked this pull request as ready for review July 22, 2023 13:49
@alamb
Copy link
Contributor Author

alamb commented Jul 22, 2023

cc @Dandandan / @andygrove in case you have time to review this code. I have big plans but this one just moves things around

@Dandandan
Copy link
Contributor

Looks good to me :)

@Dandandan Dandandan merged commit 9445aa0 into apache:main Jul 24, 2023
@alamb alamb deleted the alamb/dfbench branch July 24, 2023 14:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants