Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CORE] Minor adjustment to columnar rule list, and move all columnar sub-rules to one source folder #4863

Merged
merged 8 commits into from
Mar 7, 2024

Conversation

zhztheplayer
Copy link
Member

@zhztheplayer zhztheplayer commented Mar 6, 2024

  1. Add an extra InsertTransitions rule right after TransformPreOverrides. To verify whether we could consolidate the plan-implementing phase and C2R/R2C transition-adding phase together. (in future patch)
  2. Move all sub-rules used by ColumnarRuleOverrides to source package/folder io.glutenproject.extension.columnar.

@zhztheplayer zhztheplayer marked this pull request as ready for review March 6, 2024 06:36
Copy link

github-actions bot commented Mar 6, 2024

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/oap-project/gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

Copy link

github-actions bot commented Mar 6, 2024

Run Gluten Clickhouse CI

@zhztheplayer zhztheplayer changed the title [VL] Minor adjustments to columnar rule list, and move all columnar sub-rules to one source folder [VL] Minor adjustment to columnar rule list, and move all columnar sub-rules to one source folder Mar 6, 2024
Copy link

github-actions bot commented Mar 6, 2024

Run Gluten Clickhouse CI

@zhztheplayer
Copy link
Member Author

/Benchmark Velox

Copy link

github-actions bot commented Mar 6, 2024

Run Gluten Clickhouse CI

@zhztheplayer zhztheplayer changed the title [VL] Minor adjustment to columnar rule list, and move all columnar sub-rules to one source folder [CORE] Minor adjustment to columnar rule list, and move all columnar sub-rules to one source folder Mar 6, 2024
Copy link

github-actions bot commented Mar 6, 2024

Run Gluten Clickhouse CI

@zhztheplayer
Copy link
Member Author

/Benchmark Velox

@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_4863_time.csv log/native_master_03_06_2024_10910e900_time.csv difference percentage
q1 35.78 35.69 -0.093 99.74%
q2 28.16 24.09 -4.068 85.55%
q3 37.77 37.17 -0.595 98.42%
q4 39.01 37.31 -1.701 95.64%
q5 70.93 70.29 -0.640 99.10%
q6 5.59 7.07 1.481 126.49%
q7 83.80 84.52 0.718 100.86%
q8 84.23 86.24 2.009 102.38%
q9 123.29 125.77 2.483 102.01%
q10 43.60 42.91 -0.694 98.41%
q11 20.33 20.70 0.372 101.83%
q12 24.66 29.30 4.640 118.81%
q13 45.86 45.63 -0.229 99.50%
q14 18.87 20.55 1.681 108.91%
q15 27.41 29.07 1.656 106.04%
q16 13.98 13.92 -0.057 99.59%
q17 100.14 102.68 2.537 102.53%
q18 144.52 145.05 0.527 100.36%
q19 12.52 12.61 0.088 100.70%
q20 28.46 25.67 -2.792 90.19%
q21 219.63 224.29 4.654 102.12%
q22 15.21 13.74 -1.465 90.37%
total 1223.77 1234.28 10.511 100.86%

Copy link

github-actions bot commented Mar 6, 2024

Run Gluten Clickhouse CI

Copy link

github-actions bot commented Mar 6, 2024

Run Gluten Clickhouse CI

Copy link

github-actions bot commented Mar 6, 2024

Run Gluten Clickhouse CI

@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_4863_time.csv log/native_master_03_06_2024_10910e900_time.csv difference percentage
q1 35.99 35.69 -0.300 99.17%
q2 24.09 24.09 0.005 100.02%
q3 37.67 37.17 -0.494 98.69%
q4 39.23 37.31 -1.919 95.11%
q5 70.72 70.29 -0.424 99.40%
q6 5.43 7.07 1.643 130.28%
q7 83.58 84.52 0.943 101.13%
q8 84.96 86.24 1.275 101.50%
q9 120.82 125.77 4.951 104.10%
q10 43.08 42.91 -0.170 99.60%
q11 20.27 20.70 0.429 102.11%
q12 26.71 29.30 2.594 109.71%
q13 46.32 45.63 -0.689 98.51%
q14 20.03 20.55 0.522 102.61%
q15 27.90 29.07 1.170 104.19%
q16 15.58 13.92 -1.659 89.35%
q17 102.85 102.68 -0.170 99.84%
q18 140.87 145.05 4.173 102.96%
q19 12.71 12.61 -0.096 99.24%
q20 26.08 25.67 -0.408 98.44%
q21 223.28 224.29 1.004 100.45%
q22 14.00 13.74 -0.258 98.15%
total 1222.16 1234.28 12.121 100.99%

Copy link

github-actions bot commented Mar 6, 2024

Run Gluten Clickhouse CI

Copy link

github-actions bot commented Mar 7, 2024

Run Gluten Clickhouse CI

3 similar comments
@zhztheplayer
Copy link
Member Author

Run Gluten Clickhouse CI

@zhztheplayer
Copy link
Member Author

Run Gluten Clickhouse CI

@lwz9103
Copy link
Contributor

lwz9103 commented Mar 7, 2024

Run Gluten Clickhouse CI

Copy link

github-actions bot commented Mar 7, 2024

Run Gluten Clickhouse CI

JkSelf
JkSelf previously approved these changes Mar 7, 2024
Copy link
Contributor

@JkSelf JkSelf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks.

Copy link

github-actions bot commented Mar 7, 2024

Run Gluten Clickhouse CI

@zhztheplayer zhztheplayer merged commit 3d3bf62 into apache:main Mar 7, 2024
20 checks passed
@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_4863_time.csv log/native_master_03_06_2024_bddc3fd79_time.csv difference percentage
q1 38.85 38.77 -0.081 99.79%
q2 23.82 24.36 0.534 102.24%
q3 39.01 39.60 0.585 101.50%
q4 37.25 37.58 0.326 100.87%
q5 69.02 69.92 0.907 101.31%
q6 7.44 8.34 0.894 112.01%
q7 80.96 84.40 3.435 104.24%
q8 87.15 86.73 -0.413 99.53%
q9 123.98 119.49 -4.494 96.38%
q10 47.07 43.18 -3.885 91.74%
q11 20.39 20.86 0.477 102.34%
q12 24.83 28.05 3.226 112.99%
q13 46.93 44.55 -2.387 94.91%
q14 17.27 17.03 -0.243 98.60%
q15 30.17 28.15 -2.026 93.29%
q16 14.29 14.07 -0.210 98.53%
q17 101.99 101.46 -0.533 99.48%
q18 143.34 145.26 1.919 101.34%
q19 14.12 13.99 -0.131 99.07%
q20 27.28 28.21 0.936 103.43%
q21 225.52 224.18 -1.333 99.41%
q22 13.72 14.99 1.262 109.19%
total 1234.41 1233.17 -1.236 99.90%

taiyang-li pushed a commit to bigo-sg/gluten that referenced this pull request Mar 25, 2024
taiyang-li pushed a commit to bigo-sg/gluten that referenced this pull request Oct 8, 2024
taiyang-li pushed a commit to bigo-sg/gluten that referenced this pull request Oct 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants