Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GLUTEN-5016][CH] Fix exchange fallback in simple aggregation sql if spark.gluten.sql.columnar.preferColumnar=false #5042

Merged
merged 1 commit into from
Mar 22, 2024

Conversation

lwz9103
Copy link
Contributor

@lwz9103 lwz9103 commented Mar 20, 2024

What changes were proposed in this pull request?

(Fixes: #5016)

How was this patch tested?

unit tests

Copy link

#5016

Copy link

Run Gluten Clickhouse CI

Copy link

Run Gluten Clickhouse CI

1 similar comment
Copy link

Run Gluten Clickhouse CI

@zhztheplayer
Copy link
Member

I'll have a check to see if we can solve the issue without reverting. The change was important somehow. Thanks.

Copy link

Run Gluten Clickhouse CI

@lwz9103
Copy link
Contributor Author

lwz9103 commented Mar 20, 2024

retest this please

Copy link

Run Gluten Clickhouse CI

@lwz9103 lwz9103 force-pushed the GLUTEN-5016 branch 2 times, most recently from cf25c1e to a4516ca Compare March 21, 2024 03:33
Copy link

Run Gluten Clickhouse CI

1 similar comment
Copy link

Run Gluten Clickhouse CI

Copy link

Run Gluten Clickhouse CI

Copy link

Run Gluten Clickhouse CI

@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_5042_time.csv log/native_master_03_21_2024_9d859d8e4_time.csv difference percentage
q1 36.25 36.78 0.532 101.47%
q2 24.14 23.82 -0.326 98.65%
q3 38.17 36.97 -1.199 96.86%
q4 38.63 38.29 -0.346 99.10%
q5 71.12 68.82 -2.299 96.77%
q6 8.81 7.79 -1.013 88.50%
q7 83.23 82.16 -1.063 98.72%
q8 84.13 85.93 1.793 102.13%
q9 122.85 120.31 -2.544 97.93%
q10 45.58 43.15 -2.422 94.69%
q11 19.97 20.44 0.470 102.35%
q12 29.91 26.75 -3.160 89.43%
q13 46.33 48.78 2.444 105.27%
q14 23.95 21.16 -2.786 88.37%
q15 29.81 28.73 -1.085 96.36%
q16 15.10 14.42 -0.685 95.47%
q17 98.65 99.19 0.534 100.54%
q18 142.70 139.46 -3.234 97.73%
q19 15.67 15.07 -0.598 96.18%
q20 27.06 27.86 0.802 102.96%
q21 226.69 226.99 0.296 100.13%
q22 13.96 14.13 0.173 101.24%
total 1242.71 1226.99 -15.717 98.74%

Copy link
Member

@zhztheplayer zhztheplayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Copy link
Contributor

@baibaichen baibaichen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@baibaichen baibaichen merged commit 0702eb8 into apache:main Mar 22, 2024
18 of 20 checks passed
@zhouyuan
Copy link
Contributor

@zhztheplayer @baibaichen do we still need this "preferColumnar" rule? can we remove this feature?

Thanks,
-yuan

@zhztheplayer
Copy link
Member

Verified locally with Celeborn 0.4.0.

@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_5042_time.csv log/native_master_03_21_2024_9d859d8e4_time.csv difference percentage
q1 36.08 36.78 0.696 101.93%
q2 23.86 23.82 -0.038 99.84%
q3 37.28 36.97 -0.304 99.18%
q4 37.93 38.29 0.354 100.93%
q5 69.26 68.82 -0.436 99.37%
q6 7.20 7.79 0.591 108.20%
q7 81.73 82.16 0.429 100.52%
q8 85.72 85.93 0.207 100.24%
q9 123.69 120.31 -3.386 97.26%
q10 44.34 43.15 -1.182 97.33%
q11 19.94 20.44 0.499 102.50%
q12 22.18 26.75 4.563 120.57%
q13 46.69 48.78 2.084 104.46%
q14 19.28 21.16 1.883 109.77%
q15 29.91 28.73 -1.185 96.04%
q16 14.18 14.42 0.240 101.70%
q17 99.60 99.19 -0.417 99.58%
q18 142.41 139.46 -2.945 97.93%
q19 13.79 15.07 1.279 109.27%
q20 31.18 27.86 -3.317 89.36%
q21 227.01 226.99 -0.025 99.99%
q22 13.78 14.13 0.352 102.56%
total 1227.05 1226.99 -0.057 100.00%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CH] Exchange fallback in simple aggregation sql
5 participants