Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[GLUTEN-7145][CH] Decouple SerializedPlanParser from other parser modules #7250

Merged
merged 8 commits into from
Oct 15, 2024
Merged

[GLUTEN-7145][CH] Decouple SerializedPlanParser from other parser modules #7250

merged 8 commits into from
Oct 15, 2024

Conversation

lgbo-ustc
Copy link
Contributor

@lgbo-ustc lgbo-ustc commented Sep 14, 2024 •
edited
Loading

What changes were proposed in this pull request?

(Please fill in changes proposed in this fix)

Fixes: #7145

Decouple SerializedPlanParser from all other modules. Introduce ParserContext and ExpressionParser in this PR.

ParserContext is the minimal core that need be shared among parser modules.

ExpressionParser is responsilbe for parser substrait expession into CH actions dag. It unifies this tranformation among all parser modules. Base on this, some optimization will be applied at later.

All these should make the code more flexible.

How was this patch tested?

(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)

unit tests

(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)

@lgbo-ustc lgbo-ustc marked this pull request as draft September 14, 2024 07:55
@github-actions GitHub Actions
Copy link

#7145

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

1 similar comment
@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@lgbo-ustc lgbo-ustc changed the title [GLUTEN-7145][CH] Add ExpressionParser [GLUTEN-7145][CH] Decouple SerializedPlanParser from other parser modules Sep 24, 2024
liuneng1994
liuneng1994 reviewed Sep 24, 2024
View reviewed changes
Copy link
Contributor

@liuneng1994 liuneng1994 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@baibaichen
Copy link
Contributor

@lgbo-ustc please hold this PR

@lgbo-ustc
Copy link
Contributor Author

@lgbo-ustc please hold this PR

?

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@lgbo-ustc lgbo-ustc marked this pull request as ready for review October 15, 2024 04:17
@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

baibaichen
baibaichen approved these changes Oct 15, 2024
View reviewed changes
Copy link
Contributor

@baibaichen baibaichen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@baibaichen baibaichen merged commit 0bc6613 into apache:main Oct 15, 2024
7 of 8 checks passed
@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCDS SF2000 with Velox backend, for reference only ====

query log/native_master_10_15_2024_time.csv log/native_master_10_14_2024_3dc1980441_time.csv difference percentage
q1 14.47 14.58 0.104 100.72%
q2 15.07 13.45 -1.623 89.23%
q3 5.11 5.31 0.203 103.98%
q4 72.46 71.72 -0.740 98.98%
q5 9.89 8.57 -1.321 86.64%
q6 4.06 2.12 -1.935 52.30%
q7 8.11 6.06 -2.050 74.73%
q8 5.61 5.54 -0.068 98.79%
q9 26.40 26.59 0.188 100.71%
q10 7.37 9.34 1.972 126.75%
q11 37.46 37.28 -0.186 99.50%
q12 1.38 1.40 0.021 101.54%
q13 6.43 6.40 -0.030 99.53%
q14a 47.39 49.58 2.190 104.62%
q14b 41.97 43.30 1.334 103.18%
q15 2.21 2.40 0.189 108.58%
q16 48.52 49.36 0.839 101.73%
q17 4.81 4.90 0.089 101.85%
q18 6.56 8.54 1.979 130.16%
q19 3.40 2.14 -1.268 62.74%
q20 1.54 1.50 -0.038 97.54%
q21 3.11 2.66 -0.449 85.59%
q22 7.99 7.98 -0.002 99.97%
q23a 103.88 107.62 3.740 103.60%
q23b 126.49 130.50 4.008 103.17%
q24a 104.94 109.12 4.174 103.98%
q24b 111.12 109.94 -1.177 98.94%
q25 4.22 4.12 -0.098 97.67%
q26 4.36 4.19 -0.172 96.05%
q27 5.55 4.66 -0.892 83.94%
q28 31.95 33.37 1.416 104.43%
q29 12.08 10.37 -1.711 85.83%
q30 5.27 4.78 -0.488 90.73%
q31 6.84 7.02 0.179 102.62%
q32 1.22 1.31 0.086 107.07%
q33 4.82 4.33 -0.489 89.86%
q34 3.79 4.09 0.304 108.02%
q35 7.75 8.70 0.957 112.35%
q36 5.47 5.76 0.297 105.43%
q37 4.90 4.99 0.096 101.96%
q38 13.64 15.91 2.263 116.59%
q39a 3.65 3.16 -0.497 86.39%
q39b 3.37 2.86 -0.504 85.04%
q40 3.85 3.89 0.040 101.03%
q41 0.65 0.64 -0.011 98.31%
q42 0.91 0.94 0.028 103.02%
q43 4.59 4.70 0.106 102.31%
q44 9.92 9.63 -0.283 97.15%
q45 3.29 3.35 0.063 101.92%
q46 3.96 3.98 0.019 100.49%
q47 17.83 17.65 -0.180 98.99%
q48 5.16 5.28 0.112 102.18%
q49 7.05 8.22 1.176 116.69%
q50 21.86 21.82 -0.039 99.82%
q51 9.29 9.40 0.110 101.18%
q52 1.06 1.01 -0.046 95.64%
q53 2.44 2.43 -0.014 99.42%
q54 3.76 3.77 0.011 100.29%
q55 1.05 1.02 -0.023 97.84%
q56 4.03 4.11 0.077 101.91%
q57 10.35 10.89 0.544 105.26%
q58 2.55 2.55 0.003 100.13%
q59 10.94 11.83 0.889 108.12%
q60 4.01 4.16 0.156 103.91%
q61 4.71 4.06 -0.649 86.24%
q62 5.24 4.47 -0.765 85.40%
q63 2.41 2.33 -0.081 96.64%
q64 59.72 64.36 4.643 107.78%
q65 16.64 17.55 0.906 105.44%
q66 4.45 6.22 1.772 139.86%
q67 439.01 422.72 -16.287 96.29%
q68 3.86 3.71 -0.152 96.05%
q69 7.49 5.28 -2.210 70.51%
q70 11.78 11.62 -0.160 98.65%
q71 2.54 2.28 -0.260 89.76%
q72 214.35 218.83 4.476 102.09%
q73 2.27 2.36 0.082 103.58%
q74 22.72 23.84 1.115 104.91%
q75 27.86 25.05 -2.818 89.89%
q76 14.24 12.68 -1.568 88.99%
q77 2.09 2.08 -0.007 99.66%
q78 49.36 50.19 0.825 101.67%
q79 3.82 3.75 -0.070 98.17%
q80 11.52 11.62 0.103 100.89%
q81 4.59 4.57 -0.023 99.50%
q82 6.92 7.28 0.358 105.17%
q83 1.61 1.70 0.092 105.72%
q84 3.52 3.21 -0.310 91.18%
q85 7.82 7.43 -0.395 94.95%
q86 4.26 4.15 -0.104 97.55%
q87 16.65 13.90 -2.747 83.50%
q88 18.07 18.10 0.029 100.16%
q89 3.63 3.10 -0.527 85.49%
q90 2.88 3.54 0.658 122.82%
q91 2.46 2.13 -0.324 86.83%
q92 1.42 1.25 -0.163 88.48%
q93 41.62 39.82 -1.806 95.66%
q94 27.17 27.22 0.049 100.18%
q9 90.20 93.05 2.848 103.16%
q5 2.54 2.85 0.306 112.06%
q96 18.36 18.14 -0.222 98.79%
q97 2.02 1.81 -0.203 89.93%
q98 11.55 10.71 -0.844 92.70%
q99 11.55 10.71 -0.844 92.70%
total 2248.56 2247.75 -0.802 99.96%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@liuneng1994 liuneng1994 liuneng1994 left review comments

@baibaichen baibaichen baibaichen approved these changes

Assignees
No one assigned
Labels
CLICKHOUSE
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[CH] Well-structured parser layers
4 participants
@lgbo-ustc @baibaichen @GlutenPerfBot @liuneng1994