Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

planner,executor: fix cached table query with filter condition #32590

Merged
merged 9 commits into from
Mar 9, 2022

Conversation

tiancaiamao
Copy link
Contributor

@tiancaiamao tiancaiamao commented Feb 24, 2022

What problem does this PR solve?

Issue Number: close #32422

Problem Summary:

This bug was introduced by #32139
In the initial implementation, I generate UnionScan plan when reading from cache, and no UnionScan when the cache is bypassed or unavailable.
But since the shape of the plan is different, the prepared plan cache feature is not applicable.

Then, #32139 try to fix the prepare plan cache issue by defer the UnionScanExec generation to the executor build phase. It did make plan cache possible, but introduce bug #32422

Handling the filter condition is not a trivial work if we continue generating UnionScanExec in the executor build phase:

  • In the logical plan phase, by the predication push down optimize rule, the condition is seperated.
    Some are pushed down to TableReader and some remains, cause it's difficult to rebuild the filter condition for UnionScan in the executor build phase.
  • And also the existence of generated column make the situation from bad to worse.

What is changed and how it works?

So I change the implementation the third time.
This time, I will always keep the UnionScan in the planner phase for a cached table, and whether the UnionScan is really used is decided by whether the cached data is available.

In this way, prepared plan cache is applicable, since the shape of plan is unchanged (although maybe less efficient in some minor cases), and also the filter condition bug is fixed.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

None

@ti-chi-bot
Copy link
Member

ti-chi-bot commented Feb 24, 2022

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • lcwangchao
  • tangenta

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added release-note-none Denotes a PR that doesn't merit a release note. do-not-merge/needs-triage-completed labels Feb 24, 2022
@ti-chi-bot ti-chi-bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Feb 24, 2022
@tiancaiamao
Copy link
Contributor Author

/rebuild

@sre-bot
Copy link
Contributor

sre-bot commented Feb 24, 2022

executor/builder.go Outdated Show resolved Hide resolved
@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Mar 2, 2022
@ti-chi-bot
Copy link
Member

@tiancaiamao: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@ti-chi-bot ti-chi-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 5, 2022
@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Mar 7, 2022
@tiancaiamao
Copy link
Contributor Author

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 93331c8

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Mar 9, 2022
@tiancaiamao
Copy link
Contributor Author

/run-unit-test

@tiancaiamao tiancaiamao removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 9, 2022
@tiancaiamao
Copy link
Contributor Author

/run-mysql-test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Filter condition is missing caused wrong result in cached table scanning
5 participants