Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

planner: remove useless predicates after partition pruning #24282

Merged
merged 25 commits into from
May 2, 2021
Merged

planner: remove useless predicates after partition pruning #24282

merged 25 commits into from
May 2, 2021

Conversation

zhuo-zhi
Copy link
Contributor

What problem does this PR solve?

Issue Number: close #22079

Problem Summary:

remove useless predicates after partition pruning

What is changed and how it works?

What's Changed:

Add pruning rules to remove useless predicates after partition pruning.

How it Works:

If predicates lie on the boundary of some partitions, we can add some rules to remove these predicates after partition pruning.

Related changes

  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test

Side effects

  • N/A

Release note

  • remove useless predicates after partition pruning

@zhuo-zhi zhuo-zhi requested a review from a team as a code owner April 26, 2021 05:45
@zhuo-zhi zhuo-zhi requested review from qw4990 and removed request for a team April 26, 2021 05:45
@ti-chi-bot ti-chi-bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Apr 26, 2021
@zhuo-zhi zhuo-zhi marked this pull request as draft April 26, 2021 05:45
@ti-chi-bot ti-chi-bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 26, 2021
@qw4990 qw4990 added the sig/planner SIG: Planner label Apr 26, 2021
@qw4990 qw4990 requested a review from rebelice April 26, 2021 05:57
@ti-chi-bot ti-chi-bot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Apr 26, 2021
@qw4990
Copy link
Contributor

qw4990 commented Apr 27, 2021

@zhuo-zhi Ping, is this PR ready for review?

@zhuo-zhi
Copy link
Contributor Author

zhuo-zhi commented Apr 27, 2021 •
edited
Loading

@zhuo-zhi Ping, is this PR ready for review?

Still some unit test cases to add. Maybe later on today.

@ti-chi-bot ti-chi-bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Apr 27, 2021
@zhuo-zhi
Copy link
Contributor Author

zhuo-zhi commented Apr 27, 2021 •
edited
Loading

There might be other non-equal predicates lying on the boundary of range partitions that can be pruned out as well. eg:

CREATE TABLE `t` (
  `a` int(11) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin
PARTITION BY RANGE ( `a` ) (
  PARTITION `p0` VALUES LESS THAN (10),
  PARTITION `p1` VALUES LESS THAN (22),
  PARTITION `p2` VALUES LESS THAN (31),
  PARTITION `p_max` VALUES LESS THAN (MAXVALUE)
);

desc select * from t where a >= 22 and a < 31

We can leave this for future development.

@zhuo-zhi zhuo-zhi marked this pull request as ready for review April 27, 2021 11:36
@ti-chi-bot ti-chi-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 27, 2021
winoros
winoros reviewed Apr 28, 2021
View reviewed changes
Copy link
Member

@winoros winoros left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

others lgtm

planner/core/rule_partition_processor.go Outdated Show resolved Hide resolved
zhuo-zhi
zhuo-zhi commented Apr 28, 2021
View reviewed changes
Copy link
Contributor Author

@zhuo-zhi zhuo-zhi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done with more unit tests

@qw4990
Copy link
Contributor

qw4990 commented Apr 29, 2021

/merge

@ti-chi-bot
Copy link
Member

/run-all-tests

This bot automatically retries jobs that failed on can merge PRs (send feedback to hi-rustin).

Silence the bot with the /merge cancel comment for consistent failures.

@ti-chi-bot
Copy link
Member

/run-all-tests

This bot automatically retries jobs that failed on can merge PRs (send feedback to hi-rustin).

Silence the bot with the /merge cancel comment for consistent failures.

@ti-chi-bot
Copy link
Member

/run-all-tests

This bot automatically retries jobs that failed on can merge PRs (send feedback to hi-rustin).

Silence the bot with the /merge cancel comment for consistent failures.

@ti-chi-bot
Copy link
Member

/run-all-tests

This bot automatically retries jobs that failed on can merge PRs (send feedback to hi-rustin).

Silence the bot with the /merge cancel comment for consistent failures.

@ti-chi-bot
Copy link
Member

/run-all-tests

This bot automatically retries jobs that failed on can merge PRs (send feedback to hi-rustin).

Silence the bot with the /merge cancel comment for consistent failures.

@ti-chi-bot
Copy link
Member

/run-all-tests

This bot automatically retries jobs that failed on can merge PRs (send feedback to hi-rustin).

Silence the bot with the /merge cancel comment for consistent failures.

@ti-chi-bot
Copy link
Member

@zhuo-zhi: Your PR was out of date, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@sre-bot
Copy link
Contributor

sre-bot commented May 2, 2021

Please follow PR Title Format:

  • pkg [, pkg2, pkg3]: what's changed

Or if the count of mainly changed packages are more than 3, use

  • *: what's changed

1 similar comment
@sre-bot
Copy link
Contributor

sre-bot commented May 2, 2021

Please follow PR Title Format:

  • pkg [, pkg2, pkg3]: what's changed

Or if the count of mainly changed packages are more than 3, use

  • *: what's changed

@ti-chi-bot ti-chi-bot merged commit 207ce34 into pingcap:master May 2, 2021
@qw4990 qw4990 mentioned this pull request Jun 8, 2021
Closed
mjonss added a commit to mjonss/tidb that referenced this pull request Jun 10, 2022
@mjonss
Manual revert of pingcap#24282 / 207ce34
08fa343
@mjonss mjonss mentioned this pull request Jun 10, 2022
Merged
12 tasks
ti-chi-bot pushed a commit that referenced this pull request Jun 14, 2022
@mjonss
planner: Manual revert of #24282 (#35298)
2ba2a9e 
ref #22079, closes #33966, close #35181
@ti-srebot ti-srebot mentioned this pull request Aug 22, 2022
Merged
12 tasks
ti-chi-bot pushed a commit that referenced this pull request Aug 22, 2022
@ti-srebot
planner: Manual revert of #24282 (#35298) (#37268)
7ddd028 
ref #22079, closes #33966, close #35181
@ti-srebot ti-srebot mentioned this pull request Sep 22, 2022
Merged
12 tasks
ti-chi-bot pushed a commit that referenced this pull request Sep 22, 2022
@ti-srebot
planner: Manual revert of #24282 (#35298) (#38076)
6f15862 
ref #22079, closes #33966, close #35181
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@winoros winoros winoros approved these changes

@qw4990 qw4990 qw4990 approved these changes

@rebelice rebelice Awaiting requested review from rebelice

Assignees
No one assigned
Labels
sig/planner SIG: Planner size/L Denotes a PR that changes 100-499 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

remove invalid predicates after partition pruning
5 participants
@zhuo-zhi @qw4990 @ti-chi-bot @sre-bot @winoros