executor: open childExec during execution for UnionExec #24899
Conversation
XuHuaiyu
added
type/enhancement
ti-chi-bot
added
the
size/M
| // promised to exit when reaching here (e.childIDChan been closed). | ||
| var firstErr error | ||
| for i := 0; i <= e.mu.maxOpenedChildID; i++ { | ||
| if err := e.children[i].Close(); err != nil && firstErr == nil { |
There was a problem hiding this comment.
will close a not opened children cause problems here?
There was a problem hiding this comment.
Looks like maxOpenedChildID can guarantee all children is opend.
|
[REVIEW NOTIFICATION] This pull request has been approved by:
To complete the pull request process, please ask the reviewers in the list to review by filling The full list of commands accepted by this bot can be found here. Reviewer can indicate their review by submitting an approval review. |
ti-chi-bot
added
status/LGT2
|
/merge |
|
This pull request has been accepted and is ready to merge. Commit hash: cc7ac97
|
ti-chi-bot
added
the
status/can-merge
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
|
cherry pick to release-5.1 in PR #25334 |
What problem does this PR solve?
Issue Number: close #21441
re-revert #21561
Problem Summary:
What is changed and how it works?
Proposal: xxx
What's Changed:
Before this commit,
Unionopens all the child executors during theOpenphase.This increases the overhead of
buildRequestand the risk of OOM.After this commit, the
Openof childExec inUnionis moved to theNextphase. The childExec be opened simultaneously is limited totidb_executor_concurrencyatmost.How it Works:
Related changes
N/A
Check List
Tests
Suppose we have a table
t, it contains 100 partitions, each partition contains 10,000,000 tuples.The result of
select * from t;Before:
After:
The query works fine.
Side effects
tidb_executor_concurrency.Release note
UnionExecto reduce the overhead ofUnionExec.