GetExplainRowsForPlan called twice for the same query

[mjonss]

Bug Report

When profiling a SELECT * FROM partitioned_table I found that GetExplainRowsForPlan was called twice and took about 25% of the time when executing the query.

I wonder if it is even needed to call at all, since the plan is already stored, and could possibly be extended with GetExplainRowsForPlan when needed by EXPLAIN FOR ...

Found when first hacking out/disable reArrangeFallback in #30353 to see what else could be made faster.

1. Minimal reproduce step (Required)

create table tp (a int primary key, b varchar(255)) partition by hash (a) partitions 8192;
insert into tp values (1, 'row with id 1, partition p1 filler data'), (2, 'partition p2, filler data for row with id 2'), (9, 'filler data for row with id 9, partition p9');
-- start profiling and run this over and over again for profiling:
select * from tp;

2. What did you expect to see? (Required)

GetExplainRowsForPlan to be called once, or not at all, except when EXPLAIN FOR... would be called.

3. What did you see instead (Required)

GetExplainRowsForPlan was called twice per query.

4. What is your TiDB version? (Required)

tidb_version(): Release Version: v5.4.0-alpha-289-g92c7c075d5-dirty
Edition: Community
Git Commit Hash: 92c7c07
Git Branch: master
UTC Build Time: 2021-12-02 10:39:36
GoVersion: go1.17
Race Enabled: false
TiKV Min Version: v3.0.0-60965b006877ca7234adaced7890d7b029ed1306
Check Table Before Drop: false

Some flame graphs and cumulative profile:

[Reminiscent]

@mjonss How to disable reArrangeFallback for the test?

[mjonss]

@mjonss How to disable reArrangeFallback for the test?

Something hackish like this:

diff --git a/util/memory/tracker.go b/util/memory/tracker.go
index 470029e309..9d861e81c7 100644
--- a/util/memory/tracker.go
+++ b/util/memory/tracker.go
@@ -159,17 +159,23 @@ func (t *Tracker) SetActionOnExceed(a ActionOnExceed) {
 // FallbackOldAndSetNewAction sets the action when memory usage exceeds bytesHardLimit
 // and set the original action as its fallback.
 func (t *Tracker) FallbackOldAndSetNewAction(a ActionOnExceed) {
-       t.actionMuForHardLimit.Lock()
-       defer t.actionMuForHardLimit.Unlock()
-       t.actionMuForHardLimit.actionOnExceed = reArrangeFallback(t.actionMuForHardLimit.actionOnExceed, a)
+       return
+       /*
+               t.actionMuForHardLimit.Lock()
+               defer t.actionMuForHardLimit.Unlock()
+               t.actionMuForHardLimit.actionOnExceed = reArrangeFallback(t.actionMuForHardLimit.actionOnExceed, a)
+       */
 }
 
 // FallbackOldAndSetNewActionForSoftLimit sets the action when memory usage exceeds bytesSoftLimit
 // and set the original action as its fallback.
 func (t *Tracker) FallbackOldAndSetNewActionForSoftLimit(a ActionOnExceed) {
-       t.actionMuForSoftLimit.Lock()
-       defer t.actionMuForSoftLimit.Unlock()
-       t.actionMuForSoftLimit.actionOnExceed = reArrangeFallback(t.actionMuForSoftLimit.actionOnExceed, a)
+       return
+       /*
+               t.actionMuForSoftLimit.Lock()
+               defer t.actionMuForSoftLimit.Unlock()
+               t.actionMuForSoftLimit.actionOnExceed = reArrangeFallback(t.actionMuForSoftLimit.actionOnExceed, a)
+       */
 }
 
 // GetFallbackForTest get the oom action used by test.
@@ -306,7 +312,8 @@ func (t *Tracker) ReplaceChild(oldChild, newChild *Tracker) {
 // which means this is a memory release operation. When memory usage of a tracker
 // exceeds its bytesSoftLimit/bytesHardLimit, the tracker calls its action, so does each of its ancestors.
 func (t *Tracker) Consume(bytes int64) {
-       if bytes == 0 {
+
+       if bytes != 999999999 {
                return
        }
        var rootExceed, rootExceedForSoftLimit *Tracker