RFC + executor: Support global memory tracker #16777

Yisaer · 2020-04-23T15:11:47Z

What problem does this PR solve?

UCP: #15407
Issue Number: close #15407

Support Global Memory tracker to control the memory usage and kill the current session's if the global memory usage in tracker exceeds the limit.

What is changed and how it works?

Currently, we have already support the Global Disk Tracker for Executor And the Support of Global memory Tracker for Executor is quite likely the same.

Proposal

see #17103

Related changes

PR to update pingcap/docs/pingcap/docs-cn:
Need to cherry-pick to the release branch

Check List

Tests

Unit test

Release note

Support Global Memory tracker for tidb-server

…cker

executor/executor.go

sre-bot · 2020-04-23T16:21:34Z

Thanks for your contribution. If your PR get merged, you will be rewarded 3488 points.

Yisaer · 2020-05-12T11:34:08Z

/run-all-tests

Yisaer · 2020-05-12T11:39:52Z

/run-unit-test

lzmhhh123 · 2020-05-13T07:01:52Z

util/memory/tracker.go

+	t.parMu.Lock()
+	defer t.parMu.Unlock()


Why don't we lock the parent in functions like Consume, instead, to lock it here?

Lock parent is my first trial when I found the data race error, maybe adding lock in the Consume function is also enough. But the tpcc test result already seems ok if we lock parent here.

lzmhhh123

LGTM

sre-bot · 2020-05-13T07:42:34Z

/run-all-tests

sre-bot · 2020-05-13T07:49:58Z

Team Yisaer complete task #15407 and get 3488 score, current score 4038

SunRunAway · 2020-05-13T07:52:25Z

@Yisaer Would you like to add documentation related to this PR?

Yisaer · 2020-05-13T07:54:35Z

@SunRunAway ok.

sre-bot · 2020-05-13T09:27:36Z

Benchmark Report

Run Sysbench Performance Test on VMs

@@                               Benchmark Diff                               @@
================================================================================
--- tidb: 3f2d35a60bbc37382d991ecbcd48ebec9b6a3f8c
+++ tidb: 12596f1a74b6a6c4079423536817a57c4b853c7d
tikv: 58e5b3a1f39bf6473b06ed41c3371b64c7c374be
pd: 0e3d21884c952afe96af96b8c4f0ef851d5a9204
================================================================================
oltp_update_index:
    * QPS: 4449.87 ± 3.83% (std=104.04) delta: 1.51% (p=0.593)
    * Latency p50: 28.78 ± 3.94% (std=0.69) delta: -1.59%
    * Latency p99: 54.62 ± 5.95% (std=2.04) delta: -3.13%
            
oltp_insert:
    * QPS: 7102.58 ± 0.41% (std=17.68) delta: -0.06% (p=0.860)
    * Latency p50: 17.99 ± 0.13% (std=0.02) delta: -0.08%
    * Latency p99: 31.09 ± 0.90% (std=0.28) delta: -0.04%
            
oltp_read_write:
    * QPS: 14392.08 ± 0.03% (std=3.79) delta: -0.16% (p=0.842)
    * Latency p50: 178.17 ± 0.13% (std=0.15) delta: 0.12%
    * Latency p99: 331.91 ± 0.00% (std=0.00) delta: -0.58%
            
oltp_point_select:
    * QPS: 41406.07 ± 0.08% (std=25.29) delta: 3.20% (p=0.143)
    * Latency p50: 3.09 ± 0.00% (std=0.00) delta: -3.13%
    * Latency p99: 10.60 ± 2.24% (std=0.16) delta: -1.78%
            
oltp_update_non_index:
    * QPS: 4732.56 ± 6.86% (std=201.32) delta: -3.38% (p=0.619)
    * Latency p50: 27.09 ± 7.18% (std=1.19) delta: 3.68%
    * Latency p99: 41.35 ± 1.21% (std=0.35) delta: 0.10%

Yisaer · 2020-09-08T04:05:15Z

/label needs-cherry-pick-4.0

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>

ti-srebot · 2020-09-08T04:05:55Z

cherry pick to release-4.0 in PR #19861

zz-jason · 2020-09-08T05:05:09Z

@Yisaer @SunRunAway Is there any QA test to cover this new change?

zz-jason · 2020-09-08T05:09:51Z

BTW, please add a user document for it.

@Yisaer

Since @Yisaer has contributed a lot to sig-exec, according to sig-exec [roles-and-organization-management.md#promotion](https://github.com/pingcap/community/blob/master/special-interest-groups/sig-exec/roles-and-organization-management.md#from-reviewer-to-committer), we would like to promote him to our new committer, and his detailed contribution is shown below: 1. contribute [20+](https://github.com/pingcap/tidb/pulls?q=is%3Apr+author%3Ayisaer+label%3Astatus%2Fcan-merge+label%3Asig%2Fexecution) PRs to the executor, 2. support the [global memory tracker](pingcap/tidb#16777) (a medium task) 4. support the ratelimit action to solve the [table reader oom problem](pingcap/tidb#16104) (a hard task) 5. help other contributors and review [25+](https://github.com/pingcap/tidb/pulls?q=is%3Apr+reviewed-by%3Ayisaer+-author%3Ayisaer) PRs. Thanks for his contribution!!

@Yisaer

Since @Yisaer has contributed a lot to sig-exec, according to sig-exec [roles-and-organization-management.md#promotion](https://github.com/pingcap/community/blob/master/special-interest-groups/sig-exec/roles-and-organization-management.md#from-reviewer-to-committer), we would like to promote him to our new committer, and his detailed contribution is shown below: 1. contribute [20+](https://github.com/pingcap/tidb/pulls?q=is%3Apr+author%3Ayisaer+label%3Astatus%2Fcan-merge+label%3Asig%2Fexecution) PRs to the executor, 2. support the [global memory tracker](pingcap/tidb#16777) (a medium task) 4. support the ratelimit action to solve the [table reader oom problem](pingcap/tidb#16104) (a hard task) 5. help other contributors and review [25+](https://github.com/pingcap/tidb/pulls?q=is%3Apr+reviewed-by%3Ayisaer+-author%3Ayisaer) PRs. Thanks for his contribution!!

@Yisaer

Since @Yisaer has contributed a lot to sig-exec, according to sig-exec [roles-and-organization-management.md#promotion](https://github.com/pingcap/community/blob/master/special-interest-groups/sig-exec/roles-and-organization-management.md#from-reviewer-to-committer), we would like to promote him to our new committer, and his detailed contribution is shown below: 1. contribute [20+](https://github.com/pingcap/tidb/pulls?q=is%3Apr+author%3Ayisaer+label%3Astatus%2Fcan-merge+label%3Asig%2Fexecution) PRs to the executor, 2. support the [global memory tracker](pingcap/tidb#16777) (a medium task) 4. support the ratelimit action to solve the [table reader oom problem](pingcap/tidb#16104) (a hard task) 5. help other contributors and review [25+](https://github.com/pingcap/tidb/pulls?q=is%3Apr+reviewed-by%3Ayisaer+-author%3Ayisaer) PRs. Thanks for his contribution!!

Song Gao added 10 commits April 9, 2020 10:51

current

e466d26

lru memconsume

8530d08

now

17be621

Merge remote-tracking branch 'upstream/master' into global_memory_tra…

736abd0

…cker

move global tracker

ecb2329

tracker

5fa2a3d

revise

3fc17ce

remove memtracker

08e309a

fix import

e063b1f

trackerrecorder

2355ce4

Yisaer requested a review from a team as a code owner April 23, 2020 15:11

ghost requested review from wshwsh12 and removed request for a team April 23, 2020 15:11

Yisaer marked this pull request as draft April 23, 2020 15:11

Yisaer commented Apr 23, 2020

View reviewed changes

executor/executor.go Outdated Show resolved Hide resolved

Yisaer commented Apr 23, 2020

View reviewed changes

executor/executor.go Outdated Show resolved Hide resolved

github-actions bot added the sig/execution SIG execution label Apr 23, 2020

Song Gao added 9 commits April 23, 2020 23:16

remove error

419ebb2

replaceUsed

b891abd

remove useless

2998464

add note

8f89ca7

refix name

89513e7

Update trackerRecorder.go

875e27a

change back tracker

4d73672

fix name

052c2fa

fix lint

cb70d5a

Yisaer changed the title ~~[Draft] Global Mem tracker~~ [Draft] executor: add global memory tracker Apr 23, 2020

add todo

7be9e4d

zz-jason requested a review from eurekaka April 24, 2020 03:23

Merge branch 'master' into global_memory_tracker

41df8be

lzmhhh123 reviewed May 13, 2020

View reviewed changes

Yisaer requested a review from lzmhhh123 May 13, 2020 07:19

lzmhhh123 approved these changes May 13, 2020

View reviewed changes

lzmhhh123 added the status/LGT2 Indicates that a PR has LGTM 2. label May 13, 2020

wshwsh12 removed their request for review May 13, 2020 07:37

lzmhhh123 added the status/can-merge Indicates a PR has been approved by a committer. label May 13, 2020

Merge branch 'master' into global_memory_tracker

12596f1

sre-bot merged commit 3f2d35a into pingcap:master May 13, 2020

jackysp mentioned this pull request May 14, 2020

*: fix setHeapProfileTracker panic on Windows #17226

Merged

Yisaer mentioned this pull request Jun 1, 2020

Revise the Configuration for SimpleLRUCache #17567

Closed

ti-srebot added the needs-cherry-pick-4.0 label Sep 8, 2020

ti-srebot pushed a commit to ti-srebot/tidb that referenced this pull request Sep 8, 2020

cherry pick pingcap#16777 to release-4.0

f27d448

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>

ti-srebot mentioned this pull request Sep 8, 2020

RFC + executor: Support global memory tracker (#16777) #19861

Closed

SunRunAway mentioned this pull request Nov 26, 2020

Propose @Yisaer as committer of sig-exec pingcap/community#353

Closed

SunRunAway mentioned this pull request Nov 27, 2020

Propose @Yisaer as committer of sig-exec pingcap/community#355

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

RFC + executor: Support global memory tracker #16777

RFC + executor: Support global memory tracker #16777

Yisaer commented Apr 23, 2020 •

edited by SunRunAway

Loading

sre-bot commented Apr 23, 2020

Yisaer commented May 12, 2020

Yisaer commented May 12, 2020

lzmhhh123 May 13, 2020

Yisaer May 13, 2020 •

edited

Loading

lzmhhh123 left a comment

sre-bot commented May 13, 2020

sre-bot commented May 13, 2020

SunRunAway commented May 13, 2020

Yisaer commented May 13, 2020

sre-bot commented May 13, 2020

Yisaer commented Sep 8, 2020

ti-srebot commented Sep 8, 2020

zz-jason commented Sep 8, 2020

zz-jason commented Sep 8, 2020

RFC + executor: Support global memory tracker #16777

RFC + executor: Support global memory tracker #16777

Conversation

Yisaer commented Apr 23, 2020 • edited by SunRunAway Loading

What problem does this PR solve?

What is changed and how it works?

Proposal

Related changes

Check List

Release note

sre-bot commented Apr 23, 2020

Yisaer commented May 12, 2020

Yisaer commented May 12, 2020

lzmhhh123 May 13, 2020

Choose a reason for hiding this comment

Yisaer May 13, 2020 • edited Loading

Choose a reason for hiding this comment

lzmhhh123 left a comment

Choose a reason for hiding this comment

sre-bot commented May 13, 2020

sre-bot commented May 13, 2020

SunRunAway commented May 13, 2020

Yisaer commented May 13, 2020

sre-bot commented May 13, 2020

Benchmark Report

Yisaer commented Sep 8, 2020

ti-srebot commented Sep 8, 2020

zz-jason commented Sep 8, 2020

zz-jason commented Sep 8, 2020

Yisaer commented Apr 23, 2020 •

edited by SunRunAway

Loading

Yisaer May 13, 2020 •

edited

Loading