Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

UCP: make memory usage of executors fit global memory limitation #15407

Closed
SunRunAway opened this issue Mar 16, 2020 · 8 comments · Fixed by #16777
Closed

UCP: make memory usage of executors fit global memory limitation #15407

SunRunAway opened this issue Mar 16, 2020 · 8 comments · Fixed by #16777
Assignees
@fzhedu @Yisaer
Labels
epic/memory-management feature/accepted This feature request is accepted by product managers sig/execution SIG execution type/feature-request Categorizes issue or PR as related to a new feature.

Comments

@SunRunAway
Copy link
Contributor

SunRunAway commented Mar 16, 2020 •
edited
Loading

Description

Now nearly every executor has been tracking its memory usage using memory.Tracker. And here's a max-memory configuration but is not taking effect on executors.

# Max memory size to use, 0 use the total usable memory in the machine.
max-memory = 0

If the memory usage of a SQL is exceeded by the global memory limitation, we need either,

  1. kill this SQL
  2. kill the SQL which is consuming the most memory

Tips:

  1. We may need to maintain a global memory.Tracker
  2. max-memory is currently used by PrepareCache, we may need to find a way to re-implement it.

Score

  • 3488

Mentor(s)

Contact the mentors: #tidb-challenge-program channel in TiDB Community Slack Workspace

Recommended Skills

  • skills 1
  • skills 1

Learning Materials
@zz-jason zz-jason added type/feature-request Categorizes issue or PR as related to a new feature. and removed type/new-feature labels Apr 3, 2020
@Yisaer
Copy link
Contributor

Yisaer commented Apr 8, 2020

/pick-up-challenge

@sre-bot
Copy link
Contributor

sre-bot commented Apr 8, 2020

@Yisaer pick up issue success

@sre-bot
Copy link
Contributor

sre-bot commented Apr 15, 2020

This pick has been automatically canceled after more than a week.

@Yisaer
Copy link
Contributor

Yisaer commented Apr 16, 2020

/pick-up-challenge

@sre-bot
Copy link
Contributor

sre-bot commented Apr 16, 2020

@Yisaer pick up issue success

@bb7133 bb7133 changed the title Make memory usage of executors fit global memory limitation UCP: make memory usage of executors fit global memory limitation Apr 17, 2020
@sre-bot
Copy link
Contributor

sre-bot commented Apr 23, 2020

This pick has been automatically canceled after more than a week.

@Yisaer
Copy link
Contributor

Yisaer commented Apr 23, 2020

/pick-up-challenge

@sre-bot
Copy link
Contributor

sre-bot commented Apr 23, 2020

@Yisaer pick up issue success

@Yisaer Yisaer mentioned this issue Apr 23, 2020
Merged
@zz-jason zz-jason added the feature/accepted This feature request is accepted by product managers label Jul 29, 2020
@ti-srebot ti-srebot mentioned this issue Sep 8, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@fzhedu fzhedu

@Yisaer Yisaer

Labels
epic/memory-management feature/accepted This feature request is accepted by product managers sig/execution SIG execution type/feature-request Categorizes issue or PR as related to a new feature.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

RFC + executor: Support global memory tracker Yisaer/tidb
5 participants
@SunRunAway @zz-jason @fzhedu @Yisaer @sre-bot