[fix] (mem tracker) Fix some memory leaks, inaccurate statistics, core dump, deadlock bugs #10072
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
xinyiZzz
force-pushed
the
fix_memtracker_0608
branch
from
d9df8ce
to
590dc6a
Compare
morningman
added
kind/fix
morningman
reviewed
Jun 12, 2022
be/src/runtime/mem_tracker.cpp
Outdated
| @@ -124,14 +124,17 @@ std::shared_ptr<MemTracker> MemTracker::create_tracker_impl( | |||
| std::string reset_label; | |||
| MemTracker* task_parent_tracker = reset_parent->parent_task_mem_tracker(); | |||
| if (task_parent_tracker) { | |||
| reset_label = fmt::format("{}:{}", label, split(task_parent_tracker->label(), ":")[1]); | |||
| reset_label = fmt::format("{}&{}", label, split(task_parent_tracker->label(), "&")[1]); | |||
There was a problem hiding this comment.
Because : has many meanings as a separator in task mem tracker label, and & is only used to split out queryId or loadID or tabletID.
The latest code replaces & with #, which seems a little nicer :)
| } | ||
| } | ||
| for (auto tid : expired_tasks) { | ||
| // This means that after all RuntimeState is destructed, | ||
| // there are still task mem trackers that are get or register. | ||
| // The only known case: after an load task ends all fragments on a BE,`tablet_writer_open` is still | ||
| // called to create a channel, and the load task tracker will be re-registered in the channel open. | ||
| // https://github.com/apache/incubator-doris/issues/9905 | ||
| if (_task_mem_trackers[tid].use_count() == 1) { |
There was a problem hiding this comment.
_task_mem_trackers[tid] maybe nullptr here?
Because you add a case if (!it->second) before.
There was a problem hiding this comment.
yes it's a bug,
fixed, thks~
|
And need to rebase to make p0 happy |
xinyiZzz
force-pushed
the
fix_memtracker_0608
branch
from
590dc6a
to
6b3fede
Compare
done |
dataroaring
approved these changes
Jun 13, 2022
github-actions
bot
added
the
approved
|
PR approved by at least one committer and no changes requested. |
|
PR approved by anyone and no changes requested. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Proposed changes
Issue Number: close #10006 #10071
Problem Summary:
Fix the memory leak. When the load task is canceled, the
IndexChannelandNodeChannelmem trackers cannot be destructed in time.Fix Load task being frequently canceled by oom and inaccurate
LoadChannelmem tracker limit, and rewrite the variable name ofmem limitinLoadChannel.Fix core dump, when logout task mem tracker, phmap erase fails, resulting in repeated logout of the same tracker.
Fix the deadlock, when add_child_tracker mem limit exceeds, calling log_usage causes
_child_trackers_lockdeadlock.Fix frequent log printing when thread mem tracker limit exceeds, which will affect readability and performance.
Optimize some details of mem tracker display.
Checklist(Required)
Further comments
If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...