Skip to content

[Bug] Fix tablet shared ptr circular reference causing the tablet not to be cleared #5100

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
morningman merged 1 commit into from
Dec 18, 2020
Merged

[Bug] Fix tablet shared ptr circular reference causing the tablet not to be cleared #5100

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 14 additions & 4 deletions be/src/olap/olap_server.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -344,12 +344,15 @@ void StorageEngine::_compaction_tasks_producer_callback() {
[=] { return _wakeup_producer_flag == 1; });
continue;
}

/// Regardless of whether the tablet is submitted for compaction or not,
/// we need to call 'reset_compaction' to clean up the base_compaction or cumulative_compaction objects
/// in the tablet, because these two objects store the tablet's own shared_ptr.
/// If it is not cleaned up, the reference count of the tablet will always be greater than 1,
/// thus cannot be collected by the garbage collector. (TabletManager::start_trash_sweep)
for (const auto& tablet : tablets_compaction) {
int64_t permits = tablet->prepare_compaction_and_calculate_permits(compaction_type, tablet);
if (permits == 0) {
continue;
}
if (_permit_limiter.request(permits)) {
if (permits > 0 && _permit_limiter.request(permits)) {
{
// Push to _tablet_submitted_compaction before submitting task
std::unique_lock<std::mutex> lock(_tablet_submitted_compaction_mutex);
Expand All @@ -371,10 +374,17 @@ void StorageEngine::_compaction_tasks_producer_callback() {
_wakeup_producer_flag = 1;
_compaction_producer_sleep_cv.notify_one();
}
// reset compaction
tablet->reset_compaction(compaction_type);
});
if (!st.ok()) {
_permit_limiter.release(permits);
// reset compaction
tablet->reset_compaction(compaction_type);
}
} else {
// reset compaction
tablet->reset_compaction(compaction_type);
}
}
} else {
Expand Down
8 changes: 8 additions & 0 deletions be/src/olap/tablet.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1426,4 +1426,12 @@ void Tablet::execute_compaction(CompactionType compaction_type) {
}
}

void Tablet::reset_compaction(CompactionType compaction_type) {
if (compaction_type == CompactionType::CUMULATIVE_COMPACTION) {
_cumulative_compaction.reset();
} else {
_base_compaction.reset();
}
}

} // namespace doris
1 change: 1 addition & 0 deletions be/src/olap/tablet.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -246,6 +246,7 @@ class Tablet : public BaseTablet {

int64_t prepare_compaction_and_calculate_permits(CompactionType compaction_type, TabletSharedPtr tablet);
void execute_compaction(CompactionType compaction_type);
void reset_compaction(CompactionType compaction_type);

void set_clone_occurred(bool clone_occurred) { _is_clone_occurred = clone_occurred; }
bool get_clone_occurred() { return _is_clone_occurred; }
Expand Down