Storages: Refine memory tracker of data sharing (#8857) #8934
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is an automated cherry-pick of #8857
What problem does this PR solve?
Issue Number: close #8856
What is changed and how it works?
If
has_concurrent_reader
is true, we will consider the blocks to be read will being shared and setcurrent_memory_tracker
to null intiflash/dbms/src/Storages/DeltaMerge/File/DMFileReader.cpp
Lines 583 to 606 in fdf8bf2
But in reality, it may not true because there are also a
dt_max_sharing_column_count
limitations insideColumnSharingCacheMap
. This will causeshared_column_data_mem_tracker
to count data that will not actually be shared.This PR deprecated
dt_max_sharing_column_count
and only usedt_max_sharing_column_bytes_for_all
as the memory limitation of data-sharing.Only need to limit the sharing memory usage in
DMFileReader::readColumn
.Check List
Tests
Unit test
Integration test
Manual test (add detailed scripts or steps below)
select * from lineitem
concurrently.No code
Side effects
Documentation
Release note