Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(storage): limit hummock write based on LSM tree stats #8383

Merged
merged 4 commits into from
Mar 12, 2023
Merged

feat(storage): limit hummock write based on LSM tree stats #8383

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
21f8ea0
feat(storage): limit hummock write based on LSM tree stats
zwang28 Mar 6, 2023
ad8e09b
rename
zwang28 Mar 7, 2023
d62d304
Merge branch 'main' into wangzheng/write_stall
zwang28 Mar 12, 2023
7deed26
refactor
zwang28 Mar 12, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
149 changes: 148 additions & 1 deletion dashboard/proto/gen/hummock.ts
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

48 changes: 38 additions & 10 deletions dashboard/proto/gen/meta.ts
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 12 additions & 0 deletions proto/hummock.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -519,6 +519,7 @@ message RiseCtlUpdateCompactionConfigRequest {
uint64 target_file_size_base = 7;
uint32 compaction_filter_mask = 8;
uint32 max_sub_compaction = 9;
uint64 level0_stop_write_threshold_sub_level_number = 10;
}
}
repeated uint64 compaction_group_ids = 1;
Expand Down Expand Up @@ -602,6 +603,8 @@ message CompactionConfig {
uint32 max_sub_compaction = 12;
uint64 max_space_reclaim_bytes = 13;
bool split_by_state_table = 14;
// soft limit for max number of sub level number
uint64 level0_stop_write_threshold_sub_level_number = 15;
}

message TableStats {
Expand All @@ -614,3 +617,12 @@ message HummockVersionStats {
uint64 hummock_version_id = 1;
map<uint32, TableStats> table_stats = 2;
}

message WriteLimits {
message WriteLimit {
repeated uint32 table_ids = 1;
string reason = 2;
}
// < compaction group id, write limit info >
map<uint64, WriteLimit> write_limits = 1;
}
Loading