Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

statistics: pd can calculate the hot degree of the buckets. #4727

Merged
merged 10 commits into from
May 19, 2022

Conversation

bufferflies
Copy link
Contributor

@bufferflies bufferflies commented Mar 15, 2022

What problem does this PR solve?

Issue Number: Close #4726

What is changed and how it works?

using btree to store the map relation betweeen the key range and buckekItem.
using map to store the map relation between the region-id and bucketItem.

every bucket has own hotgree and inherits the highest degree of the overlaps.
suck like this:

origin buckets
|--10--|--20--|--30--|--40--|
|--2--|--3--|--4--|--5--|

after inheriting
|--10-|--30--|--35--|
|--3-|--3--|--4--|

Check List

Tests

  • Unit test
  • Integration test

Code changes

Side effects

Related changes

Release note

pd can calculate the hot degree of the buckets.

@ti-chi-bot
Copy link
Member

ti-chi-bot commented Mar 15, 2022

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • HunDunDM
  • nolouch

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added the release-note Denotes a PR that will be considered when it comes time to generate release notes. label Mar 15, 2022
@bufferflies bufferflies force-pushed the feature/bucket-hot branch 13 times, most recently from 7599e80 to d145071 Compare March 15, 2022 07:34
@codecov
Copy link

codecov bot commented Mar 15, 2022

Codecov Report

Merging #4727 (d68802f) into master (563a93d) will decrease coverage by 0.07%.
The diff coverage is 84.53%.

@@            Coverage Diff             @@
##           master    #4727      +/-   ##
==========================================
- Coverage   75.43%   75.36%   -0.08%     
==========================================
  Files         303      306       +3     
  Lines       30015    30194     +179     
==========================================
+ Hits        22643    22756     +113     
- Misses       5386     5436      +50     
- Partials     1986     2002      +16     
Flag Coverage Δ
unittests 75.36% <84.53%> (-0.08%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
server/cluster/cluster_worker.go 68.96% <50.00%> (-0.95%) ⬇️
server/statistics/buckets/hot_bucket_task.go 50.00% <50.00%> (ø)
server/statistics/buckets/hot_bucket_cache.go 82.41% <82.41%> (ø)
server/statistics/buckets/bucket_stat_informer.go 91.17% <91.17%> (ø)
pkg/keyutil/util.go 100.00% <100.00%> (ø)
pkg/rangetree/range_tree.go 98.30% <100.00%> (+0.05%) ⬆️
server/cluster/cluster.go 83.02% <100.00%> (+0.39%) ⬆️
pkg/metricutil/metricutil.go 82.75% <0.00%> (-10.35%) ⬇️
server/tso/local_allocator.go 64.86% <0.00%> (-6.76%) ⬇️
server/tso/tso.go 68.92% <0.00%> (-5.65%) ⬇️
... and 11 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 563a93d...d68802f. Read the comment docs.

@ti-chi-bot ti-chi-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 16, 2022
@ti-chi-bot ti-chi-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 29, 2022
@bufferflies bufferflies force-pushed the feature/bucket-hot branch 2 times, most recently from 4fe144d to 5864e71 Compare March 29, 2022 10:27
@ti-chi-bot ti-chi-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 1, 2022
@ti-chi-bot ti-chi-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 27, 2022
@bufferflies bufferflies force-pushed the feature/bucket-hot branch 3 times, most recently from c3ffc6d to 78c504a Compare April 28, 2022 06:49
@bufferflies bufferflies force-pushed the feature/bucket-hot branch 2 times, most recently from 4e624da to 9b36a67 Compare May 6, 2022 02:55
Signed-off-by: bufferflies <1045931706@qq.com>
Copy link
Contributor

@nolouch nolouch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label May 19, 2022
Signed-off-by: bufferflies <1045931706@qq.com>
Signed-off-by: bufferflies <1045931706@qq.com>
@ti-chi-bot ti-chi-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 19, 2022
pkg/keyutil/util.go Outdated Show resolved Hide resolved
Signed-off-by: bufferflies <1045931706@qq.com>
@ti-chi-bot ti-chi-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 19, 2022
@bufferflies bufferflies force-pushed the feature/bucket-hot branch 2 times, most recently from 50ff2f3 to a5b0830 Compare May 19, 2022 07:25
@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels May 19, 2022
Signed-off-by: bufferflies <1045931706@qq.com>
@bufferflies
Copy link
Contributor Author

/merge

@ti-chi-bot
Copy link
Member

@bufferflies: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: d68802f

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label May 19, 2022
@ti-chi-bot ti-chi-bot merged commit 61842c3 into tikv:master May 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-note Denotes a PR that will be considered when it comes time to generate release notes. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

pd can calculate the hot degree of the buckets.
6 participants