Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

statistics, schedulers: split hot cache to each store #1641

Merged
merged 24 commits into from
Aug 6, 2019

Conversation

nolouch
Copy link
Contributor

@nolouch nolouch commented Jul 16, 2019

What problem does this PR solve?

Our concert about hotspot should the hot store then the hot region more reasonable.

What is changed and how it works?

hot cache as a statistic for each store.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)

@nolouch nolouch force-pushed the hot-sche-to-store branch from 3136d89 to e347d3a Compare July 16, 2019 08:52
Signed-off-by: nolouch <nolouch@gmail.com>
@nolouch nolouch force-pushed the hot-sche-to-store branch from e347d3a to b262392 Compare July 16, 2019 08:56
nolouch added 2 commits July 16, 2019 17:31
Signed-off-by: nolouch <nolouch@gmail.com>
Signed-off-by: nolouch <nolouch@gmail.com>
server/statistics/hot_cache.go Outdated Show resolved Hide resolved
server/statistics/hot_cache.go Outdated Show resolved Hide resolved
hotRegionThreshold := calculateWriteHotThreshold(stats)
return w.isNeedUpdateStatCache(region, WrittenBytesPerSec, hotRegionThreshold, value, WriteFlow)
for _, storeID := range storeIDs {
WrittenBytesPerSec = uint64(float64(region.GetBytesWritten()) / float64(RegionHeartBeatReportInterval))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be calculated before the for loop

server/statistics/hot_cache.go Show resolved Hide resolved
server/statistics/hot_cache.go Outdated Show resolved Hide resolved
server/statistics/hot_cache.go Show resolved Hide resolved
@nolouch nolouch changed the title [DNM] statistics, schedulers: split hot cache to each store [DNM] Jul 26, 2019
@nolouch nolouch added the DNM label Jul 26, 2019
@nolouch nolouch changed the title [DNM] statistics, schedulers: split hot cache to each store Jul 26, 2019
server/statistics/hot_cache.go Show resolved Hide resolved
server/statistics/hot_cache.go Outdated Show resolved Hide resolved
server/statistics/hot_cache.go Outdated Show resolved Hide resolved
server/statistics/hot_cache.go Outdated Show resolved Hide resolved
nolouch added 5 commits July 26, 2019 19:51
Signed-off-by: nolouch <nolouch@gmail.com>
Signed-off-by: nolouch <nolouch@gmail.com>
Signed-off-by: nolouch <nolouch@gmail.com>
Signed-off-by: nolouch <nolouch@gmail.com>
Signed-off-by: nolouch <nolouch@gmail.com>
server/statistics/hot_cache.go Outdated Show resolved Hide resolved
server/statistics/hot_cache.go Outdated Show resolved Hide resolved
nolouch added 4 commits August 1, 2019 20:07
Signed-off-by: nolouch <nolouch@gmail.com>
Signed-off-by: nolouch <nolouch@gmail.com>
Signed-off-by: nolouch <nolouch@gmail.com>
Signed-off-by: nolouch <nolouch@gmail.com>
@nolouch nolouch removed the DNM label Aug 2, 2019
@nolouch
Copy link
Contributor Author

nolouch commented Aug 2, 2019

PTAL @Luffbee @jiyingtk @shafreeck

Copy link
Contributor

@Luffbee Luffbee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why remove region flow metrics?

@nolouch
Copy link
Contributor Author

nolouch commented Aug 2, 2019

@shafreeck Usually a metric traces a stable object, too much region may affect the performance.

Signed-off-by: nolouch <nolouch@gmail.com>
@nolouch
Copy link
Contributor Author

nolouch commented Aug 5, 2019

PTAL @rleungx @shafreeck

server/statistics/store.go Show resolved Hide resolved
server/statistics/hot_cache.go Show resolved Hide resolved
Signed-off-by: nolouch <nolouch@gmail.com>
@nolouch
Copy link
Contributor Author

nolouch commented Aug 6, 2019

PTAL @shafreeck @Luffbee

@nolouch nolouch merged commit 82f1dd1 into tikv:master Aug 6, 2019
@nolouch nolouch deleted the hot-sche-to-store branch August 6, 2019 09:51
sre-bot pushed a commit that referenced this pull request Jan 19, 2020
sre-bot pushed a commit that referenced this pull request Jan 19, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants