Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

config: add hot region history configurations #4019

Merged
merged 7 commits into from
Sep 3, 2021
Merged

config: add hot region history configurations #4019

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
f7b4a6a
config:add hot_regions_write_interval&hot_regions_remian_days in to c…
qidi1 Aug 23, 2021
e030e96
change config part
qidi1 Aug 23, 2021
ab00dd4
config:merge from remote
qidi1 Sep 1, 2021
f24060e
config:remove old code
qidi1 Sep 2, 2021
243c5d6
config:change discprtion
qidi1 Sep 2, 2021
d42ac6f
config:change default
qidi1 Sep 2, 2021
210ba1b
Merge branch 'master' into history_hot_region_config
ti-chi-bot Sep 3, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions conf/config.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,10 @@
## When PD fails to receive the heartbeat from a store after the specified period of time,
## it adds replicas at other nodes.
# max-store-down-time = "30m"
## Controls the time interval between write hot regions info into leveldb
# hot-regions-write-interval= "10m"
## The day of hot regions data to be reserved. 0 means close.
# hot-regions-reserved-days= "7"
## The number of Leader scheduling tasks performed at the same time.
# leader-schedule-limit = 4
## The number of Region scheduling tasks performed at the same time.
Expand Down
16 changes: 16 additions & 0 deletions server/config/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -737,6 +737,12 @@ type ScheduleConfig struct {
// is overwritten, the value is fixed until it is deleted.
// Default: manual
StoreLimitMode string `toml:"store-limit-mode" json:"store-limit-mode"`

// Controls the time interval between write hot regions info into leveldb.
HotRegionsWriteInterval typeutil.Duration `toml:"hot-regions-write-interval" json:"hot-regions-write-interval"`

// The day of hot regions data to be reserved. 0 means close.
HotRegionsResevervedDays int64 `toml:"hot-regions-reserved-days" json:"hot-regions-reserved-days"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use typeutil.Duration?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The unit of this is days, does toml support days?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, the minimum time of reserving the hot regions is 1 day?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes,the minimum time of reserving the hot regions is 1 day

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if we just want to keep the history for several hours?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we delete history hot region every 24 hours,if we want reserve several hours,we must change delete function

}

// Clone returns a cloned scheduling configuration.
Expand Down Expand Up @@ -782,6 +788,8 @@ const (
defaultStoreLimitMode = "manual"
defaultEnableJointConsensus = true
defaultEnableCrossTableMerge = true
defaultHotRegionsWriteInterval = 10 * time.Minute
defaultHotRegionsResevervedDays = 7
)

func (c *ScheduleConfig) adjust(meta *configMetaData, reloading bool) error {
Expand Down Expand Up @@ -863,6 +871,14 @@ func (c *ScheduleConfig) adjust(meta *configMetaData, reloading bool) error {
c.StoreLimit = make(map[uint64]StoreLimitConfig)
}

if !meta.IsDefined("hot-regions-write-interval") {
adjustDuration(&c.HotRegionsWriteInterval, defaultHotRegionsWriteInterval)
}

if !meta.IsDefined("hot-regions-reserved-days") {
adjustInt64(&c.HotRegionsResevervedDays, defaultHotRegionsResevervedDays)
}

return c.Validate()
}

Expand Down
15 changes: 15 additions & 0 deletions server/config/config_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -460,6 +460,21 @@ wait-store-timeout = "120s"
c.Assert(cfg.ReplicationMode.ReplicationMode, Equals, "majority")
}

func (s *testConfigSuite) TestHotRegionConfig(c *C) {
cfgData := `
[schedule]
hot-regions-reserved-days= 30
hot-regions-write-interval= "30m"
`
cfg := NewConfig()
meta, err := toml.Decode(cfgData, &cfg)
c.Assert(err, IsNil)
err = cfg.Adjust(&meta, false)
c.Assert(err, IsNil)
c.Assert(cfg.Schedule.HotRegionsWriteInterval.Duration, Equals, time.Minute*30)
c.Assert(cfg.Schedule.HotRegionsResevervedDays, Equals, int64(30))
}

func (s *testConfigSuite) TestConfigClone(c *C) {
cfg := &Config{}
cfg.Adjust(nil, false)
Expand Down