datarace in test about store label

[lhy1024]

Flaky Test

Which jobs are failing

TestHotWithoutHotPeer

2023-12-12T08:34:49.8593756Z ==================
2023-12-12T08:34:49.8594085Z WARNING: DATA RACE
2023-12-12T08:34:49.8594485Z Read at 0x00c0074a27a0 by goroutine 16076:
2023-12-12T08:34:49.8595135Z   github.com/tikv/pd/pkg/schedule/placement.MatchLabelConstraints()
2023-12-12T08:34:49.8596042Z       /home/runner/work/pd/pd/pkg/schedule/placement/label_constraint.go:87 +0x1aa
2023-12-12T08:34:49.8597324Z   github.com/tikv/pd/pkg/schedule/placement.(*fitWorker).fitRule.checkRule.func1()
2023-12-12T08:34:49.8598292Z       /home/runner/work/pd/pd/pkg/schedule/placement/rule_manager.go:794 +0xbc
2023-12-12T08:34:49.8599001Z   github.com/tikv/pd/pkg/slice.AnyOf[go.shape.*uint8]()
2023-12-12T08:34:49.8599685Z       /home/runner/work/pd/pd/pkg/slice/slice.go:20 +0xb3
2023-12-12T08:34:49.8600278Z   github.com/tikv/pd/pkg/schedule/placement.checkRule()
2023-12-12T08:34:49.8601067Z       /home/runner/work/pd/pd/pkg/schedule/placement/rule_manager.go:793 +0x27c
2023-12-12T08:34:49.8601901Z   github.com/tikv/pd/pkg/schedule/placement.(*fitWorker).fitRule()
2023-12-12T08:34:49.8602685Z       /home/runner/work/pd/pd/pkg/schedule/placement/fit.go:243 +0x125
2023-12-12T08:34:49.8603382Z   github.com/tikv/pd/pkg/schedule/placement.(*fitWorker).run()
2023-12-12T08:34:49.8604207Z       /home/runner/work/pd/pd/pkg/schedule/placement/fit.go:222 +0x84
2023-12-12T08:34:49.8604821Z   github.com/tikv/pd/pkg/schedule/placement.fitRegion()
2023-12-12T08:34:49.8605532Z       /home/runner/work/pd/pd/pkg/schedule/placement/fit.go:176 +0xd6
2023-12-12T08:34:49.8606348Z   github.com/tikv/pd/pkg/schedule/placement.(*RuleManager).FitRegion()
2023-12-12T08:34:49.8607184Z       /home/runner/work/pd/pd/pkg/schedule/placement/rule_manager.go:405 +0x249
2023-12-12T08:34:49.8607992Z   github.com/tikv/pd/pkg/schedule/filter.newRuleLeaderFitFilter()
2023-12-12T08:34:49.8608848Z       /home/runner/work/pd/pd/pkg/schedule/filter/filters.go:680 +0x1b6
2023-12-12T08:34:49.8609578Z   github.com/tikv/pd/pkg/schedule/filter.NewPlacementLeaderSafeguard()
2023-12-12T08:34:49.8610437Z       /home/runner/work/pd/pd/pkg/schedule/filter/filters.go:785 +0x14e
2023-12-12T08:34:49.8611364Z   github.com/tikv/pd/pkg/schedule/schedulers.(*balanceLeaderScheduler).transferLeaderOut()
2023-12-12T08:34:49.8612368Z       /home/runner/work/pd/pd/pkg/schedule/schedulers/balance_leader.go:476 +0xf6c
2023-12-12T08:34:49.8613377Z   github.com/tikv/pd/pkg/schedule/schedulers.(*balanceLeaderScheduler).transferLeaderOut-fm()
2023-12-12T08:34:49.8614208Z       <autogenerated>:1 +0x47
2023-12-12T08:34:49.8614825Z   github.com/tikv/pd/pkg/schedule/schedulers.createTransferLeaderOperator()
2023-12-12T08:34:49.8615745Z       /home/runner/work/pd/pd/pkg/schedule/schedulers/balance_leader.go:420 +0x7c1
2023-12-12T08:34:49.8617006Z   github.com/tikv/pd/pkg/schedule/schedulers.(*balanceLeaderScheduler).Schedule()
2023-12-12T08:34:49.8617989Z       /home/runner/work/pd/pd/pkg/schedule/schedulers/balance_leader.go:377 +0xcf4
2023-12-12T08:34:49.8618823Z   github.com/tikv/pd/pkg/schedule/schedulers.(*ScheduleController).Schedule()
2023-12-12T08:34:49.8619877Z       /home/runner/work/pd/pd/pkg/schedule/schedulers/scheduler_controller.go:473 +0x2d0
2023-12-12T08:34:49.8620781Z   github.com/tikv/pd/pkg/schedule/schedulers.(*Controller).runScheduler()
2023-12-12T08:34:49.8621787Z       /home/runner/work/pd/pd/pkg/schedule/schedulers/scheduler_controller.go:360 +0x57b
2023-12-12T08:34:49.8622672Z   github.com/tikv/pd/pkg/schedule/schedulers.(*Controller).AddScheduler.func2()
2023-12-12T08:34:49.8623658Z       /home/runner/work/pd/pd/pkg/schedule/schedulers/scheduler_controller.go:209 +0x44
2023-12-12T08:34:49.8624159Z 
2023-12-12T08:34:49.8624460Z Previous write at 0x00c0074a27a0 by goroutine 60238:
2023-12-12T08:34:49.8624981Z   github.com/tikv/pd/pkg/core.MergeLabels()
2023-12-12T08:34:49.8625584Z       /home/runner/work/pd/pd/pkg/core/store.go:620 +0x3cd
2023-12-12T08:34:49.8626288Z   github.com/tikv/pd/server/cluster.(*RaftCluster).putStoreImpl()
2023-12-12T08:34:49.8627027Z       /home/runner/work/pd/pd/server/cluster/cluster.go:1320 +0xb58
2023-12-12T08:34:49.8627689Z   github.com/tikv/pd/server/cluster.(*RaftCluster).PutStore()
2023-12-12T08:34:49.8628458Z       /home/runner/work/pd/pd/server/cluster/cluster.go:1278 +0x7b
2023-12-12T08:34:49.8629086Z   github.com/tikv/pd/server.(*GrpcServer).PutStore()
2023-12-12T08:34:49.8629718Z       /home/runner/work/pd/pd/server/grpc_service.go:748 +0x89b
2023-12-12T08:34:49.8630358Z   github.com/tikv/pd/tests.MustPutStore()
2023-12-12T08:34:49.8630945Z       /home/runner/work/pd/pd/tests/testutil.go:172 +0x4f2
2023-12-12T08:34:49.8631648Z   github.com/tikv/pd/tests/pdctl/hot_test.(*hotTestSuite).checkHotWithoutHotPeer()
2023-12-12T08:34:49.8632575Z       /home/runner/work/pd/pd/tests/pdctl/hot/hot_test.go:335 +0x4d6
2023-12-12T08:34:49.8633464Z   github.com/tikv/pd/tests/pdctl/hot_test.(*hotTestSuite).checkHotWithoutHotPeer-fm()
2023-12-12T08:34:49.8634106Z       <autogenerated>:1 +0x3d
2023-12-12T08:34:49.8634768Z   github.com/tikv/pd/tests.(*SchedulingTestEnvironment).RunTestInPDMode()
2023-12-12T08:34:49.8635505Z       /home/runner/work/pd/pd/tests/testutil.go:279 +0x177
2023-12-12T08:34:49.8636371Z   github.com/tikv/pd/tests.(*SchedulingTestEnvironment).RunTestInTwoModes()
2023-12-12T08:34:49.8637235Z       /home/runner/work/pd/pd/tests/testutil.go:269 +0x79
2023-12-12T08:34:49.8637965Z   github.com/tikv/pd/tests/pdctl/hot_test.(*hotTestSuite).TestHotWithoutHotPeer()
2023-12-12T08:34:49.8638827Z       /home/runner/work/pd/pd/tests/pdctl/hot/hot_test.go:310 +0x59
2023-12-12T08:34:49.8639400Z   runtime.call16()
2023-12-12T08:34:49.8639977Z       /opt/hostedtoolcache/go/1.21.4/x64/src/runtime/asm_amd64.s:747 +0x42
2023-12-12T08:34:49.8640559Z   reflect.Value.Call()
2023-12-12T08:34:49.8641184Z       /opt/hostedtoolcache/go/1.21.4/x64/src/reflect/value.go:380 +0xb5
2023-12-12T08:34:49.8641795Z   github.com/stretchr/testify/suite.Run.func1()
2023-12-12T08:34:49.8642605Z       /home/runner/go/pkg/mod/github.com/stretchr/testify@v1.8.3/suite/suite.go:197 +0x684
2023-12-12T08:34:49.8643272Z   testing.tRunner()
2023-12-12T08:34:49.8643865Z       /opt/hostedtoolcache/go/1.21.4/x64/src/testing/testing.go:1595 +0x238
2023-12-12T08:34:49.8644459Z   testing.(*T).Run.func1()
2023-12-12T08:34:49.8645085Z       /opt/hostedtoolcache/go/1.21.4/x64/src/testing/testing.go:1648 +0x44

CI link

https://github.com/lhy1024/pd/actions/runs/7178820114/job/19547746473?pr=32

Reason for failure (if possible)

Anything else

[lhy1024]

Do we need a clone to avoid putStore affecting scheduling?

[rleungx]

Is it related to #6083?

[rleungx]

https://github.com/tikv/pd/actions/runs/7956490337/job/21717313724?pr=7819