Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

waitForLeader in redirector panic #8406

Closed
HuSharp opened this issue Jul 17, 2024 · 0 comments · Fixed by #8407
Closed

waitForLeader in redirector panic #8406

HuSharp opened this issue Jul 17, 2024 · 0 comments · Fixed by #8407
Labels
impact/panic severity/minor type/bug The issue is confirmed as a bug.

Comments

@HuSharp
Copy link
Member

HuSharp commented Jul 17, 2024

Bug Report

waitForLeader in redirector

pd/pkg/utils/apiutil/serverapi/middleware.go

Line 273 in 96bb8b0

case <-h.s.Context().Done():

detail info https://github.com/tikv/pd/actions/runs/9952768096/job/27494776623

2024-07-16T07:46:44.8014579Z [negroni] PANIC: runtime error: invalid memory address or nil pointer dereference
2024-07-16T07:46:44.8015460Z goroutine 23391 [running]:
2024-07-16T07:46:44.8016075Z github.com/urfave/negroni.(*Recovery).ServeHTTP.func1()
2024-07-16T07:46:44.8017185Z 	/home/runner/go/pkg/mod/github.com/urfave/negroni@v0.3.0/recovery.go:159 +0x14f
2024-07-16T07:46:44.8018105Z panic({0x43a7720?, 0x6a32d80?})
2024-07-16T07:46:44.8018892Z 	/opt/hostedtoolcache/go/1.21.12/x64/src/runtime/panic.go:920 +0x270
2024-07-16T07:46:44.8020271Z github.com/tikv/pd/pkg/utils/apiutil/serverapi.(*redirector).waitForLeader(0xc0013f54e0, 0xc0012f4100)
2024-07-16T07:46:44.8021852Z 	/home/runner/work/pd/pd/pkg/utils/apiutil/serverapi/middleware.go:273 +0x296
2024-07-16T07:46:44.8023741Z github.com/tikv/pd/pkg/utils/apiutil/serverapi.(*redirector).ServeHTTP(0xc0013f54e0, {0x7efd40d085b8, 0xc000043fb0}, 0xc0012f4100, 0xc003ef4400)
2024-07-16T07:46:44.8025626Z 	/home/runner/work/pd/pd/pkg/utils/apiutil/serverapi/middleware.go:210 +0xbcf
2024-07-16T07:46:44.8027371Z github.com/urfave/negroni.middleware.ServeHTTP({{0x554e2c0?, 0xc0013f54e0?}, 0xc002c20510?}, {0x7efd40d085b8, 0xc000043fb0}, 0x11425ac?)
2024-07-16T07:46:44.8029103Z 	/home/runner/go/pkg/mod/github.com/urfave/negroni@v0.3.0/negroni.go:38 +0x135
2024-07-16T07:46:44.8031074Z github.com/tikv/pd/pkg/utils/apiutil/serverapi.(*runtimeServiceValidator).ServeHTTP(0xc0045f3180, {0x7efd40d085b8, 0xc000043fb0}, 0xc000502400?, 0xc003ef43e0)
2024-07-16T07:46:44.8033085Z 	/home/runner/work/pd/pd/pkg/utils/apiutil/serverapi/middleware.go:46 +0x15f
2024-07-16T07:46:44.8034821Z github.com/urfave/negroni.middleware.ServeHTTP({{0x554e4e0?, 0xc0045f3180?}, 0xc002c204f8?}, {0x7efd40d085b8, 0xc000043fb0}, 0x4289e20?)
2024-07-16T07:46:44.8036502Z 	/home/runner/go/pkg/mod/github.com/urfave/negroni@v0.3.0/negroni.go:38 +0x135
2024-07-16T07:46:44.8037890Z github.com/urfave/negroni.(*Negroni).ServeHTTP(0xc0010d42a0, {0x7efd40d085b8?, 0xc000043fa8}, 0x0?)
2024-07-16T07:46:44.8039496Z 	/home/runner/go/pkg/mod/github.com/urfave/negroni@v0.3.0/negroni.go:96 +0x1b9
2024-07-16T07:46:44.8040887Z github.com/gorilla/mux.(*Router).ServeHTTP(0xc001bbd140, {0x7efd40d085b8, 0xc000043fa8}, 0xc002d57f00)
2024-07-16T07:46:44.8042260Z 	/home/runner/go/pkg/mod/github.com/gorilla/mux@v1.7.4/mux.go:210 +0x362
2024-07-16T07:46:44.8043614Z github.com/gorilla/mux.(*Router).ServeHTTP(0xc0001fd080, {0x7efd40d085b8, 0xc000043fa8}, 0xc002d57d00)
2024-07-16T07:46:44.8044997Z 	/home/runner/go/pkg/mod/github.com/gorilla/mux@v1.7.4/mux.go:210 +0x362
2024-07-16T07:46:44.8046499Z github.com/urfave/negroni.(*Negroni).UseHandler.Wrap.func1({0x7efd40d085b8, 0xc000043fa8}, 0x10d8185?, 0xc003ef43c0)
2024-07-16T07:46:44.8048266Z 	/home/runner/go/pkg/mod/github.com/urfave/negroni@v0.3.0/negroni.go:46 +0x67
2024-07-16T07:46:44.8049924Z github.com/urfave/negroni.HandlerFunc.ServeHTTP(0xc002c20600, {0x7efd40d085b8, 0xc000043fa8}, 0x197ba72?, 0x0?)
2024-07-16T07:46:44.8051458Z 	/home/runner/go/pkg/mod/github.com/urfave/negroni@v0.3.0/negroni.go:29 +0x52
2024-07-16T07:46:44.8053070Z github.com/urfave/negroni.middleware.ServeHTTP({{0x554e2e0?, 0xc002c20600?}, 0xc002c20630?}, {0x7efd40d085b8, 0xc000043fa8}, 0x4?)
2024-07-16T07:46:44.8054701Z 	/home/runner/go/pkg/mod/github.com/urfave/negroni@v0.3.0/negroni.go:38 +0x135
2024-07-16T07:46:44.8056271Z github.com/urfave/negroni.(*Recovery).ServeHTTP(0xc0012c3cc0, {0x7efd40d085b8?, 0xc000043fa8}, 0xc002d57d00, 0xc003ef43a0)
2024-07-16T07:46:44.8057867Z 	/home/runner/go/pkg/mod/github.com/urfave/negroni@v0.3.0/recovery.go:193 +0x114
2024-07-16T07:46:44.8059573Z github.com/urfave/negroni.middleware.ServeHTTP({{0x554c4a0?, 0xc0012c3cc0?}, 0xc002c20618?}, {0x7efd40d085b8, 0xc000043fa8}, 0xc0044b5170?)
2024-07-16T07:46:44.8061260Z 	/home/runner/go/pkg/mod/github.com/urfave/negroni@v0.3.0/negroni.go:38 +0x135
2024-07-16T07:46:44.8062679Z github.com/urfave/negroni.(*Negroni).ServeHTTP(0xc003fc2b40, {0x555d0c0?, 0xc00130aee0}, 0xc0044b5170?)
2024-07-16T07:46:44.8064113Z 	/home/runner/go/pkg/mod/github.com/urfave/negroni@v0.3.0/negroni.go:96 +0x1b9
2024-07-16T07:46:44.8065354Z net/http.(*ServeMux).ServeHTTP(0x555d0c0?, {0x555d0c0, 0xc00130aee0}, 0xc002d57d00)
2024-07-16T07:46:44.8066534Z 	/opt/hostedtoolcache/go/1.21.12/x64/src/net/http/server.go:2519 +0xbd
2024-07-16T07:46:44.8067885Z go.etcd.io/etcd/embed.(*accessController).ServeHTTP(0xc0058d0d20, {0x555d0c0, 0xc00130aee0}, 0xc002d57d00)
2024-07-16T07:46:44.8069823Z 	/home/runner/go/pkg/mod/go.etcd.io/etcd@v0.5.0-alpha.5.0.20240320135013-950cd5fbe6ca/embed/serve.go:475 +0x98e
2024-07-16T07:46:44.8071399Z net/http.serverHandler.ServeHTTP({0xc0044b50b0?}, {0x555d0c0, 0xc00130aee0}, 0xc002d57d00)
2024-07-16T07:46:44.8072672Z 	/opt/hostedtoolcache/go/1.21.12/x64/src/net/http/server.go:2943 +0x2a2
2024-07-16T07:46:44.8073668Z net/http.(*conn).serve(0xc000053170, {0x556a300, 0xc002016330})
2024-07-16T07:46:44.8074689Z 	/opt/hostedtoolcache/go/1.21.12/x64/src/net/http/server.go:2014 +0xc25
2024-07-16T07:46:44.8075637Z created by net/http.(*Server).Serve in goroutine 20645
2024-07-16T07:46:44.8076621Z 	/opt/hostedtoolcache/go/1.21.12/x64/src/net/http/server.go:3091 +0x86d

we can check data race info

2024-07-16T07:46:44.7720638Z ==================
2024-07-16T07:46:44.7720879Z WARNING: DATA RACE
2024-07-16T07:46:44.7721166Z Write at 0x00c003914060 by goroutine 4736:
2024-07-16T07:46:44.7721619Z   github.com/tikv/pd/server.(*Server).startServerLoop()
2024-07-16T07:46:44.7722178Z       /home/runner/work/pd/pd/server/server.go:661 +0xaf
2024-07-16T07:46:44.7722635Z   github.com/tikv/pd/server.(*Server).Run()
2024-07-16T07:46:44.7723142Z       /home/runner/work/pd/pd/server/server.go:635 +0x326
2024-07-16T07:46:44.7723705Z   github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
2024-07-16T07:46:44.7724904Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/storage/file_storage.go:458 +0x3fd
2024-07-16T07:46:44.7725651Z   fmt.Fscanf()
2024-07-16T07:46:44.7726100Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:143 +0xdc
2024-07-16T07:46:44.7726528Z   fmt.Sscanf()
2024-07-16T07:46:44.7726980Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:114 +0x264
2024-07-16T07:46:44.7727544Z   github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
2024-07-16T07:46:44.7728694Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/storage/file_storage.go:657 +0x18e
2024-07-16T07:46:44.7729437Z   fmt.(*ss).doScanf()
2024-07-16T07:46:44.7730228Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:1230 +0x40e
2024-07-16T07:46:44.7730664Z   fmt.Fscanf()
2024-07-16T07:46:44.7731100Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:143 +0xdc
2024-07-16T07:46:44.7731520Z   fmt.Sscanf()
2024-07-16T07:46:44.7731955Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:114 +0x184
2024-07-16T07:46:44.7732514Z   github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
2024-07-16T07:46:44.7733637Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/storage/file_storage.go:643 +0x1a
2024-07-16T07:46:44.7734548Z   github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
2024-07-16T07:46:44.7735690Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/storage/file_storage.go:458 +0x3fd
2024-07-16T07:46:44.7736583Z   github.com/syndtr/goleveldb/leveldb.(*DB).checkAndCleanFiles()
2024-07-16T07:46:44.7737637Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/db_util.go:52 +0x288
2024-07-16T07:46:44.7738394Z   github.com/syndtr/goleveldb/leveldb.openDB()
2024-07-16T07:46:44.7739340Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/db.go:130 +0x884
2024-07-16T07:46:44.7740166Z   github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
2024-07-16T07:46:44.7741314Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/storage/file_storage.go:458 +0x3fd
2024-07-16T07:46:44.7742080Z   fmt.Fscanf()
2024-07-16T07:46:44.7742836Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:143 +0xdc
2024-07-16T07:46:44.7743546Z   fmt.Sscanf()
2024-07-16T07:46:44.7744302Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:114 +0x184
2024-07-16T07:46:44.7745269Z   github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
2024-07-16T07:46:44.7747208Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/storage/file_storage.go:643 +0x1a
2024-07-16T07:46:44.7748931Z   github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
2024-07-16T07:46:44.7751146Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/storage/file_storage.go:458 +0x3fd
2024-07-16T07:46:44.7752516Z   fmt.Fscanf()
2024-07-16T07:46:44.7753307Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:143 +0xdc
2024-07-16T07:46:44.7754059Z   fmt.Sscanf()
2024-07-16T07:46:44.7754847Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:114 +0x184
2024-07-16T07:46:44.7755843Z   github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
2024-07-16T07:46:44.7757934Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/storage/file_storage.go:643 +0x1a
2024-07-16T07:46:44.7759792Z   github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
2024-07-16T07:46:44.7762008Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/storage/file_storage.go:458 +0x3fd
2024-07-16T07:46:44.7763390Z   fmt.Fscanf()
2024-07-16T07:46:44.7764185Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:143 +0xdc
2024-07-16T07:46:44.7764924Z   fmt.Sscanf()
2024-07-16T07:46:44.7765710Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:114 +0x264
2024-07-16T07:46:44.7766708Z   github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
2024-07-16T07:46:44.7768786Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/storage/file_storage.go:657 +0x18e
2024-07-16T07:46:44.7770160Z   fmt.(*ss).doScanf()
2024-07-16T07:46:44.7771005Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:1230 +0x40e
2024-07-16T07:46:44.7771770Z   fmt.Fscanf()
2024-07-16T07:46:44.7772561Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:143 +0xdc
2024-07-16T07:46:44.7773730Z   fmt.Sscanf()
2024-07-16T07:46:44.7774543Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:114 +0x184
2024-07-16T07:46:44.7775543Z   github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
2024-07-16T07:46:44.7777652Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/storage/file_storage.go:643 +0x1a
2024-07-16T07:46:44.7779338Z   github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
2024-07-16T07:46:44.7781506Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/storage/file_storage.go:458 +0x3fd
2024-07-16T07:46:44.7783114Z   github.com/syndtr/goleveldb/leveldb.(*DB).recoverJournal()
2024-07-16T07:46:44.7784983Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/db.go:476 +0xa6
2024-07-16T07:46:44.7786329Z   github.com/syndtr/goleveldb/leveldb.openDB()
2024-07-16T07:46:44.7788106Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/db.go:125 +0x86d
2024-07-16T07:46:44.7789441Z   github.com/syndtr/goleveldb/leveldb.Open()
2024-07-16T07:46:44.7791204Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/db.go:197 +0x364
2024-07-16T07:46:44.7792397Z   fmt.Fscanf()
2024-07-16T07:46:44.7793193Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:143 +0xdc
2024-07-16T07:46:44.7793945Z   fmt.Sscanf()
2024-07-16T07:46:44.7794737Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:114 +0x184
2024-07-16T07:46:44.7795719Z   github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
2024-07-16T07:46:44.7797713Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/storage/file_storage.go:643 +0x1a
2024-07-16T07:46:44.7799588Z   github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
2024-07-16T07:46:44.7800942Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/storage/file_storage.go:458 +0x3fd
2024-07-16T07:46:44.7801941Z   github.com/syndtr/goleveldb/leveldb.(*session).recover.func1()
2024-07-16T07:46:44.7803013Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/session.go:134 +0xbd
2024-07-16T07:46:44.7803714Z   runtime.deferreturn()
2024-07-16T07:46:44.7804232Z       /opt/hostedtoolcache/go/1.21.12/x64/src/runtime/panic.go:477 +0x30
2024-07-16T07:46:44.7804752Z   github.com/syndtr/goleveldb/leveldb.Open()
2024-07-16T07:46:44.7805698Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/db.go:183 +0x124
2024-07-16T07:46:44.7806432Z   github.com/syndtr/goleveldb/leveldb.OpenFile()
2024-07-16T07:46:44.7807379Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/db.go:219 +0x84
2024-07-16T07:46:44.7808124Z   github.com/tikv/pd/pkg/storage/kv.NewLevelDBKV()
2024-07-16T07:46:44.7808729Z       /home/runner/work/pd/pd/pkg/storage/kv/levedb_kv.go:34 +0x92
2024-07-16T07:46:44.7809284Z   github.com/tikv/pd/pkg/storage.NewHotRegionsStorage()
2024-07-16T07:46:44.7809942Z       /home/runner/work/pd/pd/pkg/storage/hot_region_storage.go:123 +0xc4
2024-07-16T07:46:44.7810502Z   github.com/tikv/pd/server.(*Server).startServer()
2024-07-16T07:46:44.7811033Z       /home/runner/work/pd/pd/server/server.go:513 +0x34c7
2024-07-16T07:46:44.7811597Z   github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
2024-07-16T07:46:44.7812773Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/storage/file_storage.go:458 +0x3fd
2024-07-16T07:46:44.7813514Z   fmt.Fscanf()
2024-07-16T07:46:44.7813950Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:143 +0xdc
2024-07-16T07:46:44.7814372Z   fmt.Sscanf()
2024-07-16T07:46:44.7814812Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:114 +0x264
2024-07-16T07:46:44.7815581Z   github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
2024-07-16T07:46:44.7816838Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/storage/file_storage.go:657 +0x18e
2024-07-16T07:46:44.7817585Z   fmt.(*ss).doScanf()
2024-07-16T07:46:44.7818063Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:1230 +0x40e
2024-07-16T07:46:44.7818502Z   fmt.Fscanf()
2024-07-16T07:46:44.7818937Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:143 +0xdc
2024-07-16T07:46:44.7819354Z   fmt.Sscanf()
2024-07-16T07:46:44.7819796Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:114 +0x184
2024-07-16T07:46:44.7820349Z   github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
2024-07-16T07:46:44.7821463Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/storage/file_storage.go:643 +0x1a
2024-07-16T07:46:44.7822395Z   github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
2024-07-16T07:46:44.7823550Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/storage/file_storage.go:458 +0x3fd
2024-07-16T07:46:44.7824438Z   github.com/syndtr/goleveldb/leveldb.(*DB).checkAndCleanFiles()
2024-07-16T07:46:44.7825487Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/db_util.go:52 +0x288
2024-07-16T07:46:44.7826220Z   github.com/syndtr/goleveldb/leveldb.openDB()
2024-07-16T07:46:44.7827159Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/db.go:130 +0x884
2024-07-16T07:46:44.7827981Z   github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
2024-07-16T07:46:44.7829171Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/storage/file_storage.go:458 +0x3fd
2024-07-16T07:46:44.7829909Z   fmt.Fscanf()
2024-07-16T07:46:44.7830353Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:143 +0xdc
2024-07-16T07:46:44.7830772Z   fmt.Sscanf()
2024-07-16T07:46:44.7831212Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:114 +0x184
2024-07-16T07:46:44.7831771Z   github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
2024-07-16T07:46:44.7832881Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/storage/file_storage.go:643 +0x1a
2024-07-16T07:46:44.7833783Z   github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
2024-07-16T07:46:44.7834924Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/storage/file_storage.go:458 +0x3fd
2024-07-16T07:46:44.7835656Z   fmt.Fscanf()
2024-07-16T07:46:44.7836094Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:143 +0xdc
2024-07-16T07:46:44.7836515Z   fmt.Sscanf()
2024-07-16T07:46:44.7836949Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:114 +0x184
2024-07-16T07:46:44.7837502Z   github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
2024-07-16T07:46:44.7838892Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/storage/file_storage.go:643 +0x1a
2024-07-16T07:46:44.7839939Z   github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
2024-07-16T07:46:44.7841116Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/storage/file_storage.go:458 +0x3fd
2024-07-16T07:46:44.7841863Z   fmt.Fscanf()
2024-07-16T07:46:44.7842304Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:143 +0xdc
2024-07-16T07:46:44.7842722Z   fmt.Sscanf()
2024-07-16T07:46:44.7843152Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:114 +0x264
2024-07-16T07:46:44.7843705Z   github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
2024-07-16T07:46:44.7844838Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/storage/file_storage.go:657 +0x18e
2024-07-16T07:46:44.7845574Z   fmt.(*ss).doScanf()
2024-07-16T07:46:44.7846343Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:1230 +0x40e
2024-07-16T07:46:44.7846773Z   fmt.Fscanf()
2024-07-16T07:46:44.7847204Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:143 +0xdc
2024-07-16T07:46:44.7847618Z   fmt.Sscanf()
2024-07-16T07:46:44.7848051Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:114 +0x184
2024-07-16T07:46:44.7848613Z   github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
2024-07-16T07:46:44.7849718Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/storage/file_storage.go:643 +0x1a
2024-07-16T07:46:44.7850622Z   github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
2024-07-16T07:46:44.7852569Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/storage/file_storage.go:458 +0x3fd
2024-07-16T07:46:44.7854129Z   github.com/syndtr/goleveldb/leveldb.(*DB).recoverJournal()
2024-07-16T07:46:44.7855840Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/db.go:476 +0xa6
2024-07-16T07:46:44.7856597Z   github.com/syndtr/goleveldb/leveldb.openDB()
2024-07-16T07:46:44.7857624Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/db.go:125 +0x86d
2024-07-16T07:46:44.7858362Z   github.com/syndtr/goleveldb/leveldb.Open()
2024-07-16T07:46:44.7859308Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/db.go:197 +0x364
2024-07-16T07:46:44.7859956Z   fmt.Fscanf()
2024-07-16T07:46:44.7860404Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:143 +0xdc
2024-07-16T07:46:44.7860832Z   fmt.Sscanf()
2024-07-16T07:46:44.7861277Z       /opt/hostedtoolcache/go/1.21.12/x64/src/fmt/scan.go:114 +0x184
2024-07-16T07:46:44.7861844Z   github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
2024-07-16T07:46:44.7862961Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/storage/file_storage.go:643 +0x1a
2024-07-16T07:46:44.7863873Z   github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
2024-07-16T07:46:44.7865033Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/storage/file_storage.go:458 +0x3fd
2024-07-16T07:46:44.7865932Z   github.com/syndtr/goleveldb/leveldb.(*session).recover.func1()
2024-07-16T07:46:44.7866976Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/session.go:134 +0xbd
2024-07-16T07:46:44.7867668Z   runtime.deferreturn()
2024-07-16T07:46:44.7868178Z       /opt/hostedtoolcache/go/1.21.12/x64/src/runtime/panic.go:477 +0x30
2024-07-16T07:46:44.7868693Z   github.com/syndtr/goleveldb/leveldb.Open()
2024-07-16T07:46:44.7869635Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/db.go:183 +0x124
2024-07-16T07:46:44.7870371Z   github.com/syndtr/goleveldb/leveldb.OpenFile()
2024-07-16T07:46:44.7871315Z       /home/runner/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20190318030020-c3a204f8e965/leveldb/db.go:219 +0x84
2024-07-16T07:46:44.7872051Z   github.com/tikv/pd/pkg/storage/kv.NewLevelDBKV()
2024-07-16T07:46:44.7872641Z       /home/runner/work/pd/pd/pkg/storage/kv/levedb_kv.go:34 +0x92
2024-07-16T07:46:44.7873182Z   github.com/tikv/pd/pkg/storage.newLevelDBBackend()
2024-07-16T07:46:44.7873804Z       /home/runner/work/pd/pd/pkg/storage/leveldb_backend.go:62 +0xad
2024-07-16T07:46:44.7874454Z   github.com/tikv/pd/pkg/storage.NewRegionStorageWithLevelDBBackend()
2024-07-16T07:46:44.7875116Z       /home/runner/work/pd/pd/pkg/storage/storage.go:67 +0xad
2024-07-16T07:46:44.7875599Z   github.com/tikv/pd/server.(*Server).startServer()
2024-07-16T07:46:44.7876130Z       /home/runner/work/pd/pd/server/server.go:468 +0x1398
2024-07-16T07:46:44.7876571Z   github.com/tikv/pd/server.(*Server).Run()
2024-07-16T07:46:44.7877236Z       /home/runner/work/pd/pd/server/server.go:626 +0x1e6
2024-07-16T07:46:44.7877818Z   github.com/tikv/pd/tests.(*TestServer).Run()
2024-07-16T07:46:44.7878541Z       /home/runner/work/pd/pd/tests/cluster.go:127 +0x1fd
2024-07-16T07:46:44.7879514Z   github.com/tikv/pd/tests/integrations/mcs/scheduling.(*serverTestSuite).TestAllocIDAfterLeaderChange()
2024-07-16T07:46:44.7880561Z       /home/runner/work/pd/pd/tests/integrations/mcs/scheduling/server_test.go:107 +0x135
2024-07-16T07:46:44.7881196Z   github.com/pingcap/failpoint.parseTerm()
2024-07-16T07:46:44.7882127Z       /home/runner/go/pkg/mod/github.com/pingcap/failpoint@v0.0.0-20220801062533-2eaa32854a6c/terms.go:149 +0x364
2024-07-16T07:46:44.7882808Z   github.com/pingcap/failpoint.parse()
2024-07-16T07:46:44.7883692Z       /home/runner/go/pkg/mod/github.com/pingcap/failpoint@v0.0.0-20220801062533-2eaa32854a6c/terms.go:126 +0xa5
2024-07-16T07:46:44.7884382Z   github.com/pingcap/failpoint.newTerms()
2024-07-16T07:46:44.7885273Z       /home/runner/go/pkg/mod/github.com/pingcap/failpoint@v0.0.0-20220801062533-2eaa32854a6c/terms.go:98 +0x3e
2024-07-16T07:46:44.7886006Z   github.com/pingcap/failpoint.(*Failpoint).Enable()
2024-07-16T07:46:44.7886957Z       /home/runner/go/pkg/mod/github.com/pingcap/failpoint@v0.0.0-20220801062533-2eaa32854a6c/failpoint.go:54 +0x3e
2024-07-16T07:46:44.7887700Z   github.com/pingcap/failpoint.(*Failpoints).Enable()
2024-07-16T07:46:44.7888683Z       /home/runner/go/pkg/mod/github.com/pingcap/failpoint@v0.0.0-20220801062533-2eaa32854a6c/failpoints.go:105 +0x296
2024-07-16T07:46:44.7889394Z   github.com/pingcap/failpoint.Enable()
2024-07-16T07:46:44.7890327Z       /home/runner/go/pkg/mod/github.com/pingcap/failpoint@v0.0.0-20220801062533-2eaa32854a6c/failpoints.go:222 +0x9c
2024-07-16T07:46:44.7891385Z   github.com/tikv/pd/tests/integrations/mcs/scheduling.(*serverTestSuite).TestAllocIDAfterLeaderChange()
2024-07-16T07:46:44.7892409Z       /home/runner/work/pd/pd/tests/integrations/mcs/scheduling/server_test.go:104 +0x9d
2024-07-16T07:46:44.7892988Z   runtime.call16()
2024-07-16T07:46:44.7893479Z       /opt/hostedtoolcache/go/1.21.12/x64/src/runtime/asm_amd64.s:747 +0x42
2024-07-16T07:46:44.7893954Z   reflect.Value.Call()
2024-07-16T07:46:44.7894453Z       /opt/hostedtoolcache/go/1.21.12/x64/src/reflect/value.go:380 +0xb5
2024-07-16T07:46:44.7894983Z   github.com/stretchr/testify/suite.Run.func1()
2024-07-16T07:46:44.7895681Z       /home/runner/go/pkg/mod/github.com/stretchr/testify@v1.8.4/suite/suite.go:197 +0x766
2024-07-16T07:46:44.7896231Z   testing.tRunner()
2024-07-16T07:46:44.7896744Z       /opt/hostedtoolcache/go/1.21.12/x64/src/testing/testing.go:1595 +0x261
2024-07-16T07:46:44.7897225Z   testing.(*T).Run.func1()
2024-07-16T07:46:44.7897741Z       /opt/hostedtoolcache/go/1.21.12/x64/src/testing/testing.go:1648 +0x44
2024-07-16T07:46:44.7898138Z 
2024-07-16T07:46:44.7898307Z Previous read at 0x00c003914060 by goroutine 23377:
2024-07-16T07:46:44.7898776Z   github.com/tikv/pd/server.(*Server).LoopContext()
2024-07-16T07:46:44.7899310Z       /home/runner/work/pd/pd/server/server.go:657 +0x28c
2024-07-16T07:46:44.7899950Z   github.com/tikv/pd/pkg/utils/apiutil/serverapi.(*redirector).waitForLeader()
2024-07-16T07:46:44.7900799Z       /home/runner/work/pd/pd/pkg/utils/apiutil/serverapi/middleware.go:273 +0x20c
2024-07-16T07:46:44.7901558Z   github.com/tikv/pd/pkg/utils/apiutil/serverapi.(*redirector).ServeHTTP()
2024-07-16T07:46:44.7902376Z       /home/runner/work/pd/pd/pkg/utils/apiutil/serverapi/middleware.go:210 +0xbce
2024-07-16T07:46:44.7903011Z   github.com/urfave/negroni.middleware.ServeHTTP()
2024-07-16T07:46:44.7903694Z       /home/runner/go/pkg/mod/github.com/urfave/negroni@v0.3.0/negroni.go:38 +0x134
2024-07-16T07:46:44.7904363Z   github.com/urfave/negroni.middleware.ServeHTTP-fm()
2024-07-16T07:46:44.7904796Z       <autogenerated>:1 +0x71
2024-07-16T07:46:44.7905374Z   github.com/tikv/pd/pkg/utils/apiutil/serverapi.(*runtimeServiceValidator).ServeHTTP()
2024-07-16T07:46:44.7906270Z       /home/runner/work/pd/pd/pkg/utils/apiutil/serverapi/middleware.go:46 +0x15e
2024-07-16T07:46:44.7907370Z   github.com/urfave/negroni.middleware.ServeHTTP()
2024-07-16T07:46:44.7908078Z       /home/runner/go/pkg/mod/github.com/urfave/negroni@v0.3.0/negroni.go:38 +0x134
2024-07-16T07:46:44.7908673Z   github.com/urfave/negroni.(*Negroni).ServeHTTP()
2024-07-16T07:46:44.7909341Z       /home/runner/go/pkg/mod/github.com/urfave/negroni@v0.3.0/negroni.go:96 +0x1b8
2024-07-16T07:46:44.7909903Z   github.com/gorilla/mux.(*Router).ServeHTTP()
2024-07-16T07:46:44.7910533Z       /home/runner/go/pkg/mod/github.com/gorilla/mux@v1.7.4/mux.go:210 +0x361
2024-07-16T07:46:44.7911080Z   github.com/gorilla/mux.(*Router).ServeHTTP()
2024-07-16T07:46:44.7911701Z       /home/runner/go/pkg/mod/github.com/gorilla/mux@v1.7.4/mux.go:210 +0x361
2024-07-16T07:46:44.7912322Z   github.com/urfave/negroni.(*Negroni).UseHandler.Wrap.func1()
2024-07-16T07:46:44.7913047Z       /home/runner/go/pkg/mod/github.com/urfave/negroni@v0.3.0/negroni.go:46 +0x66
2024-07-16T07:46:44.7913644Z   github.com/urfave/negroni.HandlerFunc.ServeHTTP()
2024-07-16T07:46:44.7914326Z       /home/runner/go/pkg/mod/github.com/urfave/negroni@v0.3.0/negroni.go:29 +0x51
2024-07-16T07:46:44.7914918Z   github.com/urfave/negroni.middleware.ServeHTTP()
2024-07-16T07:46:44.7915595Z       /home/runner/go/pkg/mod/github.com/urfave/negroni@v0.3.0/negroni.go:38 +0x134
2024-07-16T07:46:44.7916263Z   github.com/urfave/negroni.middleware.ServeHTTP-fm()
2024-07-16T07:46:44.7916686Z       <autogenerated>:1 +0x71
2024-07-16T07:46:44.7917059Z   github.com/urfave/negroni.(*Recovery).ServeHTTP()
2024-07-16T07:46:44.7917756Z       /home/runner/go/pkg/mod/github.com/urfave/negroni@v0.3.0/recovery.go:193 +0x113
2024-07-16T07:46:44.7918596Z   github.com/urfave/negroni.middleware.ServeHTTP()
2024-07-16T07:46:44.7919284Z       /home/runner/go/pkg/mod/github.com/urfave/negroni@v0.3.0/negroni.go:38 +0x134
2024-07-16T07:46:44.7919869Z   github.com/urfave/negroni.(*Negroni).ServeHTTP()
2024-07-16T07:46:44.7920534Z       /home/runner/go/pkg/mod/github.com/urfave/negroni@v0.3.0/negroni.go:96 +0x1b8
2024-07-16T07:46:44.7921076Z   net/http.(*ServeMux).ServeHTTP()
2024-07-16T07:46:44.7921636Z       /opt/hostedtoolcache/go/1.21.12/x64/src/net/http/server.go:2519 +0xbc
2024-07-16T07:46:44.7922210Z   go.etcd.io/etcd/embed.(*accessController).ServeHTTP()
2024-07-16T07:46:44.7923194Z       /home/runner/go/pkg/mod/go.etcd.io/etcd@v0.5.0-alpha.5.0.20240320135013-950cd5fbe6ca/embed/serve.go:475 +0x98d
2024-07-16T07:46:44.7923891Z   net/http.serverHandler.ServeHTTP()
2024-07-16T07:46:44.7924480Z       /opt/hostedtoolcache/go/1.21.12/x64/src/net/http/server.go:2943 +0x2a1
2024-07-16T07:46:44.7924966Z   net/http.(*conn).serve()
2024-07-16T07:46:44.7925493Z       /opt/hostedtoolcache/go/1.21.12/x64/src/net/http/server.go:2014 +0xc24
2024-07-16T07:46:44.7925984Z   net/http.(*Server).Serve.func3()
2024-07-16T07:46:44.7926538Z       /opt/hostedtoolcache/go/1.21.12/x64/src/net/http/server.go:3091 +0x4f
2024-07-16T07:46:44.7927118Z 
2024-07-16T07:46:44.7927315Z Goroutine 4736 (running) created at:
2024-07-16T07:46:44.7927868Z   testing.(*T).Run()
2024-07-16T07:46:44.7928776Z       /opt/hostedtoolcache/go/1.21.12/x64/src/testing/testing.go:1648 +0x845
2024-07-16T07:46:44.7929680Z   github.com/stretchr/testify/suite.runTests()
2024-07-16T07:46:44.7930884Z       /home/runner/go/pkg/mod/github.com/stretchr/testify@v1.8.4/suite/suite.go:242 +0x186
2024-07-16T07:46:44.7931958Z   github.com/stretchr/testify/suite.Run()
2024-07-16T07:46:44.7933212Z       /home/runner/go/pkg/mod/github.com/stretchr/testify@v1.8.4/suite/suite.go:215 +0xa4e
2024-07-16T07:46:44.7934271Z   github.com/pingcap/failpoint.parse()
2024-07-16T07:46:44.7935922Z       /home/runner/go/pkg/mod/github.com/pingcap/failpoint@v0.0.0-20220801062533-2eaa32854a6c/terms.go:126 +0xa5
2024-07-16T07:46:44.7937257Z   github.com/pingcap/failpoint.newTerms()
2024-07-16T07:46:44.7938910Z       /home/runner/go/pkg/mod/github.com/pingcap/failpoint@v0.0.0-20220801062533-2eaa32854a6c/terms.go:98 +0x3e
2024-07-16T07:46:44.7940477Z   github.com/pingcap/failpoint.(*Failpoint).Enable()
2024-07-16T07:46:44.7942497Z       /home/runner/go/pkg/mod/github.com/pingcap/failpoint@v0.0.0-20220801062533-2eaa32854a6c/failpoint.go:54 +0x3e
2024-07-16T07:46:44.7943875Z   github.com/pingcap/failpoint.(*Failpoints).Enable()
2024-07-16T07:46:44.7945688Z       /home/runner/go/pkg/mod/github.com/pingcap/failpoint@v0.0.0-20220801062533-2eaa32854a6c/failpoints.go:105 +0x296
2024-07-16T07:46:44.7946989Z   github.com/pingcap/failpoint.Enable()
2024-07-16T07:46:44.7948699Z       /home/runner/go/pkg/mod/github.com/pingcap/failpoint@v0.0.0-20220801062533-2eaa32854a6c/failpoints.go:222 +0xf0
2024-07-16T07:46:44.7950437Z   github.com/tikv/pd/tests/integrations/mcs/scheduling.(*serverTestSuite).SetupSuite()
2024-07-16T07:46:44.7952156Z       /home/runner/work/pd/pd/tests/integrations/mcs/scheduling/server_test.go:64 +0xf1
2024-07-16T07:46:44.7953290Z   github.com/pingcap/failpoint.parseTerm()
2024-07-16T07:46:44.7954990Z       /home/runner/go/pkg/mod/github.com/pingcap/failpoint@v0.0.0-20220801062533-2eaa32854a6c/terms.go:149 +0x364
2024-07-16T07:46:44.7956249Z   github.com/pingcap/failpoint.parse()
2024-07-16T07:46:44.7957887Z       /home/runner/go/pkg/mod/github.com/pingcap/failpoint@v0.0.0-20220801062533-2eaa32854a6c/terms.go:126 +0xa5
2024-07-16T07:46:44.7959324Z   github.com/pingcap/failpoint.newTerms()
2024-07-16T07:46:44.7960993Z       /home/runner/go/pkg/mod/github.com/pingcap/failpoint@v0.0.0-20220801062533-2eaa32854a6c/terms.go:98 +0x3e
2024-07-16T07:46:44.7962320Z   github.com/pingcap/failpoint.(*Failpoint).Enable()
2024-07-16T07:46:44.7964093Z       /home/runner/go/pkg/mod/github.com/pingcap/failpoint@v0.0.0-20220801062533-2eaa32854a6c/failpoint.go:54 +0x3e
2024-07-16T07:46:44.7965460Z   github.com/pingcap/failpoint.(*Failpoints).Enable()
2024-07-16T07:46:44.7967265Z       /home/runner/go/pkg/mod/github.com/pingcap/failpoint@v0.0.0-20220801062533-2eaa32854a6c/failpoints.go:105 +0x296
2024-07-16T07:46:44.7968571Z   github.com/pingcap/failpoint.Enable()
2024-07-16T07:46:44.7970295Z       /home/runner/go/pkg/mod/github.com/pingcap/failpoint@v0.0.0-20220801062533-2eaa32854a6c/failpoints.go:222 +0xb5
2024-07-16T07:46:44.7972077Z   github.com/tikv/pd/tests/integrations/mcs/scheduling.(*serverTestSuite).SetupSuite()
2024-07-16T07:46:44.7973831Z       /home/runner/work/pd/pd/tests/integrations/mcs/scheduling/server_test.go:63 +0xb6
2024-07-16T07:46:44.7974984Z   github.com/pingcap/failpoint.parseTerm()
2024-07-16T07:46:44.7976558Z       /home/runner/go/pkg/mod/github.com/pingcap/failpoint@v0.0.0-20220801062533-2eaa32854a6c/terms.go:149 +0x364
2024-07-16T07:46:44.7977824Z   github.com/pingcap/failpoint.parse()
2024-07-16T07:46:44.7979488Z       /home/runner/go/pkg/mod/github.com/pingcap/failpoint@v0.0.0-20220801062533-2eaa32854a6c/terms.go:126 +0xa5
2024-07-16T07:46:44.7980746Z   github.com/pingcap/failpoint.newTerms()
2024-07-16T07:46:44.7982383Z       /home/runner/go/pkg/mod/github.com/pingcap/failpoint@v0.0.0-20220801062533-2eaa32854a6c/terms.go:98 +0x3e
2024-07-16T07:46:44.7983704Z   github.com/pingcap/failpoint.(*Failpoint).Enable()
2024-07-16T07:46:44.7985458Z       /home/runner/go/pkg/mod/github.com/pingcap/failpoint@v0.0.0-20220801062533-2eaa32854a6c/failpoint.go:54 +0x3e
2024-07-16T07:46:44.7986804Z   github.com/pingcap/failpoint.(*Failpoints).Enable()
2024-07-16T07:46:44.7988601Z       /home/runner/go/pkg/mod/github.com/pingcap/failpoint@v0.0.0-20220801062533-2eaa32854a6c/failpoints.go:105 +0x296
2024-07-16T07:46:44.7989891Z   github.com/pingcap/failpoint.Enable()
2024-07-16T07:46:44.7991594Z       /home/runner/go/pkg/mod/github.com/pingcap/failpoint@v0.0.0-20220801062533-2eaa32854a6c/failpoints.go:222 +0x7a
2024-07-16T07:46:44.7993331Z   github.com/tikv/pd/tests/integrations/mcs/scheduling.(*serverTestSuite).SetupSuite()
2024-07-16T07:46:44.7995008Z       /home/runner/work/pd/pd/tests/integrations/mcs/scheduling/server_test.go:62 +0x7b
2024-07-16T07:46:44.7996114Z   github.com/stretchr/testify/suite.Run()
2024-07-16T07:46:44.7997577Z       /home/runner/go/pkg/mod/github.com/stretchr/testify@v1.8.4/suite/suite.go:154 +0x63e
2024-07-16T07:46:44.7999391Z   github.com/tikv/pd/tests/integrations/mcs/scheduling.TestServerTestSuite()
2024-07-16T07:46:44.8001025Z       /home/runner/work/pd/pd/tests/integrations/mcs/scheduling/server_test.go:56 +0x3c
2024-07-16T07:46:44.8002108Z   testing.tRunner()
2024-07-16T07:46:44.8003023Z       /opt/hostedtoolcache/go/1.21.12/x64/src/testing/testing.go:1595 +0x261
2024-07-16T07:46:44.8003883Z   testing.(*T).Run.func1()
2024-07-16T07:46:44.8004825Z       /opt/hostedtoolcache/go/1.21.12/x64/src/testing/testing.go:1648 +0x44
2024-07-16T07:46:44.8005496Z 
2024-07-16T07:46:44.8005715Z Goroutine 23377 (running) created at:
2024-07-16T07:46:44.8006308Z   net/http.(*Server).Serve()
2024-07-16T07:46:44.8007264Z       /opt/hostedtoolcache/go/1.21.12/x64/src/net/http/server.go:3091 +0x86c
2024-07-16T07:46:44.8008181Z   go.etcd.io/etcd/embed.(*serveCtx).serve.func3()
2024-07-16T07:46:44.8009853Z       /home/runner/go/pkg/mod/go.etcd.io/etcd@v0.5.0-alpha.5.0.20240320135013-950cd5fbe6ca/embed/serve.go:196 +0x44
2024-07-16T07:46:44.8011109Z   go.etcd.io/etcd/embed.(*serveCtx).serve.func15()
2024-07-16T07:46:44.8012736Z       /home/runner/go/pkg/mod/go.etcd.io/etcd@v0.5.0-alpha.5.0.20240320135013-950cd5fbe6ca/embed/serve.go:197 +0x61
2024-07-16T07:46:44.8013844Z ==================

What did you do?

run ci

What did you expect to see?

no panic

What did you see instead?

meet panic

What version of PD are you using (pd-server -V)?

master
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
impact/panic severity/minor type/bug The issue is confirmed as a bug.
Projects
Questions and Bug Reports
Status: Closed
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

middleware: fix waitForLeader in redirector panic HuSharp/pd
2 participants
@jebter @HuSharp