TiKV panic during go-tpc prepare

[cosven]

Bug Report

What version of TiKV are you using?

v5.0.2

What operating system and CPU are you using?

DBaaS T3.standard

Steps to reproduce

1. deploy a T3.standard cluster with 2db 6kv
2. run go-tpc tpcc prepare -T 64 --warehouses 1000

What did you expect?

no panic

What did happened?

TiKV panic with followering log

[2021/06/18 09:37:26.917 +00:00] [FATAL] [lib.rs:465] ["unexpected error: Store(Other(\"[components/raftstore/src/store/peer_storage.rs:980]: failed to get snapshot after 5 times\")), raft_id: 2652, region_id: 2650"] [backtrace="stack backtrace:\n   0: tikv_util::set_panic_hook::{{closure}}\n             at home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tikv/components/tikv_util/src/lib.rs:464\n   1: std::panicking::rust_panic_with_hook\n             at rustc/bc39d4d9c514e5fdb40a5782e6ca08924f979c35//library/std/src/panicking.rs:595\n   2: std::panicking::begin_panic_handler::{{closure}}\n             at rustc/bc39d4d9c514e5fdb40a5782e6ca08924f979c35//library/std/src/panicking.rs:497\n   3: std::sys_common::backtrace::__rust_end_short_backtrace\n             at rustc/bc39d4d9c514e5fdb40a5782e6ca08924f979c35//library/std/src/sys_common/backtrace.rs:141\n   4: rust_begin_unwind\n             at rustc/bc39d4d9c514e5fdb40a5782e6ca08924f979c35//library/std/src/panicking.rs:493\n   5: std::panicking::begin_panic_fmt

Some metrics

[cosven]

/type bug
/severity critical
/assign @gengliqi

[cosven]

/assign @gengliqi

[gengliqi]

It's fixed by #10454.