kv/client: Fix incorrect behaviors when reconnecting the stream

[MyonKeminta]

What problem does this PR solve?

This PR fixes some bugs that the kv client didn't handle TiKV reconnecting correctly.

Bug 1:

1. In dispatchRequests, it sends a request to a region, and the region info is added to the pendingRegions map.
2. Due to some reason like unstable network or TiKV crash, it failed to send the request, and tries to send it again.
3. It adds the new pending region info to the pendingRegions map with a new requestID, and tries to re-establish the stream.
4. The receiveFromStream goroutine noticed the stream is broken, and closes all pending regions on this stream. In this case, the new pending region info will be regarded as a dead request and will be closed. However, the dispatchRequests goroutine is just trying to reconnect and send a new request to that region.
   5.1 If the reconnecting succeeded and received an event from that region, no pending region info will be found and the error "received an event but neither pending region nor running region was found" will be thrown.
   5.2 If it didn't finish reconnecting and receive an Event so quickly, receiveFromStream will send an error to errCh when closing the pending regions, and then the region's range will be unlocked, and then locked again and send another request. So the region may be requested twice, and an DuplicatedRequest error will occur.

Bug 2:
When a region meets error when trying to establish the stream, it's region info may be left in the pendingRegions map when it retries. and as a result the region may be double-cancelled, which may cause unlocking an not locked range panic.

What is changed and how it works?

Bug 1:
When reconnecting, we deleting the stream from the stream map as well as deleting the pendingRegions from the storePendingRegions map (indexed by store addr). After deleting it, the next time an request is sent to this store, a new map will be created for the new reconnected stream.

Bug 2:
On establishing stream failed, delete the region info from pendingRegions

Check List

Tests

• Manual test (add detailed scripts or steps below)

Related changes

[amyangfei]

LGTM

[amyangfei]

/run-integration-tests

[MyonKeminta]

/run-integration-tests

[zier-one]

LGTM