rpc: don't enable internalClientAdapter until rpcServer is operational #39497
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #39415.
This commit fixes the issue we saw in #39415, where a node would attempt
to allocate a store ID for a new store on an existing node and would
get stuck. The reason it would get stuck is because the KV request to
allocate the store ID could be routed to the node's own replica for
Range 1 (when doing a range descriptor lookup) over the node's
internalClientAdapter. This KV request could then get stuck attempting
to acquire a lease because we would still be blocking all Raft RPC
traffic (see rpcsAllowedWhileBootstrapping). This would cause the KV
request to stall indefinitely.
The fix is to tie the operationality of the internalClientAdapter to
that of the rpcServer. If we don't want to be serving external KV
traffic during this node initialization period then we shouldn't
be serving internal KV traffic.
DNM: this still needs a higher-level test that verifies that the bug
during the bootstrap problem we saw when adding a store to a node with
existing stores is no longer possible. I have only verified this
manually so far.
Release note: None