Skip to content

Commit

Permalink
sql: implement locality-aware DistSQL planning in multi-tenant setup
Browse files Browse the repository at this point in the history
This commit implements the region-aware DistSQL physical planning in
the multi-tenant setup. Previously, we would use a round-robin strategy
of assigning single-range spans to all available SQL instances and now
we'll make a much better decision.

The algorithm is as follows:
1. find all available SQL pods and determine their locality (via the
`sqlinstance.Provider`)
2. iterate over all spans and break them down into single-range spans
using the range cache that the gateway SQL instance maintains
3. for each single-range span, find the KV leaseholder and determine its
locality (via the `kvtenantccl.Connector`), then pick a random SQL
instance in the same locality
4. if there is no SQL instance in that locality, then assign the
single-range span to the gateway SQL instance.

At the moment, the only locality "tier" that we look at is the "region",
but in the future we can easily extend it (say, to also include the
"availability zone").

The choice of randomly picking the SQL instance in the same locality in
the step 3 is done so that we distribute the load evenly among multiple
pods.

The step 2 was already being performed for the system tenant, so this
commit extracts out some helpers to reuse the code as much as possible.
Additionally, it fixes an omission in the physical planning in
multi-tenancy when the query has a LIMIT (`getInstanceIDForScan`).

If in step 1 we couldn't determine the region of any available SQL pod,
then we fallback to the old naive locality-ignorant strategy of
assigning pods in round-robin fashion.

Release note: None (I don't think that any of these changes are
user-visible since the serverless doesn't yet support the multi-region
clusters.)
  • Loading branch information
yuzefovich committed Aug 2, 2022
1 parent de3ca17 commit 9eb5ac3
Show file tree
Hide file tree
Showing 6 changed files with 488 additions and 163 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,17 @@ go_test(
data = [
"//c-deps:libgeos", # keep
"//pkg/sql/logictest:testdata", # keep
"//pkg/sql/opt/exec/execbuilder:testdata", # keep
],
shard_count = 2,
shard_count = 3,
deps = [
"//pkg/build/bazel",
"//pkg/ccl",
"//pkg/ccl/utilccl",
"//pkg/security/securityassets",
"//pkg/security/securitytest",
"//pkg/server",
"//pkg/sql",
"//pkg/sql/logictest",
"//pkg/testutils/serverutils",
"//pkg/testutils/skip",
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions pkg/sql/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -468,6 +468,7 @@ go_library(
"//pkg/util/mon",
"//pkg/util/protoutil",
"//pkg/util/quotapool",
"//pkg/util/randutil",
"//pkg/util/retry",
"//pkg/util/ring",
"//pkg/util/stop",
Expand Down
Loading

0 comments on commit 9eb5ac3

Please sign in to comment.