Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kvserver: add AdminSplit and AdminScatter to secondary tenants API #74835

Merged
merged 1 commit into from
Jan 19, 2022

Conversation

shralex
Copy link
Contributor

@shralex shralex commented Jan 14, 2022

This adds support for AdminSplit and AdminScatter for secondary tenants. This API allows indicating to KV that more data will be ingested, and so the range should be split and re-distributed across the cluster. This API will not be exposed through SQL, and in the future we might change it to give KV more control over whether and how to deal with expected ingest load. More discussion can be found in the github issue: #74389 and Epic: https://cockroachlabs.atlassian.net/browse/CRDB-10720

Release Note: None

@cockroach-teamcity
Copy link
Member

This change is Reviewable

Copy link
Collaborator

@ajstorm ajstorm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉 thanks for getting to this so quickly.

Do we have a test we need to update to validate this behavior?

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @aayushshah15, @nvanbenschoten, and @shralex)


-- commits, line 2 at r1:
Might be good to give a bit more context as to why we're making this change. The context from #74389 might help. Also, can you link this epic to that issue (but maybe informs and not resolves)?

Copy link
Contributor Author

@shralex shralex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated now, thank you!

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @aayushshah15, @nvanbenschoten, and @shralex)

Copy link
Collaborator

@ajstorm ajstorm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 1 of 1 files at r1, 1 of 1 files at r2, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @aayushshah15, @nvanbenschoten, and @shralex)

Copy link
Collaborator

@ajstorm ajstorm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be good to get someone from KV to quickly double check though.

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @aayushshah15, @nvanbenschoten, and @shralex)

Copy link
Member

@nvanbenschoten nvanbenschoten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r1, 1 of 1 files at r2, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @aayushshah15 and @shralex)


-- commits, line 13 at r3:
nit: extra release note.


pkg/rpc/auth_test.go, line 252 at r3 (raw file):

			{
				req: &roachpb.BatchRequest{Requests: makeReqs(
					makeAdminReq("a"),

These test cases were meant to represent an arbitrary unsupported admin operation — not a split in particular. Let's change makeAdminReq so that it doesn't perform an AdminSplit and instead performs an admin operation that still isn't permitted, like AdminRelocateRange. Then we can add additional test cases for AdminSplit and AdminScatter.

@shralex shralex force-pushed the split_scatter_tenants branch 2 times, most recently from d97b8a1 to 79c909e Compare January 18, 2022 23:40
This adds support for AdminSplit and AdminScatter for secondary tenants.
This API allows indicating to KV that more data will be ingested, and
so the range should be split and re-distributed across the cluster.
This API will not be exposed through SQL, and in the future we might change
it to give KV more control over whether and how to deal with expected ingest
load. More discussion can be found in the github issue: cockroachdb#74389
 and Epic: https://cockroachlabs.atlassian.net/browse/CRDB-10720

Release note: None
Copy link
Contributor Author

@shralex shralex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @aayushshah15, @ajstorm, @nvanbenschoten, and @shralex)


pkg/rpc/auth_test.go, line 252 at r3 (raw file):

AdminRelocateRange
Done

Copy link
Contributor Author

@shralex shralex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @aayushshah15, @ajstorm, @nvanbenschoten, and @shralex)


-- commits, line 13 at r3:

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

nit: extra release note.

Done

Copy link
Member

@nvanbenschoten nvanbenschoten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 1 of 1 files at r4, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (and 1 stale) (waiting on @aayushshah15 and @shralex)

@shralex
Copy link
Contributor Author

shralex commented Jan 19, 2022

bors r+

@craig
Copy link
Contributor

craig bot commented Jan 19, 2022

Build succeeded:

@craig craig bot merged commit eb26eb3 into cockroachdb:master Jan 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

kv: Add API to allow for split/scatter behavior in multi-tenant environments
4 participants