asim: support load based range splitting #82630
Assignees
Labels
A-kv
Anything in KV that doesn't belong in a more specific category.
A-kv-simulation
Relating to allocation simulation.
C-enhancement
Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)
sync-me-7
sync-me-8
T-kv
KV Team
Comments
kvoli
added
C-enhancement
kvoli
changed the title
Merged
kvoli
added a commit
to kvoli/cockroach
that referenced
this issue
Jul 13, 2022
This patch adds load based splitting to the allocation simulator. It uses the production code path, `pkg/kv/kvserver/split`, to decide when and which key to split on. To enable split recommendations from this package, load events are recorded to the splitter and split suggestions enqueued into the simulator split queue. Split keys are likewise found via consulting the split decider first and when not found, the split queue wil instead split evenly (50/50) on the number of keys instead. resolves cockroachdb#82630 Release note: None
kvoli
added a commit
to kvoli/cockroach
that referenced
this issue
Jul 14, 2022
This patch adds load based splitting to the allocation simulator. It uses the production code path, `pkg/kv/kvserver/split`, to decide when and which key to split on. To enable split recommendations from this package, load events are recorded to the splitter and split suggestions enqueued into the simulator split queue. Split keys are likewise found via consulting the split decider first and when not found, the split queue wil instead split evenly (50/50) on the number of keys instead. resolves cockroachdb#82630 Release note: None
craig bot
pushed a commit
that referenced
this issue
Jul 18, 2022
84306: asim: add load splits r=kvoli a=kvoli This patch adds load based splitting to the allocation simulator. It uses the production code path, `pkg/kv/kvserver/split`, to decide when and which key to split on. To enable split recommendations from this package, load events are recorded to the splitter and split suggestions enqueued into the simulator split queue. Split keys are likewise found via consulting the split decider first and when not found, the split queue wil instead split evenly (50/50) on the number of keys instead. resolves #82630 Release note: None 84451: dev,genbzl: add support for generating syntax diagrams r=ajwerner a=ajwerner Fixes #84443. Release note: None 84571: changefeedccl: prerequisite changes for `DROP COLUMN` r=ajwerner a=ajwerner This is the first two commits from #84563. They are needed to ensure that we don't change the behavior of `DROP COLUMN` when we support it in the declarative schema changer. The issue is that there the protocol is to create a new primary index and swap to it. The column becomes a non-public before the index swap, so the primary index swap is no longer a logical schema change of any kind. With this change, we can now detect that and properly restart as opposed to stop. Also, the newly added testing uncovers some badness in how we classify some other schema changes, and should generally be useful. Co-authored-by: Austen McClernon <austen@cockroachlabs.com> Co-authored-by: Andrew Werner <awerner32@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The asim package does not currently support load based splitting of ranges [code].
Load based splitting is a key component of the distribution algorithm, so it is important to include when simulating the algorithm.
The solution is to add a load based splitting component to the asim package. Namely, tracking the requests per range and selecting a split key when requests exceeds a configurable threshold. Note here that this is not QPS, rather requests [code].
This should be applied to state using the existing
SplitRangemethod, in the state package.Jira issue: CRDB-16616
Epic CRDB-14561
The text was updated successfully, but these errors were encountered: