Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kv: monitor memory usage by KV batches and write intents #10320

Closed
knz opened this issue Oct 30, 2016 · 9 comments
Closed

kv: monitor memory usage by KV batches and write intents #10320

knz opened this issue Oct 30, 2016 · 9 comments
Labels
C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. C-performance Perf of queries or internals. Solution not expected to change functional behavior. no-issue-activity O-sre For issues SRE opened or otherwise cares about tracking. T-kv KV Team X-stale

Comments

@knz
Copy link
Contributor

knz commented Oct 30, 2016 •
edited by cockroach-jira-scripts
Loading

Pursuant to #8691 we need to extend the monitoring to KV batches and write intents, which are currently unaccounted for. This is needed to ensure in particular that INSERT/UPDATE touching many rows don't blow up the coordinating node's memory.

Jira issue: CRDB-6138
@knz knz added the C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. label Oct 30, 2016
@knz knz added this to the 1.0 milestone Oct 30, 2016
@cuongdo
Copy link
Contributor

cuongdo commented Apr 4, 2017

@knz would you still advocate for fixing this in 1.0?

@dianasaur323
Copy link
Contributor

@cuongdo @knz I'm going to assign this! If it's not a blocker for 1.0, we could also stick it into the Q2 2017 milestone, but leaving it to you two to decide.

@cuongdo cuongdo modified the milestones: Later, 1.0 Apr 5, 2017
@knz knz added C-performance Perf of queries or internals. Solution not expected to change functional behavior. S-1-stability Severe stability issues that can be fixed by upgrading, but usually don’t resolve by restarting labels Jun 7, 2017
@knz
Copy link
Contributor Author

knz commented Jun 7, 2017

cc @andreimatei
I am not planning to work on this any time soon (Also Andrei helped me understand that the scope of the issue is larger than what the small description would otherwise suggest)

@knz knz removed their assignment Jun 7, 2017
@knz knz modified the milestones: 1.1, Later Jun 7, 2017
@knz knz changed the title sql: monitor memory usage by KV batches and write intents sql+kv: monitor memory usage by KV batches and write intents Jun 7, 2017
@knz knz mentioned this issue Jun 7, 2017
Closed
13 tasks
@knz
Copy link
Contributor Author

knz commented Jun 7, 2017

Issue added to #8879

@cuongdo cuongdo modified the milestones: 1.2, 1.1 Aug 22, 2017
@bdarnell bdarnell modified the milestones: 2.0, 2.1 Feb 8, 2018
@jordanlewis jordanlewis removed the S-1-stability Severe stability issues that can be fixed by upgrading, but usually don’t resolve by restarting label Apr 30, 2018
@jordanlewis jordanlewis added the A-kv-client Relating to the KV client and the KV interface. label Apr 30, 2018
@tbg
Copy link
Member

tbg commented Jul 19, 2018

I feel bad kicking this can down the road once more, but realistically this isn't going to happen for 2.1.

@tbg tbg modified the milestones: 2.1, 2.2 Jul 19, 2018
@tbg tbg removed A-disaster-recovery A-kv-transactions Relating to MVCC and the transactional model. labels Jul 31, 2018
@tbg tbg added A-coreperf and removed A-kv-distribution Relating to rebalancing and leasing. A-kv-client Relating to the KV client and the KV interface. A-storage Relating to our storage engine (Pebble) on-disk storage. A-kv-replication Relating to Raft, consensus, and coordination. labels Jul 31, 2018
@tbg tbg mentioned this issue Oct 11, 2018
Closed
@tbg tbg mentioned this issue Oct 16, 2018
Closed
@awoods187 awoods187 removed this from the 19.1 milestone Mar 22, 2019
@joshimhoff
Copy link
Collaborator

Is the current state that we don't account for memory usage that happens at KV level? Asking for oncall understanding.

@knz
Copy link
Contributor Author

knz commented Dec 6, 2019

yes correct

@joshimhoff joshimhoff added the O-sre For issues SRE opened or otherwise cares about tracking. label Nov 3, 2020
@jordanlewis jordanlewis changed the title sql+kv: monitor memory usage by KV batches and write intents kv: monitor memory usage by KV batches and write intents Jun 10, 2021
@jordanlewis
Copy link
Member

The SQL portion of this is done - we have memory monitoring in place for the KV batches that SQL sends and receives. As such I've updated the title of this issue to be about kv.

@jlinder jlinder added the T-kv KV Team label Jun 16, 2021
@ajwerner ajwerner mentioned this issue Feb 10, 2022
Open
@github-actions
Copy link

We have marked this issue as stale because it has been inactive for
18 months. If this issue is still relevant, removing the stale label
or adding a comment will keep it active. Otherwise, we'll close it in
10 days to keep the issue queue tidy. Thank you for your contribution
to CockroachDB!

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Oct 10, 2023
@github-project-automation github-project-automation bot moved this to Closed in KV Aug 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. C-performance Perf of queries or internals. Solution not expected to change functional behavior. no-issue-activity O-sre For issues SRE opened or otherwise cares about tracking. T-kv KV Team X-stale
Projects
No open projects
KV
Archived in project
Milestone
No milestone
Development

No branches or pull requests
10 participants
@jordanlewis @bdarnell @jlinder @knz @joshimhoff @cuongdo @tbg @nvanbenschoten @dianasaur323 @awoods187