Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Document new approach to locking and queueing in the key value layer #6571

Closed
2 tasks
rmloveland opened this issue Feb 19, 2020 · 2 comments · Fixed by #7020
Closed
2 tasks

Document new approach to locking and queueing in the key value layer #6571

rmloveland opened this issue Feb 19, 2020 · 2 comments · Fixed by #7020
Assignees
@rmloveland
Labels
P-1 High priority; must be done this release
Milestone
20.1

Comments

@rmloveland
Copy link
Contributor

We need to update the architecture documentation to discuss the new approach to locking and queuing in the key-value layer.

AFAICT this new approach is enabled by the new concurrency control package/lock table referred to by this PR (and this one, and probably others).

Estimated scope of work:

  • Update the architecture docs (probably the storage layer) to describe the new concurrency manager / how locking and queuing works

  • Review other areas of the architecture docs to ensure they reference (or do not reference) this new layer, as appropriate
@rmloveland rmloveland self-assigned this Feb 19, 2020
@rmloveland rmloveland added A-kv-storage P-1 High priority; must be done this release labels Feb 19, 2020
@rmloveland rmloveland added this to the 20.1 milestone Feb 19, 2020
@jseldess
Copy link
Contributor

Thanks for being proactive here, @rmloveland. Is this not reflected on the KV product roadmap? I'm surprised we don't already have a docs issue for it.

@rmloveland
Copy link
Contributor Author

Is this not reflected on the KV product roadmap?

I did some searching in AT but was not able to find anything beyond the SQL-level description of the work.

@rmloveland rmloveland mentioned this issue Feb 24, 2020
Merged
rmloveland added a commit that referenced this issue Mar 30, 2020
@rmloveland
Document new approach to locking in KV layer
2c31804 
Fixes #6571.

Summary of changes (all to 'Transaction Layer' page):

- Update 'Writes and reads' section to note that there are now *two*
  types of locks: write intents, and unreplicated locks managed by the
  concurrency manager in a per-node lock table (about which more below).

- Update 'Write intents' section with information about new concurrency
  control mechanism using unreplicated lock table.

- Add new 'Concurrency control' section describing concurrency manager
  and lock table at a high level, with links to `SELECT FOR UPDATE`
  docs.  This is followed by more detailed subsections that are mostly
  adapted from the in-code comments in
  `pkg/kv/kvserver/concurrency/concurrency_control.go`.
@rmloveland rmloveland mentioned this issue Mar 30, 2020
Merged
rmloveland added a commit that referenced this issue Mar 30, 2020
@rmloveland
Document new approach to locking in KV layer
d654d54 
Fixes #6571.

Summary of changes (all to 'Transaction Layer' page):

- Update 'Writes and reads' section to note that there are now *two*
  types of locks: write intents, and unreplicated locks managed by the
  concurrency manager in a per-node lock table (about which more below).

- Update 'Write intents' section with information about new concurrency
  control mechanism using unreplicated lock table.

- Add new 'Concurrency control' section describing concurrency manager
  and lock table at a high level, with links to `SELECT FOR UPDATE`
  docs.  This is followed by more detailed subsections that are mostly
  adapted from the in-code comments in
  `pkg/kv/kvserver/concurrency/concurrency_control.go`.
rmloveland added a commit that referenced this issue Mar 30, 2020
@rmloveland
Document new approach to locking in KV layer
d9d0aec 
Fixes #6571.

Summary of changes (all to 'Transaction Layer' page):

- Update 'Writes and reads' section to note that there are now *two*
  types of locks: write intents, and unreplicated locks managed by the
  concurrency manager in a per-node lock table (about which more below).

- Update 'Write intents' section with information about new concurrency
  control mechanism using unreplicated lock table.

- Add new 'Concurrency control' section describing concurrency manager
  and lock table at a high level, with links to `SELECT FOR UPDATE`
  docs.  This is followed by more detailed subsections that are mostly
  adapted from the in-code comments in
  `pkg/kv/kvserver/concurrency/concurrency_control.go`.
rmloveland added a commit that referenced this issue Apr 2, 2020
@rmloveland
Document new approach to locking in KV layer
4defb19 
Fixes #6571.

Summary of changes (all to 'Transaction Layer' page):

- Update 'Writes and reads' section to note that there are now *two*
  types of locks: write intents, and unreplicated locks managed by the
  concurrency manager in a per-node lock table (about which more below).

- Update 'Write intents' section with information about new concurrency
  control mechanism using unreplicated lock table.

- Add new 'Concurrency control' section describing concurrency manager
  and lock table at a high level, with links to `SELECT FOR UPDATE`
  docs.  This is followed by more detailed subsections that are mostly
  adapted from the in-code comments in
  `pkg/kv/kvserver/concurrency/concurrency_control.go`.  Also, move
  'Latch manager' section here, since it's being overseen by the
  concurrency manager.
rmloveland added a commit that referenced this issue Apr 13, 2020
@rmloveland
Document new approach to locking in KV layer
62a37da 
Fixes #6571.

Summary of changes (all to 'Transaction Layer' page):

- Update 'Writes and reads' section to note that there are now *two*
  types of locks: write intents, and unreplicated locks managed by the
  concurrency manager in a per-node lock table (about which more below).

- Update 'Write intents' section with information about new concurrency
  control mechanism using unreplicated lock table.

- Add new 'Concurrency control' section describing concurrency manager
  and lock table at a high level, with links to `SELECT FOR UPDATE`
  docs.  This is followed by more detailed subsections that are mostly
  adapted from the in-code comments in
  `pkg/kv/kvserver/concurrency/concurrency_control.go`.  Also, move
  'Latch manager' section here, since it's being overseen by the
  concurrency manager.
rmloveland added a commit that referenced this issue Apr 13, 2020
@rmloveland
Document new approach to locking in KV layer
fe055dc 
Fixes #6571.

Summary of changes (all to 'Transaction Layer' page):

- Update 'Writes and reads' section to note that there are now *two*
  types of locks: write intents, and unreplicated locks managed by the
  concurrency manager in a per-node lock table (about which more below).

- Update 'Write intents' section with information about new concurrency
  control mechanism using unreplicated lock table.

- Add new 'Concurrency control' section describing concurrency manager
  and lock table at a high level, with links to `SELECT FOR UPDATE`
  docs.  This is followed by more detailed subsections that are mostly
  adapted from the in-code comments in
  `pkg/kv/kvserver/concurrency/concurrency_control.go`.  Also, move
  'Latch manager' section here, since it's being overseen by the
  concurrency manager.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rmloveland rmloveland

Labels
P-1 High priority; must be done this release
Projects
None yet
Milestone
20.1
Development

Successfully merging a pull request may close this issue.

Document new approach to locking in KV layer cockroachdb/docs
2 participants
@rmloveland @jseldess