Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

contention: store contention events on non-SQL keys #61828

Merged
merged 1 commit into from
Mar 15, 2021

Conversation

yuzefovich
Copy link
Member

Previously, whenever we tried to add a contention event on a non-SQL
key, it would encounter an error during decoding tableID/indexID pair,
and the event was dropped. This commit extends the contention registry
to additionally store information about contention on non-SQL keys. That
information is stored in two levels:

  • on the top level, all SingleNonSQLKeyContention objects are ordered
    by their keys
  • on the bottom level, all SingleTxnContention objects are ordered by the
    number of times that transaction was observed to contend with other
    transactions.

SingleTxnContention protobuf message is moved out of
SingleKeyContention and is reused for non-SQL keys. This commit also
updates the status server API response. I assume that no changes are
needed with regards to backwards compatibility since the original
version was merged just a few weeks ago, and we haven't had a beta
released since then.

Fixes: #60669.

Release note (sql change): CockroachDB now also stores the information
about contention on non-SQL keys.

@yuzefovich yuzefovich requested review from a team as code owners March 11, 2021 02:40
@cockroach-teamcity
Copy link
Member

This change is Reviewable

Copy link
Member Author

@yuzefovich yuzefovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @yuzefovich)


pkg/sql/contention/registry.go, line 83 at r1 (raw file):

	const estimatedAverageKeySize = 64
	txnsMapSize := maxNumTxns * int(unsafe.Sizeof(uuid.UUID{})+unsafe.Sizeof(int(0)))
	orderedKeyMapSize := orderedKeyMapMaxSize * (int(unsafe.Sizeof(comparableKey{})+estimatedAverageKeySize) + txnsMapSize)

Notable change here, I think the previous calculation was incorrect.

Also, what do we want to do with this method? We're now using the registry, but it's estimated max memory footprint (assuming it is about 4.5MiB) seems low enough to me that we can just ignore it since it is on a per node basis. I think we could be concerned if the keys in the contention events are large, but then the estimate doesn't help us much.

@yuzefovich yuzefovich force-pushed the non-sql-keys branch 2 times, most recently from 75cc1c9 to c0c39ce Compare March 11, 2021 23:11
@yuzefovich yuzefovich requested review from asubiotto and a team March 11, 2021 23:11
Copy link
Contributor

@asubiotto asubiotto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 11 of 12 files at r1, 2 of 2 files at r2.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @yuzefovich)


pkg/sql/contention/registry.go, line 83 at r1 (raw file):

Previously, yuzefovich (Yahor Yuzefovich) wrote…

Notable change here, I think the previous calculation was incorrect.

Also, what do we want to do with this method? We're now using the registry, but it's estimated max memory footprint (assuming it is about 4.5MiB) seems low enough to me that we can just ignore it since it is on a per node basis. I think we could be concerned if the keys in the contention events are large, but then the estimate doesn't help us much.

I think it's fine to delete it.

Previously, whenever we tried to add a contention event on a non-SQL
key, it would encounter an error during decoding tableID/indexID pair,
and the event was dropped. This commit extends the contention registry
to additionally store information about contention on non-SQL keys. That
information is stored in two levels:
- on the top level, all `SingleNonSQLKeyContention` objects are ordered
  by their keys
- on the bottom level, all `SingleTxnContention` objects are ordered by the
  number of times that transaction was observed to contend with other
  transactions.

`SingleTxnContention` protobuf message is moved out of
`SingleKeyContention` and is reused for non-SQL keys. This commit also
updates the status server API response. I assume that no changes are
needed with regards to backwards compatibility since the original
version was merged just a few weeks ago, and we haven't had a beta
released since then.

Release note (sql change): CockroachDB now also stores the information
about contention on non-SQL keys.
Copy link
Member Author

@yuzefovich yuzefovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @asubiotto)


pkg/sql/contention/registry.go, line 83 at r1 (raw file):

Previously, asubiotto (Alfonso Subiotto Marqués) wrote…

I think it's fine to delete it.

Cool, removed.

@yuzefovich
Copy link
Member Author

TFTR!

bors r+

@yuzefovich
Copy link
Member Author

bors r+

@craig
Copy link
Contributor

craig bot commented Mar 15, 2021

Already running a review

@craig
Copy link
Contributor

craig bot commented Mar 15, 2021

Build succeeded:

@craig craig bot merged commit ae3a358 into cockroachdb:master Mar 15, 2021
@yuzefovich yuzefovich deleted the non-sql-keys branch March 15, 2021 21:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

sql: tolerate non-SQL keys in AddContentionEvent
3 participants