Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

opt: fix issue with dangling pointer in statisticsBuilder #37983

Merged
merged 1 commit into from
Jun 6, 2019

Conversation

rytaft
Copy link
Collaborator

@rytaft rytaft commented Jun 3, 2019

This commit fixes an issue where the statisticsBuilder was using
an invalid pointer to update statistics for some multi-column stats
in the local table statistics cache. The fix is to re-fetch the
pointer to the stats just before updating them, in case the
previously fetched address is no longer valid.

This problem is very hard to reproduce, so I had to use the somewhat
complex query from the issue for a regression test.

Fixes #37953

Release note: None

@rytaft rytaft requested a review from a team as a code owner June 3, 2019 21:22
@cockroach-teamcity
Copy link
Member

This change is Reviewable

Copy link
Contributor

@andy-kimball andy-kimball left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you worried that we'll continue having this kind of bug in the future, given the subtlety of it? I wonder if we should change ColStatsMap and colStatVal to directly reference *ColumnStatistic rather than keeping a pos into the initial array and other slice. This would require us to allocate ColumnStatistic objects on the heap once we got past the initial 3, but that's an uncommon case, so probably wouldn't have measurable impact on perf.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @justinj and @RaduBerinde)

@rytaft
Copy link
Collaborator Author

rytaft commented Jun 4, 2019

This is definitely a possibility. My intuition is that colStatLeaf is the only function that is susceptible to this bug since it recursively adds column stats to the same props.Statistics, while other colStatXX functions recursively add column stats only to their children. A quick audit of the code shows this to be the case, but it's hard to guarantee this will always be true....

I suppose one solution would be to just do this simple fix of re-fetching the column stat now but keep an eye out for this bug in the near future and implement your suggested change if it looks like the bug is rearing it's head again. What do you think?

Copy link
Collaborator Author

@rytaft rytaft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Friendly ping

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @justinj and @RaduBerinde)

Copy link
Contributor

@andy-kimball andy-kimball left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds OK. Though maybe add additional comments in ColStatsMap mutable methods to warn callers to be careful.
Otherwise, :lgtm:

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @justinj and @RaduBerinde)

This commit fixes an issue where the statisticsBuilder was using
an invalid pointer to update statistics for some multi-column stats
in the local table statistics cache. The fix is to re-fetch the
pointer to the stats just before updating them, in case the
previously fetched address is no longer valid.

This problem is very hard to reproduce, so I had to use the somewhat
complex query from the issue for a regression test.

Fixes cockroachdb#37953

Release note: None
@rytaft
Copy link
Collaborator Author

rytaft commented Jun 5, 2019

TFTR!

Added comments above all functions in ColStatsMap that return a *ColumnStatistic

@rytaft
Copy link
Collaborator Author

rytaft commented Jun 6, 2019

bors r+

craig bot pushed a commit that referenced this pull request Jun 6, 2019
37983: opt: fix issue with dangling pointer in statisticsBuilder r=rytaft a=rytaft

This commit fixes an issue where the `statisticsBuilder` was using
an invalid pointer to update statistics for some multi-column stats
in the local table statistics cache. The fix is to re-fetch the
pointer to the stats just before updating them, in case the
previously fetched address is no longer valid.

This problem is very hard to reproduce, so I had to use the somewhat
complex query from the issue for a regression test.

Fixes #37953

Release note: None

Co-authored-by: Rebecca Taft <becca@cockroachlabs.com>
@craig
Copy link
Contributor

craig bot commented Jun 6, 2019

Build succeeded

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

sql: internal error: estimated distinct and/or null count must be non-zero
3 participants