opt: fix overflow causing zero row-count #38036
Conversation
|
This change is |
There was a problem hiding this comment.
Reviewed 2 of 2 files at r1.
Reviewable status:complete! 1 of 0 LGTMs obtained
There was a problem hiding this comment.
Reviewed 1 of 2 files at r1.
Reviewable status:
pkg/sql/opt/memo/statistics_builder.go, line 2530 at r1 (raw file):
// by constraint.PreferInclusive). if c.Columns.Get(col).Ascending() { distinctCount += float64(end) - float64(start)
Definitely worth a comment explaining that we're avoiding overflow.
We were subtracting two ints that could overflow and then casting the result to a float64. There's an easy solution to avoid the overflow: just cast each integer to a float64 before performing the subtraction. I will backport this to 19.1 and 2.1. Release note (bug fix): Previously, due to a bug when estimating result set sizes in the optimizer, queries involving int ranges that were very large could result in poor plans being generated.
There was a problem hiding this comment.
TFTRs!
bors r+
Reviewable status:
pkg/sql/opt/memo/statistics_builder.go, line 2530 at r1 (raw file):
Previously, RaduBerinde wrote…
Definitely worth a comment explaining that we're avoiding overflow.
Done.
38036: opt: fix overflow causing zero row-count r=justinj a=justinj We were subtracting two ints that could overflow and then casting the result to a float64. There's an easy solution to avoid the overflow: just cast each integer to a float64 *before* performing the subtraction. I will backport this to 19.1 and 2.1. Release note (bug fix): Previously, due to a bug when estimating result set sizes in the optimizer, queries involving int ranges that were very large could result in poor plans being generated. Co-authored-by: Justin Jaffray <justin@cockroachlabs.com>
|
I made a linter at one point that looked for unsafe casts and int arithmetic that didn't check for overflows at one point that would have caught this. It's hard because it has a lot of false positives, but this is definitely a thing we could pursue. |
Build failed |
|
bors r+ |
38036: opt: fix overflow causing zero row-count r=justinj a=justinj We were subtracting two ints that could overflow and then casting the result to a float64. There's an easy solution to avoid the overflow: just cast each integer to a float64 *before* performing the subtraction. I will backport this to 19.1 and 2.1. Release note (bug fix): Previously, due to a bug when estimating result set sizes in the optimizer, queries involving int ranges that were very large could result in poor plans being generated. Co-authored-by: Justin Jaffray <justin@cockroachlabs.com>
|
Filed an issue for the test flake: #38037 |
Build succeeded |
|
Thanks! |
We were subtracting two ints that could overflow and then casting the
result to a float64. There's an easy solution to avoid the overflow:
just cast each integer to a float64 before performing the subtraction.
I will backport this to 19.1 and 2.1.
Release note (bug fix): Previously, due to a bug when estimating result
set sizes in the optimizer, queries involving int ranges that were very
large could result in poor plans being generated.