Skip to content

Commit

Permalink
logictest: deflake a test
Browse files Browse the repository at this point in the history
We've recently merged a test that in very rare circumstances could
produce a float result that differs from the expected one by 1 in the
15th significant digit (after rounding). I believe that could occur,
e.g. when the 15th and 16th significant digits were `35`, and we matched
the spec of supporting 15 significant digits for floats, yet the
rounding makes us return an unexpected result. This commit rounds to the
precision of 1 digit less which should make the test non-flaky.

Release note: None
  • Loading branch information
yuzefovich committed Oct 27, 2020
1 parent e1709ec commit 013da02
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions pkg/sql/logictest/testdata/logic_test/aggregate
Original file line number Diff line number Diff line change
Expand Up @@ -858,10 +858,10 @@ VALUES (1, 1.1, 1.1),
(18, 11.2, 11.2);

query FRF
select sqrdiff(i), round(sqrdiff(f), 13), sqrdiff(d)
select sqrdiff(i), round(sqrdiff(f), 12), sqrdiff(d)
from ifd
----
206.8333333333333333333334 86.2483333333333 86.24833333333333333333333
206.8333333333333333333334 86.248333333333 86.24833333333333333333333

query FRF
SELECT sqrdiff(i), round(sqrdiff(f), 2), sqrdiff(d)
Expand Down

0 comments on commit 013da02

Please sign in to comment.