Fix inaccuracy in decimal128 rounding. #14233

bdice · 2023-09-28T21:07:06Z

Description

Fixes a bug where floating-point values were used in decimal128 rounding, giving wrong results.

Checklist

I am familiar with the Contributing Guidelines.
New or existing tests cover these changes.
The documentation is up to date with these changes.

bdice · 2023-09-28T21:10:07Z

cpp/src/round/round.cu

@@ -271,7 +271,10 @@ std::unique_ptr<column> round_with(column_view const& input,
                               out_view.template end<Type>(),
                               static_cast<Type>(0));
  } else {
-    Type const n = std::pow(10, scale_movement);
+    Type n = 10;
+    for (int i = 1; i < scale_movement; ++i) {


This should use exponentiation-by-squaring for efficiency, and we should have a common implementation of that in libcudf. (We already have two implementations of exponentiation-by-squaring.) I have started work on this and will push to this PR when it's ready, but that might not happen today.

Awesome Bradley.
Would you point to the other two implementations? I was trying to look for them myself earlier today.

One is in fixed point code and has a base that is known as a template parameter:

cudf/cpp/include/cudf/fixed_point/fixed_point.hpp

Line 93 in 7825790

CUDF_HOST_DEVICE inline Rep ipow(T exponent)

The other is in binary ops code and its base and exponent are both runtime parameters:

cudf/cpp/src/binaryop/compiled/operation.cuh

Line 251 in 7825790

struct IntPow {

I am not sure how to best refactor this, but I have drafted some work locally (not yet pushed) that would add a file cudf/detail/utilities/intpow.hpp that centralizes this logic and exposes both a "constexpr base" and "runtime base" form of the function. I'll push this soon so it can be evaluated -- but there's some hangups I am seeing locally with include order and cuda_runtime.h macro conflicts (__forceinline__) with CCCL (resolved in libcudacxx 2.2.0).

Note that we do not have an intpow AST operator, because one has not been requested (to the best of my knowledge). It would go somewhere in here, but would need to have a different name like INTPOW to disambiguate it from the operators that are expected to return floating point values:

cudf/cpp/include/cudf/ast/detail/operators.hpp

Line 403 in 7825790

struct operator_functor<ast_operator::POW, false> {

The topic of integer powers was heavily discussed and analyzed in #10178.

There are two more places where I think this bug might reoccur:

fixed-point unary math ops:

cudf/cpp/src/unary/math_ops.cu

Line 298 in 7825790

Type const n = std::pow(10, -input.type().scale());

fixed-point unary cast ops:

cudf/cpp/src/unary/cast_ops.cu

Line 202 in 7825790

auto const scalar = make_fixed_point_scalar<T>(std::pow(10, -diff), scale_type{diff}, stream);

I'd love help writing some tests that fail for these cases.

I'm starting work on a follow-up PR to fix these additional rescaling issues in #14242. I have a checklist there. This PR should be limited in scope to fixing only the rounding issues, to minimize friction for this fix. I'd like to target refactoring requests to #14242 (aiming for 23.10) or a subsequent release (probably 23.12)

See also #9346

I opened #14243 to track future work on this.

galipremsagar · 2023-10-03T00:04:49Z

@bdice does this PR also fix this issue: #14169

bdice · 2023-10-03T02:35:51Z

@bdice does this PR also fix this issue: #14169

No, I don't think it does from a quick local test. I also tried #14242, which I hope handles a similar bug in rescaling operations. Could you please add more information to #14169 about the root cause and how it was fixed in Arrow? Then we can investigate that after handling this bug and #14242.

harrism

I don't really like the ITM and loop. But you say the refactoring is elsewhere, so I hope you will use something like Jake's approach instead.

bdice · 2023-10-03T05:31:35Z

I don't really like the ITM and loop. But you say the refactoring is elsewhere, so I hope you will use something like Jake's approach instead.

Right -- I definitely don't intend to keep this as a runtime loop. We should be using a lookup table or, if outside the bounds of the lookup, exponentiation-by-squaring. I just don't want to introduce that in this PR, to keep the diff minimal since we're in code freeze for 23.10. I am planning to fix #14242 in the same way as this PR, and then refactor the several places where int-pow operations occur in libcudf in a separate PR (a draft of an int-pow refactored header is also in #14242 for the moment, but I will pull it out into a separate PR before merging that into the frozen 23.10 branch).

…nt values. (#14242) This is a follow-up PR to #14233. This PR fixes a bug where floating-point values were used as intermediates in ceil/floor unary operations and cast operations that require rescaling for fixed-point types, giving inaccurate results. See also: - #14233 (comment) - #14243 Authors: - Bradley Dice (https://github.com/bdice) Approvers: - Mike Wilson (https://github.com/hyperbolic2346) - Vukasin Milovanovic (https://github.com/vuule)

bdice added 2 commits September 28, 2023 14:01

Add failing tests for decimal128 rounding.

f340609

Use integral power computation.

f571b48

github-actions bot added the libcudf Affects libcudf (C++/CUDA) code. label Sep 28, 2023

bdice commented Sep 28, 2023

View reviewed changes

bdice changed the title ~~Add failing tests for decimal128 rounding.~~ Fix inaccuracy in decimal128 rounding. Sep 28, 2023

bdice added bug Something isn't working non-breaking Non-breaking change labels Sep 28, 2023

bdice self-assigned this Sep 28, 2023

revans2 mentioned this pull request Sep 29, 2023

Put back in full decimal support for format_number NVIDIA/spark-rapids#9351

Merged

Rerun CI.

4b4e533

bdice marked this pull request as ready for review October 2, 2023 21:29

bdice requested a review from a team as a code owner October 2, 2023 21:29

bdice requested review from harrism and divyegala and removed request for a team October 2, 2023 21:29

bdice mentioned this pull request Oct 2, 2023

Fix inaccurate ceil/floor and inaccurate rescaling casts of fixed-point values. #14242

Merged

3 tasks

divyegala approved these changes Oct 2, 2023

View reviewed changes

harrism approved these changes Oct 3, 2023

View reviewed changes

bdice mentioned this pull request Oct 3, 2023

Consolidate and optimize integer power implementations in libcudf #14243

Open

raydouglass merged commit 66a655c into rapidsai:branch-23.10 Oct 3, 2023
57 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix inaccuracy in decimal128 rounding. #14233

Fix inaccuracy in decimal128 rounding. #14233

bdice commented Sep 28, 2023

bdice Sep 28, 2023

davidwendt Sep 28, 2023

bdice Sep 29, 2023

bdice Sep 29, 2023

bdice Sep 29, 2023

bdice Sep 29, 2023 •

edited

Loading

bdice Oct 2, 2023

jrhemstad Oct 2, 2023

bdice Oct 3, 2023

galipremsagar commented Oct 3, 2023

bdice commented Oct 3, 2023

harrism left a comment

bdice commented Oct 3, 2023 •

edited

Loading

Fix inaccuracy in decimal128 rounding. #14233

Fix inaccuracy in decimal128 rounding. #14233

Conversation

bdice commented Sep 28, 2023

Description

Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bdice Sep 29, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

galipremsagar commented Oct 3, 2023

bdice commented Oct 3, 2023

harrism left a comment

Choose a reason for hiding this comment

bdice commented Oct 3, 2023 • edited Loading

bdice Sep 29, 2023 •

edited

Loading

bdice commented Oct 3, 2023 •

edited

Loading