-
Notifications
You must be signed in to change notification settings - Fork 1k
Commit 7a92be5
authored
Refactor arrow-cast decimal casting to unify the rescale logic used in Parquet variant casts (#8689)
# Which issue does this PR close?
- Closes #8670.
# Rationale for this change
We currently have two separate code paths that both handle decimal
casting between different (precision, scale) pairs. Without unifying the
logic, a fix in one place often needs to be duplicated in the other
(e.g., #8579 fixed the
`arrow-cast`
and #8552 fixed the `parquet-variant-compute`), which can easily lead to
divergence when contributors lack full context. This PR consolidates the
decimal rescale logic for both `arrow-cast` and
`parquet-variant-compute`.
# What changes are included in this PR?
1. Extract the shared array-unary logic from
`convert_to_smaller_scale_decimal` and
`convert_to_bigger_or_equal_scale_decimal` into `apply_decimal_cast`
2. Move the rescale-closure creation into `make_upscaler` and
`make_downscaler` so that they can be used in `parquet-compute-variant`
3. rework `rescale_decimal` in `parquet-compute-variant` to use the new
`make_upscaler` and `make_downscaler` utilities.
One challenge is incorporating the large-scale reduction path (aka the
`delta_scale` cannot fit into `I::MAX_PRECISION`) into `make_downscaler`
without hurting performance. Returning 0 directly is usually cheaper
than applying a unary operation to return zero. Therefore,
`make_downscaler` may return None, and it is the caller’s responsibility
to handle this case appropriately based on the documented behavior.
# Are these changes tested?
Covered by existing tests
# Are there any user-facing changes?
No1 parent 06c49db commit 7a92be5Copy full SHA for 7a92be5
File tree
Expand file treeCollapse file tree
3 files changed
+289
-161
lines changedOpen diff view settings
Filter options
- arrow-cast/src/cast
- parquet-variant-compute/src
Expand file treeCollapse file tree
3 files changed
+289
-161
lines changedOpen diff view settings
0 commit comments