Skip to content

Conversation

@AdamGS
Copy link
Contributor

@AdamGS AdamGS commented Oct 3, 2025

Which issue does this PR close?

Rationale for this change

Backports initial support for decimal 32/64 to the 50.x branch

What changes are included in this PR?

See original PR:

  • Multiple aggregation functions
  • Some decimal type coercion

Are these changes tested?

Includes some tests and reinforced by SLT. Follow up PR contains more tests.

Are there any user-facing changes?

No

@AdamGS AdamGS changed the title Support Decimal32/64 types (#17501) [Branch-50] Backport: Support Decimal32/64 types (#17501) Oct 3, 2025
@github-actions github-actions bot added sql SQL Planner logical-expr Logical plan and expressions core Core DataFusion crate common Related to common crate proto Related to proto crate functions Changes to functions implementation labels Oct 3, 2025
@alamb
Copy link
Contributor

alamb commented Oct 3, 2025

Why does this PR has substantially more lines that the original PR?

This PR
Screenshot 2025-10-03 at 4 00 40 PM

Original PR: #17501

Screenshot 2025-10-03 at 4 00 29 PM

@AdamGS
Copy link
Contributor Author

AdamGS commented Oct 3, 2025

there are changes in dfschema.rs that snuck in somehow? I'll fix it.

* Support Decimal32/64 types

* Fix bugs, tests, handle more aggregate functions and schema

* Fill out more parts in expr,common and expr-common

* Some stragglers and overlooked corners

* Actually commit the avg_distinct support

---------

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
@AdamGS AdamGS force-pushed the adamg/backport-17501 branch from 3bb79ae to d8b844d Compare October 3, 2025 20:16
@AdamGS
Copy link
Contributor Author

AdamGS commented Oct 3, 2025

Ok now it seems better, its not a perfect match because some of the changes were based upon #17459, which wasn't backported.

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @AdamGS -- I worry that this is not semver compatible...

If you need this code earlier than Nov maybe we could accelerate the timeline of 51.0.0 but not take everything on main 🤔

/// 64bit float
Float64(Option<f64>),
/// 32bit decimal, using the i32 to represent the decimal, precision scale
Decimal32(Option<i32>, u8, i8),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think 🤔 this is technically a breaking API change I think because ScalarValue is a public enum and not parsed non_exhastive... So we can't backport it to the branch-50 without breaking the API

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes perfect sense, should've noticed it myself.
I can wait until November, sorry about causing so much noise pushing for this release, at least we'll get the docs fix in.

@AdamGS AdamGS closed this Oct 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

common Related to common crate core Core DataFusion crate functions Changes to functions implementation logical-expr Logical plan and expressions proto Related to proto crate sql SQL Planner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants