Skip to content

Commit

Permalink
[SPARK-42259][SQL] ResolveGroupingAnalytics should take care of Pytho…
Browse files Browse the repository at this point in the history
…n UDAF

This is a long-standing correctness issue with Python UDAF and grouping analytics. The rule `ResolveGroupingAnalytics` should take care of Python UDAF when matching aggregate expressions.

bug fix

Yes, the query result was wrong before

existing tests

Closes #39824 from cloud-fan/python.

Authored-by: Wenchen Fan <wenchen@databricks.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
(cherry picked from commit 1219c84)
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
  • Loading branch information
cloud-fan committed Feb 1, 2023
1 parent 0bb8f22 commit 80e8df1
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -617,7 +617,7 @@ class Analyzer(override val catalogManager: CatalogManager)
// AggregateExpression should be computed on the unmodified value of its argument
// expressions, so we should not replace any references to grouping expression
// inside it.
case e: AggregateExpression =>
case e if AggregateExpression.isAggregate(e) =>
aggsBuffer += e
e
case e if isPartOfAggregation(e) => e
Expand Down

0 comments on commit 80e8df1

Please sign in to comment.