Skip to content

Conversation

@MaxGekk
Copy link
Member

@MaxGekk MaxGekk commented Oct 13, 2022

What changes were proposed in this pull request?

In the PR, I propose to add new error sub-classes of the error class DATATYPE_MISMATCH, and use it in the case of type check failures of some interval expressions.

Why are the changes needed?

Migration onto error classes unifies Spark SQL error messages, and improves search-ability of errors.

Does this PR introduce any user-facing change?

Yes. The PR changes user-facing error messages.

How was this patch tested?

By running the affected test suites:

$ build/sbt "test:testOnly *AnalysisSuite"
$ build/sbt "test:testOnly *ExpressionTypeCheckingSuite"
$ build/sbt "test:testOnly *ApproxCountDistinctForIntervalsSuite"

@MaxGekk MaxGekk changed the title [WIP][SPARK-40760][SQL] Migrate type check failures of interval expressions onto error classes [SPARK-40760][SQL] Migrate type check failures of interval expressions onto error classes Oct 13, 2022
@MaxGekk MaxGekk marked this pull request as ready for review October 13, 2022 16:53
@MaxGekk
Copy link
Member Author

MaxGekk commented Oct 13, 2022

@cloud-fan @itholic Please, take a look at it.

Copy link
Contributor

@amaliujia amaliujia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

},
"WRONG_NUM_ENDPOINTS" : {
"message" : [
"The number of endpoints must be >= 2 to construct intervals."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: maybe also include what is current num but this is minor.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Contributor

@itholic itholic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. LGTM

@MaxGekk
Copy link
Member Author

MaxGekk commented Oct 17, 2022

@cloud-fan Could you approve this PR, please.

@MaxGekk MaxGekk requested a review from cloud-fan October 18, 2022 13:25
"The <functionName> accepts only arrays of pair structs, but <childExpr> is of <childType>."
]
},
"NON_FOLDABLE_ENDPOINT" : {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can NON_FOLDABLE_INPUT be reused?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

},
"UNEXPECTED_ENDPOINT_TYPE" : {
"message" : [
"Endpoints require (numeric or \"TIMESTAMP\" or \"DATE\" or \"TIMESTAMP_NTZ\" or \"INTERVAL YEAR TO MONTH\" or \"INTERVAL DAY TO SECOND\") type."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can UNEXPECTED_INPUT_TYPE be reused?

Or Should we change numeric or \"TIMESTAMP\" or \"DATE\" or \"TIMESTAMP_NTZ\" or \"INTERVAL YEAR TO MONTH\" or \"INTERVAL DAY TO SECOND\" to a placeholder?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

"The <exprName> must be between <valueRange> (current value = <currentValue>)"
]
},
"WRONG_NUM_ENDPOINTS" : {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can WRONG_NUM_PARAMS be reused?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would leave this as is because this is not about wrong number of parameters but about endpoints in a parameter.

Copy link
Contributor

@LuciferYang LuciferYang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM if test pass

@MaxGekk
Copy link
Member Author

MaxGekk commented Oct 22, 2022

@cloud-fan Could you approve the PR, please.

@MaxGekk
Copy link
Member Author

MaxGekk commented Oct 23, 2022

Merging to master. Thank you, @itholic @LuciferYang @amaliujia for review.

@MaxGekk MaxGekk closed this in 625f76d Oct 23, 2022
SandishKumarHN pushed a commit to SandishKumarHN/spark that referenced this pull request Dec 12, 2022
…s onto error classes

### What changes were proposed in this pull request?
In the PR, I propose to add new error sub-classes of the error class `DATATYPE_MISMATCH`, and use it in the case of type check failures of some interval expressions.

### Why are the changes needed?
Migration onto error classes unifies Spark SQL error messages, and improves search-ability of errors.

### Does this PR introduce _any_ user-facing change?
Yes. The PR changes user-facing error messages.

### How was this patch tested?
By running the affected test suites:
```
$ build/sbt "test:testOnly *AnalysisSuite"
$ build/sbt "test:testOnly *ExpressionTypeCheckingSuite"
$ build/sbt "test:testOnly *ApproxCountDistinctForIntervalsSuite"
```

Closes apache#38237 from MaxGekk/type-check-fails-interval-exprs.

Authored-by: Max Gekk <max.gekk@gmail.com>
Signed-off-by: Max Gekk <max.gekk@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants