-
Notifications
You must be signed in to change notification settings - Fork 29k
[SPARK-49913][SQL] Add check for unique label names in nested labeled scopes #48795
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-49913][SQL] Add check for unique label names in nested labeled scopes #48795
Conversation
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/SqlScriptingParserSuite.scala
Show resolved
Hide resolved
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParserUtils.scala
Outdated
Show resolved
Hide resolved
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParserUtils.scala
Show resolved
Hide resolved
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParserUtils.scala
Outdated
Show resolved
Hide resolved
davidm-db
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please make seenLabels thread local as Wenchen suggested, other than that LGTM.
|
|
||
| override def visitSingleCompoundStatement(ctx: SingleCompoundStatementContext): CompoundBody = { | ||
| visit(ctx.beginEndCompoundBlock()).asInstanceOf[CompoundBody] | ||
| val seenLabels = Set[String]() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes this is the implementation I was expecting. Can we do a bit more encapsulation? e.g.
val labelCtx = new ScritpingLabelContext()
visitBeginEndCompoundBlockImpl...
...
class ScritpingLabelContext {
private val seenLabels = Set[String]()
def newLabel(label: String)...
...
}
``
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@cloud-fan done.
davidm-db
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just a tiny comment, otherwise LGTM.
I would maybe use SqlScriptingLabelContext since we used SqlScripting* I think everywhere, so just to keep it consistent.
|
The linter failure is unrelated, thanks, merging to master! |
What changes were proposed in this pull request?
We are introducing checks for unique label names.
New rules for label names:
Valid code:
Invalid code:
Design explanation:
Even though there are Listeners with
enterRuleandexitRulemethods to check labels before and remove them fromseenLabelsafter visiting node, we favor this approach because minimal changes were needed and code is more compact to avoid dependency issues.Additionally, generating label text would need to be done in 2 places and we wanted to avoid duplicated logic:
enterRulevisitRuleWhy are the changes needed?
It will be needed in future when we release Local Scoped Variables for SQL Scripting so users can target variables from outer scopes if they are shadowed.
How was this patch tested?
New unit tests in 'SqlScriptingParserSuite.scala'.
Was this patch authored or co-authored using generative AI tooling?
No.