Skip to content

Conversation

Dimchikkk
Copy link
Contributor

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@github-actions github-actions bot added the sql SQL Planner label Jun 19, 2025
@Dimchikkk Dimchikkk marked this pull request as draft June 19, 2025 12:58
@Dimchikkk
Copy link
Contributor Author

(converted to draft while I'm investigating CI failures)

@github-actions github-actions bot added the sqllogictest SQL Logic Tests (.slt) label Jun 20, 2025
@Dimchikkk
Copy link
Contributor Author

(one of the test failures is related to #1898 and would be fixed if that patch is backported to 0.56)

@Dimchikkk Dimchikkk marked this pull request as ready for review June 28, 2025 18:52
@Dimchikkk Dimchikkk changed the title chore(deps): Update sqlparser to 0.56.0 chore(deps): Update sqlparser to 0.56 Jun 28, 2025
@alamb
Copy link
Contributor

alamb commented Jul 24, 2025

0.58.0 is released: apache/datafusion-sqlparser-rs#1886 (comment)

@Jefffrey
Copy link
Contributor

@Dimchikkk would you be able to pick up the work involving upgrade to 0.58 as mentioned? Otherwise I can build upon your work here to get this PR over the line 🙂

@Dimchikkk
Copy link
Contributor Author

@Jefffrey yeah, you can take it.

@Jefffrey Jefffrey marked this pull request as draft August 16, 2025 02:27
@Jefffrey Jefffrey changed the title chore(deps): Update sqlparser to 0.56 chore(deps): Update sqlparser to 0.58 Aug 16, 2025
@Jefffrey
Copy link
Contributor

Moving to draft as I work to build upon this to update to 0.58

@Jefffrey Jefffrey marked this pull request as ready for review August 18, 2025 07:29
@Jefffrey
Copy link
Contributor

PR good for review now

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @Jefffrey and @Dimchikkk . This is pretty epic

The only thing I am worried about is the direct use of "substr" string -- I left an alternate suggestion. Let me know what you think

not_impl_err!(
"Substring not supported by UserDefinedExtensionPlanners: {substring_args:?}"
)
let fun = self
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This basically hard codes the use of a function named "substr" into the planner

I think the preferred way is to use the ExprPlanner API, similarly to how plan field access is done here:

fn plan_field_access(

Copy link
Contributor

@Jefffrey Jefffrey Aug 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not familiar with that API, though it seems to be used above already?

for planner in self.context_provider.get_expr_planners() {
match planner.plan_substring(substring_args)? {
PlannerResult::Planned(expr) => return Ok(expr),
PlannerResult::Original(args) => {
substring_args = args;
}
}
}

So this code here (that hardcodes the "substr" string) is more of a fallback I guess? Might need some help understanding this part

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I spent some time debugging and I think the solution is that we needed to register the appropriate planner with the tests. I pushed a commit here: e46e532

Note I also found the name of the extension planner very confusing, and I will make a separate PR to fix it

Copy link
Contributor

@Jefffrey Jefffrey Aug 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense to me, yeah. Was confusing to have that fallback behaviour 😅

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it looks good now, though of course I am biased!

@Jefffrey let me know what you think

Also, thank you @Dimchikkk for starting this -- I would say it is a good team effort!

Copy link
Contributor

@crepererum crepererum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you

@Dimchikkk
Copy link
Contributor Author

@Jefffrey thanks for helping get this over the line.

@Jefffrey Jefffrey merged commit 8f30ac2 into apache:main Aug 20, 2025
29 checks passed
@Jefffrey
Copy link
Contributor

Thanks @Dimchikkk & all

@alamb
Copy link
Contributor

alamb commented Aug 20, 2025

🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

sql SQL Planner sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Upgrade to sqlparser 0.58.0

4 participants