Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(binder): ExprVisitor and ExprMutator should visit agg_call order_by and filter #5664

Merged
merged 5 commits into from
Sep 30, 2022

Conversation

xiangjinwu
Copy link
Contributor

@xiangjinwu xiangjinwu commented Sep 30, 2022

I hereby agree to the terms of the Singularity Data, Inc. Contributor License Agreement.

What's changed and what's your intention?

As title. By missing these fields during iteration, correlated input refs were not handled and caused panic as mentioned in #4762.

Note that correlated input refs in the filter clause are now correctly detected, but the optimizer rule is not handling it yet. Unlike input or order_by, subexpr in filter are not extracted to a child Project node, thus need special treatment in decorrelation rules. This shall be fixed separately.

Checklist

  • I have written necessary rustdoc comments
  • I have added necessary unit tests and integration tests
  • All checks passed in ./risedev check (or alias, ./risedev c)

Refer to a related PR or issue link (optional)

@github-actions github-actions bot added the type/fix Bug fix label Sep 30, 2022
Comment on lines -1045 to -1055
if select_exprs
.iter()
.chain(group_exprs.iter())
.any(|e| e.has_table_function())
{
return Err(ErrorCode::NotImplemented(
"Table functions in agg call or group by is not supported yet".to_string(),
3814.into(),
)
.into());
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check was redundant and too restrictive:

  • When table function is an input argument of agg call, or part of group by, it is already rejected by the Project node builder.
  • When table function is a sibling of agg call, we are able to handle it.

Refer to planner tests in agg.yaml for examples of both cases above.

@@ -72,13 +72,14 @@
└─BatchValues { rows: [[]] }
- sql: |
select array_cat(array[233], array[array[array[66]]]);
binder_error: "Bind error: unable to find least restrictive type between integer[] and integer[][][]"
binder_error: 'Bind error: unable to find least restrictive type between integer[]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Styling differences generated by ./risedev do-apply-planner-test. Not sure why they were introduced by #5345

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it's because styling difference won't make run-planner-test fail 😇
And the tests are hand-written

@codecov
Copy link

codecov bot commented Sep 30, 2022

Codecov Report

Merging #5664 (596f13d) into main (6ec1948) will decrease coverage by 0.01%.
The diff coverage is 89.51%.

@@            Coverage Diff             @@
##             main    #5664      +/-   ##
==========================================
- Coverage   74.32%   74.31%   -0.02%     
==========================================
  Files         919      921       +2     
  Lines      143957   143990      +33     
==========================================
+ Hits       106999   107000       +1     
- Misses      36958    36990      +32     
Flag Coverage Δ
rust 74.31% <89.51%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/batch/src/rpc/service/task_service.rs 0.00% <0.00%> (ø)
src/batch/src/task/task_execution.rs 51.28% <0.00%> (-4.09%) ⬇️
src/frontend/src/expr/mod.rs 80.32% <ø> (ø)
...rc/frontend/src/optimizer/plan_node/logical_agg.rs 93.78% <ø> (-0.05%) ⬇️
src/frontend/src/optimizer/plan_node/mod.rs 98.27% <ø> (ø)
src/frontend/src/optimizer/rule/index_selection.rs 96.08% <ø> (ø)
src/stream/src/cache/lru_manager.rs 13.33% <0.00%> (-0.81%) ⬇️
src/stream/src/executor/mod.rs 53.39% <ø> (ø)
src/stream/src/executor/top_n/group_top_n.rs 88.91% <ø> (ø)
src/stream/src/executor/top_n/top_n_appendonly.rs 95.70% <ø> (ø)
... and 25 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@xiangjinwu xiangjinwu marked this pull request as ready for review September 30, 2022 08:30
@@ -72,13 +72,14 @@
└─BatchValues { rows: [[]] }
- sql: |
select array_cat(array[233], array[array[array[66]]]);
binder_error: "Bind error: unable to find least restrictive type between integer[] and integer[][][]"
binder_error: 'Bind error: unable to find least restrictive type between integer[]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it's because styling difference won't make run-planner-test fail 😇
And the tests are hand-written

Comment on lines +159 to +174

pub fn order_by(&self) -> &OrderBy {
&self.order_by
}

pub fn order_by_mut(&mut self) -> &mut OrderBy {
&mut self.order_by
}

pub fn filter(&self) -> &Condition {
&self.filter
}

pub fn filter_mut(&mut self) -> &mut Condition {
&mut self.filter
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm ponding whether it would be better to simply publicize all fields 🤪

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was thinking the same, and an alternative to simply expose children_iter/children_mut to return a chained iterator over inputs + order_by + filter. Not sure which is better as of now.

@xxchan
Copy link
Member

xxchan commented Sep 30, 2022

Note that correlated input refs in the filter clause are now correctly detected, but the optimizer rule is not handling it yet. Unlike input or order_by, subexpr in filter are not extracted to a child Project node, thus need special treatment in decorrelation rules. This shall be fixed separately.

Is there a test case/issue for that?

@xiangjinwu
Copy link
Contributor Author

Note that correlated input refs in the filter clause are now correctly detected, but the optimizer rule is not handling it yet. Unlike input or order_by, subexpr in filter are not extracted to a child Project node, thus need special treatment in decorrelation rules. This shall be fixed separately.

Is there a test case/issue for that?

Yes. It is the same issue but in comments #4762 (comment) . For this reason I am not marking it closed by this PR.

@mergify mergify bot merged commit 1178dbe into main Sep 30, 2022
@mergify mergify bot deleted the fix-4762-agg-children-iter branch September 30, 2022 11:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/fix Bug fix
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants