-
Notifications
You must be signed in to change notification settings - Fork 24.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SQL: Pushdown WHERE clause inside subqueries #71362
Conversation
Pinging @elastic/es-ql (Team:QL) |
new SimplifyComparisonsArithmetics(DataTypes::areCompatible), | ||
// prune/elimination | ||
new PruneFilters(), | ||
new PruneLiteralsInOrderBy(), | ||
new PruneCast(), | ||
new CombineLimits()); | ||
new CombineLimits(), | ||
new PushDownAndCombineFilters() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Moved the rule here to be consistent across EQL and SQL.
@@ -189,25 +190,6 @@ protected LogicalPlan rule(Filter filter) { | |||
} | |||
} | |||
|
|||
static class PushDownAndCombineFilters extends OptimizerRule<Filter> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Extracted this rule from EQL inside QL project.
3c582a5
to
c161ba1
Compare
Push down filters inside subqueries, even when dealing with aggregates. The rule already existed however it was not being used inside SQL. When dealing with Aggregates, keep the aggregate functions in place but try and push down conjunctions on non-aggregates.
c161ba1
to
9a6d322
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good. Another very nice improvement for our subquery support!
I'd like to ask a few more tests with more complex expressions, e.g.:
group by with function:
SELECT * FROM (
SELECT languages + 1 AS lan, MAX(salary) AS max FROM test_emp WHERE languages > 1 GROUP BY lan ORDER BY 1
)
WHERE (lan - 1) > 2
aggregate with function:
SELECT * FROM (
SELECT gender, MAX(salary) / 100 AS max FROM test_emp WHERE languages > 1 GROUP BY gender ORDER BY gender
)
WHERE max > 745 AND gender IS NOT NULL
and a combination where we have both complex agg and group by:
SELECT * FROM (
SELECT lan - 1 AS lan, max / 2 AS max FROM (
SELECT languages + 1 AS lan, MAX(salary) / 10 AS max FROM test_emp WHERE languages > 1 GROUP BY lan ORDER BY 1
)
)
WHERE (lan - 1) > 1 AND max - 1000 > 2500
The query since it's a conjunction and thus the grouping expression is pushed down. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Thank you!!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Push down filters inside subqueries, even when dealing with aggregates. The rule already existed however it was not being used inside SQL. When dealing with Aggregates, keep the aggregate functions in place but try and push down conjunctions on non-aggregates.
Push down filters inside subqueries, even when dealing with aggregates.
The rule already existed however it was not being used inside SQL.
When dealing with Aggregates, keep the aggregate functions in place but
try and push down conjunctions on non-aggregates.