-
Notifications
You must be signed in to change notification settings - Fork 583
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for INTERVAL inside window frames #655
Conversation
src/ast/mod.rs
Outdated
@@ -880,9 +880,9 @@ pub enum WindowFrameBound { | |||
/// `CURRENT ROW` | |||
CurrentRow, | |||
/// `<N> PRECEDING` or `UNBOUNDED PRECEDING` | |||
Preceding(Option<u64>), | |||
Preceding(Option<RangeBounds>), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems like this in general can be any Expression (not just a Number of Interval):
https://www.postgresql.org/docs/current/sql-expressions.html
In the offset PRECEDING and offset FOLLOWING frame options, the offset must be an expression not containing any variables, aggregate functions, or window functions. The meaning of the offset depends on the frame mode:
What do you think about just parsing it as an Expression rather than special casing Number / Interval?
Thank you @mustafasrepo for the contribution |
Pull Request Test Coverage Report for Build 3250493085
💛 - Coveralls |
Hi Andrew! The SQL:2016 reference seems to expect an unsigned literal, that's why we opted for this. But you are right, PostgreSQL seems to allow expressions -- it only requires such expressions to evaluate to appropriate unsigned values. We can certainly follow the latter approach, but doing that will require both sqlparser and datafusion to accommodate that logic. It seems to me a good approach could be two-tiered: We can implement the unsigned-value-only design first (both here and in datafusion), and then subsequently work towards PSQL compatibility. What do you think? |
That is fine with me -- I was thinking that using a general Expr in the parser at first would actually make the process easier overall -- the parser / parsed structs would be simpler and DataFusion would have to special case what types it handled. Then over time datafusion could be extended to handle more and more types, but the parser wouldn't need to be changed again |
Hi Andrew, we have changed the implementation as you suggested. It improves readability and decreases the changes required. Thanks for the suggestion. In this PR https://github.com/synnada-ai/arrow-datafusion/pull/5 you can see the corresponding datafusion change as POC implementation to support non-u64 values inside window frames. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Love it -- thank you @mustafasrepo
Support for INTERVAL Inside Window Frames
Which issue does this PR close?
Closes #631 by adding
INTERVAL
support inside window frames. Now we can parse queries such asRationale for this change
Datafusion now supports window frame queries with PR #3570. However, since window frame bound only emits
u64
values, we cannot run queries like the one above in Datafusion. If this PR merges, after some minor changes we would be able to runRANGE
queries involvingINTERVAL
in Datafusion. The roadmap of Datafusion includes a plan to support different types forWindowFrameBound
.