-
Notifications
You must be signed in to change notification settings - Fork 81
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add additional structuring to Filter, Expression, and Literal #3441
Add additional structuring to Filter, Expression, and Literal #3441
Conversation
engine/table/src/main/java/io/deephaven/engine/table/impl/select/WhereFilter.java
Outdated
Show resolved
Hide resolved
engine/table/src/main/java/io/deephaven/engine/table/impl/select/CharRangeFilter.java
Outdated
Show resolved
Hide resolved
|
||
public abstract Expression expression(); | ||
|
||
public abstract List<Literal> values(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
RHS of a match filter can be a QueryScope
param. Literal
is insufficiently inclusive.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay - I'll expand this type b/c I agree. That said, there is room for a QueryScope extends Expression
type in the future that we don't currently have.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You need to support FilterMatches
in some way, but it could be via equals, in, and case-insensitive String-specific variants of each (4 total).
I still think you need param support.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should follow-up later w/ #3784
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And #3785
return ImmutableFilterMatches.builder(); | ||
} | ||
|
||
public abstract Expression expression(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Expression
seems too general. Why not ColumnName
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We're trying to be as generic enough to capture the semantics that one could reasonably want to express (or express via SQL), even though our engine might not support it for now.
As a quasi example in our parser-like syntax:
Foo + Bar in "foobar"
engine/table/src/main/java/io/deephaven/engine/table/impl/select/MatchFilter.java
Outdated
Show resolved
Hide resolved
engine/table/src/main/java/io/deephaven/engine/table/impl/select/WhereFilterAdapter.java
Show resolved
Hide resolved
engine/table/src/main/java/io/deephaven/engine/table/impl/select/WhereFilterPatternImpl.java
Outdated
Show resolved
Hide resolved
engine/table/src/main/java/io/deephaven/engine/table/impl/select/WhereFilterPatternImpl.java
Outdated
Show resolved
Hide resolved
engine/table/src/main/java/io/deephaven/engine/table/impl/select/WhereFilterPatternImpl.java
Outdated
Show resolved
Hide resolved
engine/table/src/main/java/io/deephaven/engine/table/impl/select/WhereFilterAdapter.java
Outdated
Show resolved
Hide resolved
engine/table/src/main/java/io/deephaven/engine/table/impl/select/WhereFilterAdapter.java
Outdated
Show resolved
Hide resolved
engine/table/src/main/java/io/deephaven/engine/table/impl/select/WhereFilterAdapter.java
Outdated
Show resolved
Hide resolved
engine/table/src/main/java/io/deephaven/engine/table/impl/select/MatchFilter.java
Outdated
Show resolved
Hide resolved
inverted); | ||
} | ||
// Delegate to comparisons | ||
return of(inverted ? in.inverseAsComparisons() : in.asComparisons()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We want to go to a ConditionFilter in this case; if we do a disjunction, we'll load the inputs N times.
Labels indicate documentation is required. Issues for documentation have been opened: How-to: https://github.com/deephaven/deephaven.io/issues/2563 |
The interfaces of
io.deephaven.api.expression.Filter
,io.deephaven.api.expression.Expression
, andio.deephaven.api.value.Value
were originally developed and influenced based on the structure of the gRPC messages and engine capabilities; and they were explicitly called out due to lack of completeness with respect to these constructions in #829, #830, and #831.Upon delving into SQL work in #3473, it became clear that it was time to update these interfaces with the more expressive structures that could better preserve the semantics and intentions of the SQL expressions. The alternative involves throwing up our hands and using
io.deephaven.api.RawString
everywhere instead.For example, it was possible before to represent the filters of
Foo > Bar
andFoo > 42
, but it was not possible to representFoo > Bar + 42
(again, without dropping down intoRawString
); and equivalently, anExpression
could beBar
, or42
, but notBar + 42
(essentially, limiting what could be represented in the case of view or updateSelectable
).This PR greatly expands the expressiveness of these interfaces while also cleaning up the hierarchy. The main highlights:
Filter
is now anExpression
FilterCondition
renamedFilterComparison
Value
is gone; in its place, a more specificLiteral
structureValue
replaced with the more genericExpression
type (for example, LHS / RHS ofFilterComparison
are nowExpression
s instead ofValue
s)ExpressionFunction
created to represent function calls without needing to drop intoRawString
(for example, we might want to haveio.deephaven.sql.Functions#sum_sql
as something that can be properly referenced).This PR does not aim to update the gRPC layer at this time, mainly do to time considerations and that #3473 is limited to execution completely within the server environment. That said, we would likely benefit from updating the gRPC layer to be more expressive in the ways proposed here.
There's also potential in the future for the engine layer to benefit from the additional structuring here; potentially constructing more advanced filters and expressions without needing to compile strings.