Add python filters is_null, is_not_null, and Filter.not_() #3736
Conversation
devinrsmith
added
python
NoDocumentationNeeded
NoReleaseNotesNeeded
devinrsmith
requested review from
chipkent,
jmao-denver and
rcaudy
as code owners
py/server/deephaven/filters.py
Outdated
| def is_null(col: str) -> Filter: | ||
| """Creates a new filter that evaluates to true when the col is null, and evaluates to false when col is not null. | ||
|
|
||
| Args: | ||
| col (str): the column name | ||
|
|
||
| Returns: | ||
| a new is-null Filter | ||
| """ |
There was a problem hiding this comment.
Would Filter.from_("X == null") be equivalent to 'is_null("X")?
There was a problem hiding this comment.
It's logically the same, and will produce the same looking results, but the engine will do different things.
Filter.from_("X == null") will cause java to compile a filter kernel (ConditionFilter), whereas is_null("X") is able to construct a much more efficient MatchFilter.
There was a problem hiding this comment.
The more idiomatic way to express this via string would be from_("isNull(X)"), but that still results in ConditionFilter.
There was a problem hiding this comment.
Currently we give the users two ways to specify filter conditions: the string condition expressions and the Filter objects. Since Filters are much more efficient, it makes sense to provide ways to construct Filters that represent all sorts of conditions. We could also improve the filter parser to automatically translate a string condition expression (which is more intuitive to read) into a Filter object. @chipkent your thoughts?
There was a problem hiding this comment.
"MatchFilter" is not a regex construction, it's a "match from this list of values" construction.
There was a problem hiding this comment.
We could and should be doing more smart things in how we interpret string filters. As long as we have the option to do the optimization in the future, we haven't painted ourselves into a corner, which is good.
devinrsmith
changed the title
No description provided.