Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Like, NotLike expressions work with literal NULL #2627

Merged
merged 2 commits into from
May 27, 2022

Conversation

WinkerDu
Copy link
Contributor

Which issue does this PR close?

Closes #2626 .

Rationale for this change

like, not like expressions doesn't work well with literal NULL in DF.

To Reproduce

> SELECT column1 like NULL from (values('a'), ('b'), (NULL)) as t
Plan("'Utf8 LIKE Null' can't be evaluated because there isn't a common type to coerce the types to")

Postgres works like

# SELECT column1 like NULL from (values('a'), ('b'), (NULL)) as t;
 ?column? 
----------
 
 
 
(3 rows)

What changes are included in this PR?

  • Introduces null_coercion to like_coercion
  • Enhances compute_utf8_op_scalar to produce null array when scalar value NULL inputs.

Are there any user-facing changes?

No.

Does this PR break compatibility with Ballista?

No.

@github-actions github-actions bot added core Core DataFusion crate datafusion Changes in the datafusion crate logical-expr Logical plan and expressions physical-expr Physical Expressions labels May 26, 2022
@WinkerDu
Copy link
Contributor Author

cc @andygrove @alamb @yjshen , Please have a review, thank you ❤️

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me -- thank you @WinkerDu . I do wonder if this will work with NULL like <col> but we can also fix that case in a follow on PR if you would prefer

#[tokio::test]
async fn like_nlike_with_null_lt() {
let ctx = SessionContext::new();
let sql = "SELECT column1 like NULL from (values('a'), ('b'), (NULL)) as t";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also add a test when the arguments are reversed? For example,:

Suggested change
let sql = "SELECT column1 like NULL from (values('a'), ('b'), (NULL)) as t";
let sql = "SELECT column1 like NULL as col_null, NULL like column1 as null_col from (values('a'), ('b'), (NULL)) as t";

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alamb Sounds reasonable, I've changed the ut, thank you

];
assert_batches_eq!(expected, &actual);

let sql = "SELECT column1 not like NULL from (values('a'), ('b'), (NULL)) as t";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
let sql = "SELECT column1 not like NULL from (values('a'), ('b'), (NULL)) as t";
let sql = "SELECT column1 not like NULL as col_null, NULL not like column as null_col from (values('a'), ('b'), (NULL)) as t";

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @WinkerDu

@alamb alamb merged commit 7b7edf9 into apache:master May 27, 2022
ovr pushed a commit to cube-js/arrow-datafusion that referenced this pull request Aug 15, 2022
* like / not like work with NULL

* add more ut
ovr pushed a commit to cube-js/arrow-datafusion that referenced this pull request Aug 15, 2022
* like / not like work with NULL

* add more ut
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Core DataFusion crate datafusion Changes in the datafusion crate logical-expr Logical plan and expressions physical-expr Physical Expressions
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Like, NotLike expressions work with literal NULL
2 participants