Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

During expression equality, check for new ordering information #10434

Merged
merged 4 commits into from
May 10, 2024

Conversation

mustafasrepo
Copy link
Contributor

@mustafasrepo mustafasrepo commented May 9, 2024

Which issue does this PR close?

Closes #.

Rationale for this change

See issue #9812 for rationale.

What changes are included in this PR?

When an equality expression is added to the EquivalenceProperties. In light of this information, we can deduce new information about ordering and constantness. This PR adds this support.

With the PR #9813 we have this support, in terms of behaviour. However, current solution doesn't update state and cannot generalize well.

Are these changes tested?

There is an additional TestCase, that showcases increased coverage with this implementation (That test case fails in main branch).

Are there any user-facing changes?

@github-actions github-actions bot added the physical-expr Physical Expressions label May 9, 2024
@mustafasrepo
Copy link
Contributor Author

@suremarc, I would appreciate your review if you have time

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me -- thank you @mustafasrepo

cc @suremarc -- can you also give this a review if you have a chance?

Copy link
Contributor

@ozankabak ozankabak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, left three minor comments

datafusion/physical-expr/src/equivalence/properties.rs Outdated Show resolved Hide resolved
datafusion/physical-expr/src/equivalence/mod.rs Outdated Show resolved Hide resolved
datafusion/physical-expr/src/equivalence/properties.rs Outdated Show resolved Hide resolved
Co-authored-by: Mehmet Ozan Kabak <ozankabak@gmail.com>
@ozankabak
Copy link
Contributor

Given this is a fairly small piece of code, two reviewers already took a look and all tests pass, I will go ahead and merge it. If there is any issue we discover later on, or @suremarc points out something we are missing, we will address with a follow-on PR. Thanks all

@ozankabak ozankabak merged commit 1d4f329 into apache:main May 10, 2024
23 checks passed
@suremarc
Copy link
Contributor

Sorry @alamb @mustafasrepo, I completely missed this. My GH notifications tend to get drowned out by stuff from work, but I've discovered my personal email doesn't have this issue. I will check it every day from now on.

As for this PR, looks great and I'm glad we don't have to declare equivalences backwards (f(a) = b) for the sort optimizations to work. I do think the collapse_monotonic_lex_req was a bit of a smell, so it's nice to move that logic closer into the EquivalenceProperties code 👍

alamb pushed a commit that referenced this pull request Jul 10, 2024
* discover new orderings when constants are added

* more comments

* reduce nesting + describe argument

* lint?
Lordworms pushed a commit to Lordworms/arrow-datafusion that referenced this pull request Jul 12, 2024
* discover new orderings when constants are added

* more comments

* reduce nesting + describe argument

* lint?
appletreeisyellow pushed a commit to influxdata/arrow-datafusion that referenced this pull request Jul 12, 2024
* discover new orderings when constants are added

* more comments

* reduce nesting + describe argument

* lint?
findepi pushed a commit to findepi/datafusion that referenced this pull request Jul 16, 2024
…e#10434)

* Move ordering discovery to equality check

* Minor changes

* Minor changes

* Apply suggestions from code review

Co-authored-by: Mehmet Ozan Kabak <ozankabak@gmail.com>

---------

Co-authored-by: Mehmet Ozan Kabak <ozankabak@gmail.com>
findepi pushed a commit to findepi/datafusion that referenced this pull request Jul 16, 2024
* discover new orderings when constants are added

* more comments

* reduce nesting + describe argument

* lint?
xinlifoobar pushed a commit to xinlifoobar/datafusion that referenced this pull request Jul 18, 2024
* discover new orderings when constants are added

* more comments

* reduce nesting + describe argument

* lint?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
physical-expr Physical Expressions
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants