Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expression: Part of the expression is ignored when multiple and/or expressions are specified #65

Merged
merged 1 commit into from
Oct 13, 2023

Conversation

amogh-jahagirdar
Copy link
Contributor

Fixes #64 . If there are multiple and/or conditions currently, our expression parser will ignore anything after the second predicate. This change fixes the issue by forwarding the remaining predicates as the argument to *rest: BooleanExpression for And and Or.

@Fokko Fokko added this to the PyIceberg 0.6.0 release milestone Oct 13, 2023
Comment on lines 236 to 238
if len(result[0]) > 2:
return And(result[0][0], result[0][1], *result[0][2:])
return And(result[0][0], result[0][1])
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uh maybe not the most pythonic...I guess a ternary which sets the *args option if it's > 2 otherwise AlwaysTrue (for And) and then for Or would be AlwaysFalse so it just considers the first 2.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realized I can just pass in *result[0]

@Fokko
Copy link
Contributor

Fokko commented Oct 13, 2023

This is a serious one, thanks for reporting @puchengy

@amogh-jahagirdar amogh-jahagirdar force-pushed the python-fix-multiple-and-or branch from a0db070 to 4c2c1af Compare October 13, 2023 11:29
@amogh-jahagirdar amogh-jahagirdar force-pushed the python-fix-multiple-and-or branch from 4c2c1af to 3c0d80e Compare October 13, 2023 11:33
@amogh-jahagirdar
Copy link
Contributor Author

Yes thanks @puchengy for reporting this, please also take a look a this fix when you get a chance!

@puchengy
Copy link
Contributor

@amogh-jahagirdar The test result LGTM. However, I am not familiar with the implementation, please feel free to go ahead and merge.

@@ -233,11 +233,11 @@ def handle_not(result: ParseResults) -> Not:


def handle_and(result: ParseResults) -> And:
return And(result[0][0], result[0][1])
return And(*result[0])
Copy link
Contributor

@rdblue rdblue Oct 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does the parser produce more than 2 child expressions?

Is this a result of using infix_notation that I didn't realize at the time?

@rdblue
Copy link
Contributor

rdblue commented Oct 13, 2023

Thanks for jumping on the fix, @amogh-jahagirdar!

@Fokko, should we release 0.5.1 with this patch?

@Fokko
Copy link
Contributor

Fokko commented Oct 13, 2023

@Fokko, should we release 0.5.1 with this patch?

Yes, I think that's a good idea

@Fokko Fokko merged commit cb9b4f1 into apache:main Oct 13, 2023
Fokko pushed a commit to Fokko/iceberg-python that referenced this pull request Oct 15, 2023
@Fokko Fokko changed the title Expression: Fix for when multiple and/or expressions are specified via string Expression: Part of the expression is ignored when multiple and/or expressions are specified Oct 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] string row filter ignore 2nd (and onwards) And
4 participants