Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

always expect exists filter #480

Merged
merged 8 commits into from
Nov 27, 2023
Merged

always expect exists filter #480

merged 8 commits into from
Nov 27, 2023

Conversation

ppcad
Copy link
Collaborator

@ppcad ppcad commented Nov 24, 2023

See issue #435

@ppcad ppcad self-assigned this Nov 24, 2023
@codecov-commenter
Copy link

codecov-commenter commented Nov 24, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (426b244) 92.24% compared to head (26a0221) 92.34%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #480      +/-   ##
==========================================
+ Coverage   92.24%   92.34%   +0.09%     
==========================================
  Files         130      130              
  Lines        9486     9492       +6     
==========================================
+ Hits         8750     8765      +15     
+ Misses        736      727       -9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ppcad ppcad requested a review from dtrai2 November 24, 2023 07:52
@ppcad ppcad marked this pull request as ready for review November 24, 2023 07:53
@ppcad ppcad changed the title Dev always expect exists filter Always expect exists filter Nov 24, 2023
@ppcad ppcad changed the title Always expect exists filter always expect exists filter Nov 24, 2023
@@ -126,15 +130,36 @@ def _add_exists_filter(parsed_rules: list):
for parsed_rule in parsed_rules:
temp_parsed_rule = parsed_rule.copy()
skipped_counter = 0

for segment_index, segment in enumerate(temp_parsed_rule):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you maybe flatten the if-statements in this loop? That way it is a bit easier to understand. A suggestion cold look like this:

  for segment_idx, segment in enumerate(temp_parsed_rule):
      if isinstance(segment, (Exists, Always)):
          skipped_counter += 1
          continue
      exists_filter = RuleParser._get_exists_filter(segment)
      if exists_filter is None:
          skipped_counter += 1
          continue
      if exists_filter in filter_expression:
          skipped_counter += 1
          continue
      filter_expression.insert(segment_idx * 2 - skipped_counter, exists_filter)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've changed it.

skipped_counter += 1
continue
else:
parsed_rule.insert(segment_idx * 2 - skipped_counter, exists_filter)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain where the 2 is coming from? I was trying to understand this and my first idea was that segment_idx - 1 should work as well, but it doesn't. Maybe you could give this magic number a meaningful name?

Copy link
Collaborator Author

@ppcad ppcad Nov 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was a way to account for the new elements added to the list.

segment_idx * 2 - skipped_counter
<=> segment_idx + (segment_idx - skipped_counter)
<=> segment_idx + added_exists_filter_count

It was indeed confusing, so I changed it.

Comment on lines 159 to 165
if isinstance(segment, Not):
not_child = segment.children[0]
if isinstance(not_child, KeyValueBasedFilterExpression):
return Exists(not_child.key)
elif isinstance(segment, KeyValueBasedFilterExpression):
return Exists(segment.key)
return None
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about we use this implementation? It's a bit easier to read and understand by preventing nested if-statements.

Suggested change
if isinstance(segment, Not):
not_child = segment.children[0]
if isinstance(not_child, KeyValueBasedFilterExpression):
return Exists(not_child.key)
elif isinstance(segment, KeyValueBasedFilterExpression):
return Exists(segment.key)
return None
if isinstance(segment, Not):
segment = segment.children[0]
if isinstance(segment, KeyValueBasedFilterExpression):
return Exists(segment.key)
return None

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestion. I've changed it.

@ppcad ppcad requested a review from dtrai2 November 27, 2023 06:46
Copy link
Collaborator

@dtrai2 dtrai2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah thanks for the changes, now the inserting is a bit easier to understand 👍

@ppcad ppcad merged commit 199200c into main Nov 27, 2023
ppcad added a commit that referenced this pull request Jan 29, 2024
ppcad added a commit that referenced this pull request Feb 1, 2024
…expression (#515)

* Revert not exists changes from #480, but keep refactorings in filter_expression

* Re-add refactoring of rule_parser

* Update changelog
@ekneg54 ekneg54 deleted the dev-always-expect-exists-filter branch February 11, 2025 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Predetector doesn't check for existing field when using expressions set in filter
3 participants