Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimized the exact filter performance #7311

Merged
merged 1 commit into from
Oct 6, 2023

Conversation

Cali0707
Copy link
Member

Fixes #7307

Currently, we use the attributes filter internally when running the exact filter. However, the attributes filter supports cases which the exact filter does not allow. Specifically, the current implementation of the exact filter requires that all filters have a non-empty string value for both the attribute and the desired value, while the attributes filter checks if the desired value is "", and treats that as a special "allow anything" case. By rewriting the exact filter with these more strict requirements, we see a speed up in terms of performance.

To see the current creation-time validation logic, look here:

func NewExactFilter(filters map[string]string) (eventfilter.Filter, error) {
for attribute, value := range filters {
if attribute == "" || value == "" {
return nil, fmt.Errorf("invalid arguments, attribute and value can't be empty")
}
}

Proposed Changes

  • Instead of using the attributes filter internally, just handle the filtering logic directly

With these changes, the performance difference is:

benchmark                                                                                          old ns/op     new ns/op     delta
BenchmarkExactFilter/Creation:_Pass_with_exact_match_of_id-8                                       81.4          53.4          -34.38%
BenchmarkExactFilter/Run:_Pass_with_exact_match_of_id-8                                            502           425           -15.31%
BenchmarkExactFilter/Creation:_Pass_with_exact_match_of_all_context_attributes_(except_time)-8     109           89.1          -18.44%
BenchmarkExactFilter/Run:_Pass_with_exact_match_of_all_context_attributes_(except_time)-8          1441          1326          -7.98%
BenchmarkExactFilter/Creation:_No_pass_with_exact_match_of_id_and_source-8                         84.7          61.8          -27.04%
BenchmarkExactFilter/Run:_No_pass_with_exact_match_of_id_and_source-8                              762           727           -4.50%

benchmark                                                                                          old allocs     new allocs     delta
BenchmarkExactFilter/Creation:_Pass_with_exact_match_of_id-8                                       1              1              +0.00%
BenchmarkExactFilter/Run:_Pass_with_exact_match_of_id-8                                            4              4              +0.00%
BenchmarkExactFilter/Creation:_Pass_with_exact_match_of_all_context_attributes_(except_time)-8     1              1              +0.00%
BenchmarkExactFilter/Run:_Pass_with_exact_match_of_all_context_attributes_(except_time)-8          15             15             +0.00%
BenchmarkExactFilter/Creation:_No_pass_with_exact_match_of_id_and_source-8                         1              1              +0.00%
BenchmarkExactFilter/Run:_No_pass_with_exact_match_of_id_and_source-8                              7              7              +0.00%

benchmark                                                                                          old bytes     new bytes     delta
BenchmarkExactFilter/Creation:_Pass_with_exact_match_of_id-8                                       24            8             -66.67%
BenchmarkExactFilter/Run:_Pass_with_exact_match_of_id-8                                            208           208           +0.00%
BenchmarkExactFilter/Creation:_Pass_with_exact_match_of_all_context_attributes_(except_time)-8     24            8             -66.67%
BenchmarkExactFilter/Run:_Pass_with_exact_match_of_all_context_attributes_(except_time)-8          448           448           +0.00%
BenchmarkExactFilter/Creation:_No_pass_with_exact_match_of_id_and_source-8                         24            8             -66.67%
BenchmarkExactFilter/Run:_No_pass_with_exact_match_of_id_and_source-8                              410           410           +0.00%

Pre-review Checklist

  • At least 80% unit test coverage
  • E2E tests for any new behavior
  • Docs PR for any user-facing impact
  • Spec PR for any new API feature
  • Conformance test for any change to the spec

Release Note

The exact filter now uses less memory and is faster!

Docs

Signed-off-by: Calum Murray <cmurray@redhat.com>
@knative-prow knative-prow bot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Sep 27, 2023
@knative-prow knative-prow bot requested review from aslom and creydr September 27, 2023 19:34
@Cali0707
Copy link
Member Author

/cc @Leo6Leo @creydr @pierDipi

@knative-prow knative-prow bot requested review from Leo6Leo and pierDipi September 27, 2023 19:34
@codecov
Copy link

codecov bot commented Sep 27, 2023

Codecov Report

Attention: 3 lines in your changes are missing coverage. Please review.

Comparison is base (ff9444d) 77.66% compared to head (42d850a) 77.63%.
Report is 16 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7311      +/-   ##
==========================================
- Coverage   77.66%   77.63%   -0.04%     
==========================================
  Files         250      250              
  Lines       13436    13444       +8     
==========================================
+ Hits        10435    10437       +2     
- Misses       2478     2482       +4     
- Partials      523      525       +2     
Files Coverage Δ
pkg/apis/duck/v1/subscribable_types.go 100.00% <ø> (ø)
pkg/apis/eventing/v1/trigger_types.go 100.00% <ø> (ø)
pkg/apis/flows/v1/parallel_types.go 100.00% <ø> (ø)
pkg/apis/flows/v1/sequence_types.go 100.00% <ø> (ø)
pkg/apis/messaging/v1/subscription_types.go 66.66% <ø> (ø)
pkg/eventfilter/subscriptionsapi/exact_filter.go 66.66% <80.00%> (+12.12%) ⬆️

... and 2 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Leo6Leo
Copy link
Member

Leo6Leo commented Oct 2, 2023

/lgtm

@knative-prow knative-prow bot added the lgtm Indicates that a PR is ready to be merged. label Oct 2, 2023
@Cali0707
Copy link
Member Author

Cali0707 commented Oct 4, 2023

/cc @matzew

@knative-prow knative-prow bot requested a review from matzew October 4, 2023 13:27
Copy link
Member

@pierDipi pierDipi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

@knative-prow
Copy link

knative-prow bot commented Oct 6, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Cali0707, pierDipi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@knative-prow knative-prow bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 6, 2023
@knative-prow knative-prow bot merged commit 7b3afa0 into knative:main Oct 6, 2023
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

New Event Filtering: Improve Exact filter performance
3 participants