-
Notifications
You must be signed in to change notification settings - Fork 284
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
query_key value is a string instead of a dict #340
Comments
I can submit a pull request if you're agree with the problem. |
This bug is especially annoying when we use match_enhancement because depending of the rule type, match data does not have the same data structure. |
This could break existing behavior for those who have written rules that rely upon the string format |
I understand the problematic but we can't put both version of a field (string and dict) into the same doc. Elasticsearch accepts only one because it converts automatically |
If that's the case then the we will have to identify this as a breaking change in the next release. Long term, I think making all rule types store match data consistently is the right direction. |
[+] add test for expend_string_into_dict [+] add check in test_metric_match and test_percentage_match
#340 fix match["query_key"] malformation
Behavior
Method check_matches in ruletypes.py appends the query key as a string.
So match is a dict composed of simple type like string or integer.
Rules types impacted:
On another side, all other rules create match from documents content. The method involved is add_data(self, data)
Example
Suppose that we store documents like this one in the index:
{'@timestamp': value, metadata': { 'ip': value} }
If we use a frequency rule on this index , the match generated will have this template :
[{**'metadata': {'ip**': "10.0.0.1"}, '@timestamp': datetime.datetime(20...o=tzutc()), '@version': '1', '_id': 'tR46i3oBTsznpu6_lwkh', '_index': 'index', '_type': '_doc'}]
If we use a metric rule, the match generated will have this template :
[{**'metadata.ip**': '10.0.0.1' , '@timestamp': datetime.datetime(20...o=tzutc())'}]
Possible Solution
Fix check_matches by converting query_key value=
"string1.string2.stringN"
to["string1']["string2]['stringN"]
The text was updated successfully, but these errors were encountered: