-
Notifications
You must be signed in to change notification settings - Fork 509
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cleanup rule survey code #1923
Cleanup rule survey code #1923
Conversation
There are a few linting errors to resolve |
3c2c3a1
to
739762d
Compare
Can you link an example of previous rule refactoring that highlights the break? |
@brokensound77 @terrancedejesus Can we setup time together to walk through the changes in |
|
The elastic_transport.ConnectionTimeout: Connection timed out
Exception ignored in: <function Kibana.__del__ at 0x110d2edd0>
Traceback (most recent call last):
File "/Users/stryker/workspace/Community/brokensound77/detection-rules/kibana/connector.py", line 184, in __del__
File "/Users/stryker/workspace/Community/brokensound77/detection-rules/kibana/connector.py", line 173, in logout
File "/Users/stryker/workspace/Community/brokensound77/detection-rules/kibana/connector.py", line 103, in get
File "/Users/stryker/workspace/Community/brokensound77/detection-rules/kibana/connector.py", line 87, in request
File "/Users/stryker/.virtualenvs/detection_dev/lib/python3.10/site-packages/requests/sessions.py", line 515, in request
File "/Users/stryker/.virtualenvs/detection_dev/lib/python3.10/site-packages/requests/sessions.py", line 435, in prepare_request
File "/Users/stryker/.virtualenvs/detection_dev/lib/python3.10/site-packages/requests/cookies.py", line 544, in merge_cookies
File "/Users/stryker/.virtualenvs/detection_dev/lib/python3.10/site-packages/requests/cookies.py", line 352, in update
File "/opt/homebrew/Cellar/python@3.10/3.10.2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/copy.py", line 92, in copy
ImportError: sys.meta_path is None, Python is likely shutting down
|
Here was the output generated:
|
The json file indicates some errors, which should probably be confirmed. e.g. "a16612dd-b30e-4d41-86a0-ebe70974ec00": {
"error": "Found 1 problem\nline 3:3: Unknown column [process.parent.executable], did you mean [process.executable]?",
"error_retrieving_results": true,
"name": "Potential LSASS Clone Creation via PssCaptureSnapShot",
"rule_id": "a16612dd-b30e-4d41-86a0-ebe70974ec00",
"search_count": 0
}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are a couple suggestions and comments that need to be address first.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed together with @Mikaayenson during debug session. Several changes were made to run the commands/code OOTB that may need addressed moving forward for this to be operable.
Is this only occuring when you authenticate with a cookie? What about with user/pass? I have not encountered this - does the stack have a lot of data? If so, this can also be controlled by tightening the search window with this |
This is a good thing - rule error checking for free |
If this tests good for you, this looks to address all stated issues IINM |
This was with user/pass not with a cookie. I haven't touched the stack in a while so I don't believe there's a lot of data. |
Do you know why that example is actually throwing an error? It seems like the fields are correct. |
@brokensound77 @terrancedejesus I'm not sure when I'm doing wrong, but I just retested this PR with the default command and no optional params and it doesn't work. I have a
python -m detection_rules kibana search-alerts
When I tried without the config file, it still did not work.
|
It looks like as of 8.0 the domain from cloud included the port which created an invalid kibana url with double ports resolved in 5fe2720 |
also, I am updating the table for alerts since the structure changed in 8.0 |
@brokensound77 Awesome. I think this is almost done. I was able to get the command to finally work with the config file.
However it still does not work with the command line parameters. Can you try testing the command without using the config? Or perhaps I'm passing the CLI args wrong, in which case the CLI.md should be updated.
|
You're not calling the command correctly. Kibana args should come after |
Yup . You're right, when I passed the kibana creds, it was in the wrong order. The The final thing is the
It seems to timeout. I think as a default we should limit the search window so that it at least returns something and doesn't timeout with an error. I tried the command with and without the ▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
Running survey against 679 rules
Saving detailed dump to: /Users/stryker/workspace/Community/brokensound77/detection-rules/surveys/20220821T075755L.json
/Users/stryker/workspace/Community/brokensound77/detection-rules/detection_rules/eswrap.py:277: ElasticsearchWarning: this request accesses system indices: [.apm-agent-configuration, .apm-custom-link, .async-search, .fleet-agents-7, .fleet-artifacts-7, .fleet-enrollment-api-keys-7, .fleet-policies-7, .fleet-policies-leader-7, .fleet-servers-7, .kibana_7.16.2_001, .kibana_task_manager_7.16.2_001, .ml-config, .security-7, .security-tokens-7, .transform-internal-007], but in a future major version, direct access to system indices will be prevented by default
return self.client.count(body=formatted_dsl, index=index_str, q=lucene_query, allow_no_indices=True,
/Users/stryker/workspace/Community/brokensound77/detection-rules/detection_rules/eswrap.py:183: ElasticsearchWarning: this request accesses system indices: [.apm-agent-configuration, .apm-custom-link, .async-search, .fleet-agents-7, .fleet-artifacts-7, .fleet-enrollment-api-keys-7, .fleet-policies-7, .fleet-policies-leader-7, .fleet-servers-7, .kibana_7.16.2_001, .kibana_task_manager_7.16.2_001, .ml-config, .security-7, .security-tokens-7, .transform-internal-007], but in a future major version, direct access to system indices will be prevented by default
results = self.client.eql.search(body=formatted_dsl, index=index_str, **kwargs)['hits']
/Users/stryker/workspace/Community/brokensound77/detection-rules/detection_rules/eswrap.py:183: ElasticsearchWarning: this request accesses system indices: [.apm-custom-link], but in a future major version, direct access to system indices will be prevented by default
results = self.client.eql.search(body=formatted_dsl, index=index_str, **kwargs)['hits']
/Users/stryker/workspace/Community/brokensound77/detection-rules/detection_rules/eswrap.py:183: ElasticsearchWarning: this request accesses system indices: [.fleet-policies-leader-7], but in a future major version, direct access to system indices will be prevented by default
results = self.client.eql.search(body=formatted_dsl, index=index_str, **kwargs)['hits']
/Users/stryker/workspace/Community/brokensound77/detection-rules/detection_rules/eswrap.py:183: ElasticsearchWarning: this request accesses system indices: [.kibana_task_manager_7.16.2_001], but in a future major version, direct access to system indices will be prevented by default
results = self.client.eql.search(body=formatted_dsl, index=index_str, **kwargs)['hits']
/Users/stryker/workspace/Community/brokensound77/detection-rules/detection_rules/eswrap.py:183: ElasticsearchWarning: this request accesses system indices: [.fleet-agents-7], but in a future major version, direct access to system indices will be prevented by default
results = self.client.eql.search(body=formatted_dsl, index=index_str, **kwargs)['hits']
/Users/stryker/workspace/Community/brokensound77/detection-rules/detection_rules/eswrap.py:183: ElasticsearchWarning: this request accesses system indices: [.fleet-artifacts-7], but in a future major version, direct access to system indices will be prevented by default
results = self.client.eql.search(body=formatted_dsl, index=index_str, **kwargs)['hits']
/Users/stryker/workspace/Community/brokensound77/detection-rules/detection_rules/eswrap.py:183: ElasticsearchWarning: this request accesses system indices: [.fleet-policies-7], but in a future major version, direct access to system indices will be prevented by default
results = self.client.eql.search(body=formatted_dsl, index=index_str, **kwargs)['hits']
/Users/stryker/workspace/Community/brokensound77/detection-rules/detection_rules/eswrap.py:183: ElasticsearchWarning: this request accesses system indices: [.fleet-servers-7], but in a future major version, direct access to system indices will be prevented by default
results = self.client.eql.search(body=formatted_dsl, index=index_str, **kwargs)['hits']
/Users/stryker/workspace/Community/brokensound77/detection-rules/detection_rules/eswrap.py:183: ElasticsearchWarning: this request accesses system indices: [.security-tokens-7], but in a future major version, direct access to system indices will be prevented by default
results = self.client.eql.search(body=formatted_dsl, index=index_str, **kwargs)['hits']
/Users/stryker/workspace/Community/brokensound77/detection-rules/detection_rules/eswrap.py:183: ElasticsearchWarning: this request accesses system indices: [.transform-internal-007], but in a future major version, direct access to system indices will be prevented by default
results = self.client.eql.search(body=formatted_dsl, index=index_str, **kwargs)['hits']
/Users/stryker/workspace/Community/brokensound77/detection-rules/detection_rules/eswrap.py:183: ElasticsearchWarning: this request accesses system indices: [.fleet-enrollment-api-keys-7], but in a future major version, direct access to system indices will be prevented by default
results = self.client.eql.search(body=formatted_dsl, index=index_str, **kwargs)['hits']
/Users/stryker/workspace/Community/brokensound77/detection-rules/detection_rules/eswrap.py:183: ElasticsearchWarning: this request accesses system indices: [.kibana_7.16.2_001], but in a future major version, direct access to system indices will be prevented by default
results = self.client.eql.search(body=formatted_dsl, index=index_str, **kwargs)['hits']
/Users/stryker/workspace/Community/brokensound77/detection-rules/detection_rules/eswrap.py:183: ElasticsearchWarning: this request accesses system indices: [.security-7], but in a future major version, direct access to system indices will be prevented by default
results = self.client.eql.search(body=formatted_dsl, index=index_str, **kwargs)['hits']
/Users/stryker/workspace/Community/brokensound77/detection-rules/detection_rules/eswrap.py:183: ElasticsearchWarning: this request accesses system indices: [.async-search], but in a future major version, direct access to system indices will be prevented by default
results = self.client.eql.search(body=formatted_dsl, index=index_str, **kwargs)['hits']
/Users/stryker/workspace/Community/brokensound77/detection-rules/detection_rules/eswrap.py:183: ElasticsearchWarning: this request accesses system indices: [.apm-agent-configuration], but in a future major version, direct access to system indices will be prevented by default
results = self.client.eql.search(body=formatted_dsl, index=index_str, **kwargs)['hits']
/Users/stryker/workspace/Community/brokensound77/detection-rules/detection_rules/eswrap.py:183: ElasticsearchWarning: this request accesses system indices: [.ml-config], but in a future major version, direct access to system indices will be prevented by default
results = self.client.eql.search(body=formatted_dsl, index=index_str, **kwargs)['hits']
Traceback (most recent call last):
File "/opt/homebrew/Cellar/python@3.10/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/opt/homebrew/Cellar/python@3.10/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/Users/stryker/workspace/Community/brokensound77/detection-rules/detection_rules/__main__.py", line 34, in <module>
main()
File "/Users/stryker/workspace/Community/brokensound77/detection-rules/detection_rules/__main__.py", line 31, in main
root(prog_name="detection_rules")
File "/Users/stryker/.virtualenvs/detection_dev/lib/python3.10/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/Users/stryker/.virtualenvs/detection_dev/lib/python3.10/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/Users/stryker/.virtualenvs/detection_dev/lib/python3.10/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/stryker/.virtualenvs/detection_dev/lib/python3.10/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/stryker/.virtualenvs/detection_dev/lib/python3.10/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/stryker/.virtualenvs/detection_dev/lib/python3.10/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/stryker/.virtualenvs/detection_dev/lib/python3.10/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/Users/stryker/.virtualenvs/detection_dev/lib/python3.10/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/Users/stryker/workspace/Community/brokensound77/detection-rules/detection_rules/misc.py", line 444, in _wrapped
return func(*args, **kwargs)
File "/Users/stryker/workspace/Community/brokensound77/detection-rules/detection_rules/devtools.py", line 1045, in rule_survey
counts = collector.count_from_rule(rules, start_time=start_time, end_time=end_time)
File "/Users/stryker/workspace/Community/brokensound77/detection-rules/detection_rules/eswrap.py", line 291, in count_from_rule
rule_results['search_count'] = self.count(query=rule.contents.data.query,
File "/Users/stryker/workspace/Community/brokensound77/detection-rules/detection_rules/eswrap.py", line 273, in count
results = self.search(query=query, language=language, index=index, start_time=start_time, end_time=end_time,
File "/Users/stryker/workspace/Community/brokensound77/detection-rules/detection_rules/eswrap.py", line 183, in search
results = self.client.eql.search(body=formatted_dsl, index=index_str, **kwargs)['hits']
File "/Users/stryker/.virtualenvs/detection_dev/lib/python3.10/site-packages/elasticsearch/_sync/client/utils.py", line 414, in wrapped
return api(*args, **kwargs)
File "/Users/stryker/.virtualenvs/detection_dev/lib/python3.10/site-packages/elasticsearch/_sync/client/eql.py", line 303, in search
return self.perform_request( # type: ignore[return-value]
File "/Users/stryker/.virtualenvs/detection_dev/lib/python3.10/site-packages/elasticsearch/_sync/client/_base.py", line 390, in perform_request
return self._client.perform_request(
File "/Users/stryker/.virtualenvs/detection_dev/lib/python3.10/site-packages/elasticsearch/_sync/client/_base.py", line 286, in perform_request
meta, resp_body = self.transport.perform_request(
File "/Users/stryker/.virtualenvs/detection_dev/lib/python3.10/site-packages/elastic_transport/_transport.py", line 329, in perform_request
meta, raw_data = node.perform_request(
File "/Users/stryker/.virtualenvs/detection_dev/lib/python3.10/site-packages/elastic_transport/_node/_http_urllib3.py", line 199, in perform_request
raise err from None
elastic_transport.ConnectionTimeout: Connection timed out
python -m detection_rules dev test rule-survey --es-user=elastic 17.13s user 1.29s system 1% cpu 17:05.22 total
@brokensound77 when you run it, does this happen? I also tried running the command with the date-range flag, and I don't think it's being used.
|
Update Sep 2 2022@brokensound77 @terrancedejesus I looked a little closer at this and decided to time the functions causing the bottle neck. Ironically, I was able to get the execution to fully complete, which leads me to believe the issue was due to the amount of data within my stack. I haven't tested a bunch of RTAs on the stack lately so I suspect there was less data to process. The bottleneck appears to be the
Other random thoughts.
=================================================================================================================================================
rule_id name search_count alert_count
=================================================================================================================================================
a87a4e42-1d82-44bd-b0bf-d9b7f91fb89e Web Application Suspicious Activity: POST Request Declined 0
75ee75d8-c180-481c-ba88-ee50129a6aef Web Application Suspicious Activity: Unauthorized Method 0
43303fd4-4839-4e48-b2b2-803ab060758d Web Application Suspicious Activity: No User Agent 0
d49cc73f-7a16-4def-89ce-9fc7127d7820 Web Application Suspicious Activity: sqlmap User Agent 0
027ff9ea-85e7-42e3-99d2-bbb7069e02eb Potential Cookies Theft via Browser Debugging 0
3115bd2c-0baa-4df0-80ea-45e474b5ef93 Agent Spoofing - Mismatched Agent ID 0
493834ca-f861-414c-8602-150d5505b777 Agent Spoofing - Multiple Hosts Using Same Agent 109
665e7a4f-c58e-4fc6-bc83-87a7572670ac WebServer Access Logs Deleted 0
7bcbb3ac-e533-41ad-a612-d6c3bf666aba Tampering of Bash Command-Line History 0
b627cd12-dac4-11ec-9582-f661ea17fbcd Elastic Agent Service Terminated 0
b0046934-486e-462f-9487-0d4cf9e429c6 Timestomping using Touch Command -1
870aecc0-cea4-4110-af3f-e02e9b373655 Security Software Discovery via Grep -1
c85eb82c-d2c8-485c-a36f-534f914b7663 Virtual Machine Fingerprinting via Grep -1
41824afb-d68c-4d0e-bfee-474dac1fa56e EggShell Backdoor Execution 0
a1a0375f-22c2-48c0-81a4-7c2d11cc6856 Potential Reverse Shell Activity via Terminal 0
8acb7614-1d92-4359-bfcf-478b6d9de150 Suspicious JAVA Child Process 0
c3f5e1d8-910e-43b4-8d44-d748e498ca86 Potential JAVA/JNDI Exploitation Attempt -1
9c260313-c811-4ec8-ab89-8f6530e0246c Hosts File Modified 0
58ac2aa5-6718-427c-a845-5f3ac5af00ba Zoom Meeting with no Passcode 0
93f47b6f-5728-4004-ba00-625083b3dcb0 Modification of Standard Authentication Module or Configuration 0
e6c1a552-7776-44ad-ae0f-8746cc07773c Bash Shell Profile Modification 0
2215b8bd-1759-4ffa-8ab8-55c8e6b32e7f SSH Authorized Keys File Modification 0
76152ca1-71d0-4003-9e37-0983e12832da Potential Privilege Escalation via Sudoers File Modification 0
8a1b0278-0f9a-487d-96bd-d4833298e87a Setuid / Setgid Bit Set via chmod -1
f37f3054-d40b-49ac-aa9b-a786c74c58b8 Sudo Heap-Based Buffer Overflow Attempt 0
931e25a5-0f5e-4ae0-ba0d-9e94eff7e3a4 Sudoers File Modification 0
699e9fdb-b77c-4c01-995c-1c15019b9c43 Threat Intel Filebeat Module (v8.x) Indicator Match 0
0c9a14d9-d65d-486f-9b5b-91e4e6b22bd0 Threat Intel Indicator Match 0
collector = CollectEvents(elasticsearch_client)
start = time.time()
details = collector.search_from_rule(rules, start_time=start_time, end_time=end_time)
end = time.time()
print(f'Survey search took {end - start} seconds')
collector = CollectEvents(elasticsearch_client)
start = time.time()
counts = collector.count_from_rule(rules, start_time=start_time, end_time=end_time)
end = time.time()
print(f'Survey count took {end - start} seconds') First run with 10 second window (detection_dev) ➜ detection-rules git:(cleanup-survey-code) ✗ python -m detection_rules dev test rule-survey -d now-10s now
Loaded config file: /Users/stryker/workspace/ElasticGitHub/community/bksound/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
Running survey against 680 rules
Saving detailed dump to: /Users/stryker/workspace/ElasticGitHub/community/bksound/detection-rules/surveys/20220902T140101L.json
Survey search took 45.89954900741577 seconds
Survey count took 322.0308530330658 seconds
(detection_dev) ➜ detection-rules git:(cleanup-survey-code) ✗
Second run with 10 second window (detection_dev) ➜ detection-rules git:(cleanup-survey-code) ✗ python -m detection_rules dev test rule-survey -d now-10s now
Loaded config file: /Users/stryker/workspace/ElasticGitHub/community/bksound/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
Running survey against 680 rules
Saving detailed dump to: /Users/stryker/workspace/ElasticGitHub/community/bksound/detection-rules/surveys/20220902T140919L.json
Survey search took 40.028501987457275 seconds
Survey count took 336.776211977005 seconds
(detection_dev) ➜ detection-rules git:(cleanup-survey-code) ✗
Third run with 30 second window (detection_dev) ➜ detection-rules git:(cleanup-survey-code) ✗ python -m detection_rules dev test rule-survey -d now-30s now
Loaded config file: /Users/stryker/workspace/ElasticGitHub/community/bksound/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
Running survey against 680 rules
Saving detailed dump to: /Users/stryker/workspace/ElasticGitHub/community/bksound/detection-rules/surveys/20220902T141609L.json
Survey search took 43.72844910621643 seconds
Survey count took 316.44426012039185 seconds
(detection_dev) ➜ detection-rules git:(cleanup-survey-code) ✗
Final (fourth) run with 60 second window (detection_dev) ➜ detection-rules git:(cleanup-survey-code) ✗ python -m detection_rules dev test rule-survey -d now-60s now
Loaded config file: /Users/stryker/workspace/ElasticGitHub/community/bksound/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
Running survey against 680 rules
Saving detailed dump to: /Users/stryker/workspace/ElasticGitHub/community/bksound/detection-rules/surveys/20220902T142519L.json
Survey search took 39.9447717666626 seconds
Survey count took 331.9557840824127 seconds
(detection_dev) ➜ detection-rules git:(cleanup-survey-code) ✗
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I finally identified the bottleneck with this bug. If may be good to adjust the default to a smaller list of indices instead of *
as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
Can you add a note somewhere (readme, CLI.md, docstring or something that explains the meaning of search count numbers mean (-1, 0, +1)
in the table generated.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great work! LGTM
* Cleanup rule survey code * default to only unique-ing on process name for lucene rules * fix bug in kibana url parsing by removing redundant port from domain * update search-alerts columns and nest fields * fix rule.contents.data.index Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com> (cherry picked from commit 332ea40)
* Cleanup rule survey code * default to only unique-ing on process name for lucene rules * fix bug in kibana url parsing by removing redundant port from domain * update search-alerts columns and nest fields * fix rule.contents.data.index Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com> (cherry picked from commit 332ea40)
* Cleanup rule survey code * default to only unique-ing on process name for lucene rules * fix bug in kibana url parsing by removing redundant port from domain * update search-alerts columns and nest fields * fix rule.contents.data.index Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com> (cherry picked from commit 332ea40)
* Cleanup rule survey code * default to only unique-ing on process name for lucene rules * fix bug in kibana url parsing by removing redundant port from domain * update search-alerts columns and nest fields * fix rule.contents.data.index Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com> (cherry picked from commit 332ea40)
* Cleanup rule survey code * default to only unique-ing on process name for lucene rules * fix bug in kibana url parsing by removing redundant port from domain * update search-alerts columns and nest fields * fix rule.contents.data.index Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com> (cherry picked from commit 332ea40)
* Cleanup rule survey code * default to only unique-ing on process name for lucene rules * fix bug in kibana url parsing by removing redundant port from domain * update search-alerts columns and nest fields * fix rule.contents.data.index Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com> (cherry picked from commit 332ea40)
Issues
None
Summary
I was working on a related project using this code and founds some errors due to changes from previous rule object refactoring