-
Notifications
You must be signed in to change notification settings - Fork 8.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Ingest Manager] Manually build Fleet kuery with Node arguments #76589
[Ingest Manager] Manually build Fleet kuery with Node arguments #76589
Conversation
Pinging @elastic/ingest-management (Team:Ingest Management) |
d18b7a0
to
a0b7c77
Compare
a0b7c77
to
7ed80d5
Compare
Very nice! I did not know this was an option. @lukasolson, you're the KQL expert, does this make sense to you? |
Hmm, if this performance improvement is really necessary, this is fine, but I'd generally like to avoid using |
@lukasolson perhaps we should address this by providing a better interface for building these KQL queries which hide the intricacies? I believe you mentioned considering implementing this before, is there an existing GitHub issue for that? |
@elasticmachine merge upstream |
@lukasolson Do you think we can move forward with this PR? |
Yes, it's fine. I've opened #77085 to track building a better programmatic API. |
@lukasolson thanks for the issue, a simple API like the one you put in the issue will be amazing :) There is one change to the |
@elasticmachine merge upstream |
💚 Build SucceededBuild metrics
History
To update your PR or re-run it, just comment with: |
* master: (25 commits) [Security Solution] Add unit tests for Network search strategy (elastic#77416) [Alerting] Improves performance of the authorization filter in AlertsClient.find by skipping KQL parsing (elastic#77040) [Ingest Manager] Add route for package installation by upload (elastic#77044) [APM-UI][E2E] filter PRs from the uptime GH team (elastic#77359) [APM] Remove useLocation and some minor route improvements (elastic#76343) [Enterprise Search] Update enterpriseSearchRequestHandler to manage range of errors + add handleAPIErrors helper (elastic#77258) [SECURITY_SOLUTION] Task/hostname policy response ux updates (elastic#76444) Move remaining uses of serviceName away from urlParams (elastic#77248) [Lens] Move configuration popover to flyout (elastic#76046) [Ingest Manager] Manually build Fleet kuery with Node arguments (elastic#76589) skip flaky suite (elastic#59975) Neutral-naming in reporting plugin (elastic#77371) [Enterprise Search] Add UserIcon styles (elastic#77385) [RUM Dashboard] Added loading state to visitor breakdown pie charts (elastic#77201) [Ingest Manager] Fix polling for new agent action (elastic#77339) Remote cluster - Functional UI test to change the superuser to a test_user with limited role (elastic#77212) Stacked headers and navigational search (elastic#72331) [ML] DF Analytics creation wizard: Fixing field loading race condition (elastic#77326) [Monitoring] Handle no mappings found for sort and collapse fields (elastic#77099) Add Lens to Recently Accessed (elastic#77249) ...
* master: (293 commits) Fix tsvb filter ration for table (elastic#77272) [Security Solution] Add unit tests for Network search strategy (elastic#77416) [Alerting] Improves performance of the authorization filter in AlertsClient.find by skipping KQL parsing (elastic#77040) [Ingest Manager] Add route for package installation by upload (elastic#77044) [APM-UI][E2E] filter PRs from the uptime GH team (elastic#77359) [APM] Remove useLocation and some minor route improvements (elastic#76343) [Enterprise Search] Update enterpriseSearchRequestHandler to manage range of errors + add handleAPIErrors helper (elastic#77258) [SECURITY_SOLUTION] Task/hostname policy response ux updates (elastic#76444) Move remaining uses of serviceName away from urlParams (elastic#77248) [Lens] Move configuration popover to flyout (elastic#76046) [Ingest Manager] Manually build Fleet kuery with Node arguments (elastic#76589) skip flaky suite (elastic#59975) Neutral-naming in reporting plugin (elastic#77371) [Enterprise Search] Add UserIcon styles (elastic#77385) [RUM Dashboard] Added loading state to visitor breakdown pie charts (elastic#77201) [Ingest Manager] Fix polling for new agent action (elastic#77339) Remote cluster - Functional UI test to change the superuser to a test_user with limited role (elastic#77212) Stacked headers and navigational search (elastic#72331) [ML] DF Analytics creation wizard: Fixing field loading race condition (elastic#77326) [Monitoring] Handle no mappings found for sort and collapse fields (elastic#77099) ...
Summary
This PR #75693 introduced a way to manually build filter instead of parsing the KQL string to improve perfomance. While improving a lot the performance there is still room for improvment if we use
buildNodeWithArgumentNodes
.Benchmarking the potential improvment
Give the following result