Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[8.x](backport #6114) Store AST directly in provider mappings #6158

Merged
merged 1 commit into from
Nov 27, 2024

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Nov 27, 2024

What does this PR do?

For all variable provider types, we store mappings as untyped dictionaries and only convert them to AST when we're about to generate actual vars. This means we have to recompute all of it every time there's a change to any of the provider data, even if all the other data is unaffected. For example, if a Kubernetes Pod gains a new label, we need to regenerate vars for all Pods.

Instead, we can store mappings as AST to begin with. Then the only cost of emitting new vars after a change is updating a bunch of references.

Why is it important?

This unnecessary recomputation can be quite expensive when there's a lot of mappings from the dynamic provider. See the benchstat report below:

goos: linux
goarch: amd64
pkg: github.com/elastic/elastic-agent/internal/pkg/composable
cpu: 13th Gen Intel(R) Core(TM) i7-13700H
                       │ bench_main.txt │          bench_dynamic.txt          │
                       │     sec/op     │   sec/op     vs base                │
GenerateVars100Pods-20     755.49µ ± 3%   76.87µ ± 2%  -89.82% (p=0.000 n=10)

                       │ bench_main.txt │          bench_dynamic.txt           │
                       │      B/op      │     B/op      vs base                │
GenerateVars100Pods-20    735.00Ki ± 0%   98.69Ki ± 0%  -86.57% (p=0.000 n=10)

                       │ bench_main.txt │          bench_dynamic.txt          │
                       │   allocs/op    │  allocs/op   vs base                │
GenerateVars100Pods-20     21.335k ± 0%   1.364k ± 0%  -93.61% (p=0.000 n=10)

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas

Related issues


This is an automatic backport of pull request #6114 done by [Mergify](https://mergify.com).

* Store AST directly in dynamic provider mappings

* Store AST directly in context provider mappings

(cherry picked from commit fe0f6b0)
@mergify mergify bot requested a review from a team as a code owner November 27, 2024 15:46
@mergify mergify bot added the backport label Nov 27, 2024
@mergify mergify bot requested review from andrzej-stencel and pchila and removed request for a team November 27, 2024 15:46
@swiatekm swiatekm enabled auto-merge (squash) November 27, 2024 15:56
@pierrehilbert pierrehilbert added the Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team label Nov 27, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

Copy link

Quality Gate failed Quality Gate failed

Failed conditions
0.0% Coverage on New Code (required ≥ 40%)

See analysis details on SonarQube

@swiatekm swiatekm merged commit ab5e07a into 8.x Nov 27, 2024
15 of 16 checks passed
@swiatekm swiatekm deleted the mergify/bp/8.x/pr-6114 branch November 27, 2024 17:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants