Store AST directly in provider mappings #6114
Conversation
swiatekm
added
enhancement
pierrehilbert
added
the
Team:Elastic-Agent-Control-Plane
|
Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane) |
swiatekm
force-pushed
the
fix/dynamic-provider-ast
branch
from
3c62d14
to
8cd14c8
Compare
|
/test |
swiatekm
force-pushed
the
fix/dynamic-provider-ast
branch
from
8cd14c8
to
6860e91
Compare
|
| @@ -825,6 +825,9 @@ func (a *AST) HashStr() string { | |||
|
|
|||
| // Equal check if two AST are equals by using the computed hash. | |||
| func (a *AST) Equal(other *AST) bool { | |||
| if a.root == nil || other.root == nil { | |||
| return a.root == other.root | |||
| } | |||
There was a problem hiding this comment.
nit 🙂
| } | |
| } | |
| // nil check above is mandatory as calling Hash() on a nil may cause a panic |
There was a problem hiding this comment.
This code change LGTM. the CI failures are due to TestOtelAPMIngestion (#4148) and TestRpmFleetUpgrade. The last one fails because it doesn't download in time the elastic-agent-8.17.0-SNAPSHOT-x86_64.rpm artifact with a context deadline exceeded (code ref)
cc @ycombinator should we create an issue about the latter?
I'm not seeing these tests failing on |
swiatekm
removed
the
backport-8.15
* Store AST directly in dynamic provider mappings * Store AST directly in context provider mappings (cherry picked from commit fe0f6b0)
* Store AST directly in dynamic provider mappings * Store AST directly in context provider mappings (cherry picked from commit fe0f6b0)
* Store AST directly in dynamic provider mappings * Store AST directly in context provider mappings (cherry picked from commit fe0f6b0) # Conflicts: # internal/pkg/composable/controller.go
What does this PR do?
For all variable provider types, we store mappings as untyped dictionaries and only convert them to AST when we're about to generate actual vars. This means we have to recompute all of it every time there's a change to any of the provider data, even if all the other data is unaffected. For example, if a Kubernetes Pod gains a new label, we need to regenerate vars for all Pods.
Instead, we can store mappings as AST to begin with. Then the only cost of emitting new vars after a change is updating a bunch of references.
Why is it important?
This unnecessary recomputation can be quite expensive when there's a lot of mappings from the dynamic provider. See the benchstat report below:
Checklist
Related issues