eBPF instrumentation manager #1776
Conversation
RonFed
requested review from
tamirdavid1,
damemi,
edeNFed,
blumamir and
david336362
and removed request for
tamirdavid1
RonFed
changed the title
| m.stop = stop | ||
|
|
||
| // main event loop for handling instrumentations | ||
| for { |
There was a problem hiding this comment.
I am not sure how or if we should handle it, but I think this design has a noisy neighbor issue:
Imaging a Pod with the following bash script:
while true; do
echo "test"
doneThis will trigger tons of ProcEvents and will cause starvation for all other pods / configuration changes.
What do you think? cc @blumamir
odiglet/pkg/ebpf/manager.go
Outdated
|
|
||
| m.Logger.Info("cleaning instrumentation resources", "pid", pid) | ||
|
|
||
| err := details.Inst.Close(ctx) |
There was a problem hiding this comment.
same nil check, maybe we should move the Close function to the details instance instead of closing the individual fields?
This will probably never be nil but still afraid of panics
There was a problem hiding this comment.
I created the startTrackInstrumentation which is the only place where instrumentationDetails is created, since this is an internal logic to this package, and not dependant on user input I think if the inst is nill - it is a result of some logical error on our side which should never happen.
odiglet/pkg/ebpf/manager.go
Outdated
|
|
||
| // active instrumentations by workload, and aggregated by pid | ||
| // this map is not concurrent safe, so it should be accessed only from the main event loop | ||
| detailsByWorkload map[types.NamespacedName]map[int]*instrumentationDetails |
There was a problem hiding this comment.
add workload kind and language here and simplify the usage
…he instrumentation manager
| // In case of a failure, an error will be returned and all the resources will be cleaned up. | ||
| Load(ctx context.Context) error | ||
|
|
||
| // Run will attach the probes to the relevant process, and will start the instrumentation. |
There was a problem hiding this comment.
| // Run will attach the probes to the relevant process, and will start the instrumentation. | |
| // Run will start the instrumentation. |
A follow up to #1776: * Make the instrumentation manager generic, and move it to a new module. * Add a k8s-oriented implementation of the manager's different interfaces under `odiglet/pkg/ebpf`
This PR is a follow-up to #1645.
Adding the new
Managerwhich will eventually replace theDirector.The new design has the following key features and improvements:
note: the
runtime-detectoris configured to filter process events and will only pass events according to its configuration.Managerdoes not have locks and uses an internal event loop.InstrumentationConfigreconciler (same as before) - those updates will be handled in the event loop.Factoryinterface is refactored and aSettingsoption can be expanded in the future to add more initial configuration options.Instrumentationinterface is introduced and will replaceOtelEbpfSdk.go.opentelemetry.io/autotov0.18.0-alpha.This change will currently only apply for OSS Go instrumentation.