-
Notifications
You must be signed in to change notification settings - Fork 467
feat(azure_eventhubs): add azure eventhubs integration #14636
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(azure_eventhubs): add azure eventhubs integration #14636
Conversation
|
|
Bootstrap import analysisComparison of import times between this PR and base. SummaryThe average import time from this PR is: 241 ± 3 ms. The average import time from base is: 243 ± 2 ms. The import time difference between this PR and base is: -2.4 ± 0.1 ms. Import time breakdownThe following import paths have disappeared:
|
Performance SLOsComparing candidate duncan-harvey/azure-eventhubs-integration (8046615) with baseline main (0dc1c91) 📈 Performance Regressions (1 suite)📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 4.406µs (SLO: <10.000µs 📉 -55.9%) vs baseline: +1.6% Memory: ✅ 37.690MB (SLO: <39.000MB -3.4%) vs baseline: +5.1% ✅ ospathbasename_noaspectTime: ✅ 1.077µs (SLO: <10.000µs 📉 -89.2%) vs baseline: -0.8% Memory: ✅ 37.650MB (SLO: <39.000MB -3.5%) vs baseline: +4.9% ✅ ospathjoin_aspectTime: ✅ 6.866µs (SLO: <10.000µs 📉 -31.3%) vs baseline: 📈 +12.4% Memory: ✅ 37.631MB (SLO: <39.000MB -3.5%) vs baseline: +4.8% ✅ ospathjoin_noaspectTime: ✅ 2.310µs (SLO: <10.000µs 📉 -76.9%) vs baseline: +0.3% Memory: ✅ 37.729MB (SLO: <39.000MB -3.3%) vs baseline: +5.1% ✅ ospathnormcase_aspectTime: ✅ 3.906µs (SLO: <10.000µs 📉 -60.9%) vs baseline: 📈 +12.7% Memory: ✅ 37.631MB (SLO: <39.000MB -3.5%) vs baseline: +4.7% ✅ ospathnormcase_noaspectTime: ✅ 0.575µs (SLO: <10.000µs 📉 -94.2%) vs baseline: +1.3% Memory: ✅ 37.650MB (SLO: <39.000MB -3.5%) vs baseline: +4.7% ✅ ospathsplit_aspectTime: ✅ 4.967µs (SLO: <10.000µs 📉 -50.3%) vs baseline: +0.7% Memory: ✅ 37.729MB (SLO: <39.000MB -3.3%) vs baseline: +5.1% ✅ ospathsplit_noaspectTime: ✅ 1.613µs (SLO: <10.000µs 📉 -83.9%) vs baseline: +1.0% Memory: ✅ 37.709MB (SLO: <39.000MB -3.3%) vs baseline: +4.9% ✅ ospathsplitdrive_aspectTime: ✅ 3.696µs (SLO: <10.000µs 📉 -63.0%) vs baseline: +0.8% Memory: ✅ 37.729MB (SLO: <39.000MB -3.3%) vs baseline: +4.9% ✅ ospathsplitdrive_noaspectTime: ✅ 0.707µs (SLO: <10.000µs 📉 -92.9%) vs baseline: +0.7% Memory: ✅ 37.709MB (SLO: <39.000MB -3.3%) vs baseline: +5.2% ✅ ospathsplitext_aspectTime: ✅ 4.546µs (SLO: <10.000µs 📉 -54.5%) vs baseline: -0.8% Memory: ✅ 37.690MB (SLO: <39.000MB -3.4%) vs baseline: +4.9% ✅ ospathsplitext_noaspectTime: ✅ 1.395µs (SLO: <10.000µs 📉 -86.0%) vs baseline: +0.7% Memory: ✅ 37.670MB (SLO: <39.000MB -3.4%) vs baseline: +4.7% 🟡 Near SLO Breach (6 suites)🟡 djangosimple - 30/30✅ appsecTime: ✅ 20.445ms (SLO: <22.300ms -8.3%) vs baseline: -0.1% Memory: ✅ 65.357MB (SLO: <67.000MB -2.5%) vs baseline: +4.6% ✅ exception-replay-enabledTime: ✅ 1.343ms (SLO: <1.450ms -7.3%) vs baseline: -0.1% Memory: ✅ 64.576MB (SLO: <67.000MB -3.6%) vs baseline: +4.7% ✅ iastTime: ✅ 20.470ms (SLO: <22.250ms -8.0%) vs baseline: ~same Memory: ✅ 65.416MB (SLO: <67.000MB -2.4%) vs baseline: +4.8% ✅ profilerTime: ✅ 15.324ms (SLO: <16.550ms -7.4%) vs baseline: +0.5% Memory: ✅ 53.671MB (SLO: <54.500MB 🟡 -1.5%) vs baseline: +4.7% ✅ resource-renamingTime: ✅ 20.572ms (SLO: <21.750ms -5.4%) vs baseline: ~same Memory: ✅ 65.414MB (SLO: <67.000MB -2.4%) vs baseline: +4.6% ✅ span-code-originTime: ✅ 26.178ms (SLO: <28.200ms -7.2%) vs baseline: -0.3% Memory: ✅ 67.556MB (SLO: <69.500MB -2.8%) vs baseline: +4.8% ✅ tracerTime: ✅ 20.445ms (SLO: <21.750ms -6.0%) vs baseline: -0.4% Memory: ✅ 65.526MB (SLO: <67.000MB -2.2%) vs baseline: +5.0% ✅ tracer-and-profilerTime: ✅ 21.995ms (SLO: <23.500ms -6.4%) vs baseline: -0.6% Memory: ✅ 66.690MB (SLO: <67.500MB 🟡 -1.2%) vs baseline: +5.0% ✅ tracer-dont-create-db-spansTime: ✅ 19.295ms (SLO: <21.500ms 📉 -10.3%) vs baseline: -0.5% Memory: ✅ 65.409MB (SLO: <66.000MB 🟡 -0.9%) vs baseline: +4.7% ✅ tracer-minimalTime: ✅ 16.641ms (SLO: <17.500ms -4.9%) vs baseline: +0.2% Memory: ✅ 65.465MB (SLO: <66.000MB 🟡 -0.8%) vs baseline: +4.9% ✅ tracer-nativeTime: ✅ 20.463ms (SLO: <21.750ms -5.9%) vs baseline: -0.2% Memory: ✅ 71.341MB (SLO: <72.500MB 🟡 -1.6%) vs baseline: +4.8% ✅ tracer-no-cachesTime: ✅ 18.418ms (SLO: <19.650ms -6.3%) vs baseline: -0.4% Memory: ✅ 65.497MB (SLO: <67.000MB -2.2%) vs baseline: +4.8% ✅ tracer-no-databasesTime: ✅ 18.754ms (SLO: <20.100ms -6.7%) vs baseline: -0.3% Memory: ✅ 65.453MB (SLO: <67.000MB -2.3%) vs baseline: +5.0% ✅ tracer-no-middlewareTime: ✅ 20.148ms (SLO: <21.500ms -6.3%) vs baseline: ~same Memory: ✅ 65.396MB (SLO: <67.000MB -2.4%) vs baseline: +4.7% ✅ tracer-no-templatesTime: ✅ 20.281ms (SLO: <22.000ms -7.8%) vs baseline: ~same Memory: ✅ 65.425MB (SLO: <67.000MB -2.4%) vs baseline: +4.8% 🟡 errortrackingdjangosimple - 6/6✅ errortracking-enabled-allTime: ✅ 18.251ms (SLO: <19.850ms -8.1%) vs baseline: +0.8% Memory: ✅ 65.215MB (SLO: <66.500MB 🟡 -1.9%) vs baseline: +4.8% ✅ errortracking-enabled-userTime: ✅ 18.057ms (SLO: <19.400ms -6.9%) vs baseline: ~same Memory: ✅ 65.294MB (SLO: <66.500MB 🟡 -1.8%) vs baseline: +4.9% ✅ tracer-enabledTime: ✅ 18.097ms (SLO: <19.450ms -7.0%) vs baseline: ~same Memory: ✅ 65.254MB (SLO: <66.500MB 🟡 -1.9%) vs baseline: +4.9% 🟡 flasksimple - 18/18✅ appsec-getTime: ✅ 4.567ms (SLO: <4.750ms -3.8%) vs baseline: -0.4% Memory: ✅ 61.932MB (SLO: <65.000MB -4.7%) vs baseline: +4.9% ✅ appsec-postTime: ✅ 6.563ms (SLO: <6.750ms -2.8%) vs baseline: ~same Memory: ✅ 61.833MB (SLO: <65.000MB -4.9%) vs baseline: +4.8% ✅ appsec-telemetryTime: ✅ 4.571ms (SLO: <4.750ms -3.8%) vs baseline: -0.2% Memory: ✅ 61.932MB (SLO: <65.000MB -4.7%) vs baseline: +4.9% ✅ debuggerTime: ✅ 1.856ms (SLO: <2.000ms -7.2%) vs baseline: -0.5% Memory: ✅ 45.534MB (SLO: <47.000MB -3.1%) vs baseline: +5.1% ✅ iast-getTime: ✅ 1.863ms (SLO: <2.000ms -6.9%) vs baseline: -0.3% Memory: ✅ 42.408MB (SLO: <49.000MB 📉 -13.5%) vs baseline: +5.0% ✅ profilerTime: ✅ 1.909ms (SLO: <2.100ms -9.1%) vs baseline: -0.2% Memory: ✅ 46.458MB (SLO: <47.000MB 🟡 -1.2%) vs baseline: +4.9% ✅ resource-renamingTime: ✅ 3.377ms (SLO: <3.650ms -7.5%) vs baseline: -0.3% Memory: ✅ 52.180MB (SLO: <53.500MB -2.5%) vs baseline: +4.8% ✅ tracerTime: ✅ 3.369ms (SLO: <3.650ms -7.7%) vs baseline: -0.3% Memory: ✅ 52.199MB (SLO: <53.500MB -2.4%) vs baseline: +4.9% ✅ tracer-nativeTime: ✅ 3.367ms (SLO: <3.650ms -7.7%) vs baseline: ~same Memory: ✅ 58.323MB (SLO: <60.000MB -2.8%) vs baseline: +4.9% 🟡 flasksqli - 6/6✅ appsec-enabledTime: ✅ 3.925ms (SLO: <4.200ms -6.6%) vs baseline: ~same Memory: ✅ 62.305MB (SLO: <66.000MB -5.6%) vs baseline: +5.1% ✅ iast-enabledTime: ✅ 2.450ms (SLO: <2.800ms 📉 -12.5%) vs baseline: +0.4% Memory: ✅ 58.805MB (SLO: <60.000MB 🟡 -2.0%) vs baseline: +4.9% ✅ tracer-enabledTime: ✅ 2.065ms (SLO: <2.250ms -8.2%) vs baseline: -0.5% Memory: ✅ 52.219MB (SLO: <54.500MB -4.2%) vs baseline: +4.9% 🟡 otelspan - 22/22✅ add-eventTime: ✅ 43.279ms (SLO: <47.150ms -8.2%) vs baseline: +1.7% Memory: ✅ 44.430MB (SLO: <47.000MB -5.5%) vs baseline: +4.8% ✅ add-metricsTime: ✅ 316.923ms (SLO: <344.800ms -8.1%) vs baseline: ~same Memory: ✅ 594.452MB (SLO: <600.000MB 🟡 -0.9%) vs baseline: +4.6% ✅ add-tagsTime: ✅ 286.380ms (SLO: <314.000ms -8.8%) vs baseline: -0.6% Memory: ✅ 596.140MB (SLO: <600.000MB 🟡 -0.6%) vs baseline: +4.8% ✅ get-contextTime: ✅ 80.958ms (SLO: <92.350ms 📉 -12.3%) vs baseline: +0.7% Memory: ✅ 39.988MB (SLO: <46.500MB 📉 -14.0%) vs baseline: +5.0% ✅ is-recordingTime: ✅ 39.699ms (SLO: <44.500ms 📉 -10.8%) vs baseline: +2.0% Memory: ✅ 43.895MB (SLO: <47.500MB -7.6%) vs baseline: +4.7% ✅ record-exceptionTime: ✅ 58.917ms (SLO: <67.650ms 📉 -12.9%) vs baseline: -0.3% Memory: ✅ 40.077MB (SLO: <47.000MB 📉 -14.7%) vs baseline: +4.3% ✅ set-statusTime: ✅ 45.069ms (SLO: <50.400ms 📉 -10.6%) vs baseline: +1.1% Memory: ✅ 43.939MB (SLO: <47.000MB -6.5%) vs baseline: +4.9% ✅ startTime: ✅ 38.390ms (SLO: <43.450ms 📉 -11.6%) vs baseline: +0.2% Memory: ✅ 43.983MB (SLO: <47.000MB -6.4%) vs baseline: +5.0% ✅ start-finishTime: ✅ 82.924ms (SLO: <88.000ms -5.8%) vs baseline: -0.2% Memory: ✅ 34.544MB (SLO: <46.500MB 📉 -25.7%) vs baseline: +4.9% ✅ start-finish-telemetryTime: ✅ 84.546ms (SLO: <89.000ms -5.0%) vs baseline: +0.2% Memory: ✅ 34.544MB (SLO: <46.500MB 📉 -25.7%) vs baseline: +4.7% ✅ update-nameTime: ✅ 40.252ms (SLO: <45.150ms 📉 -10.8%) vs baseline: ~same Memory: ✅ 44.205MB (SLO: <47.000MB -5.9%) vs baseline: +4.7% 🟡 packagesupdateimporteddependencies - 24/24✅ import_manyTime: ✅ 154.907µs (SLO: <170.000µs -8.9%) vs baseline: -0.3% Memory: ✅ 37.832MB (SLO: <38.500MB 🟡 -1.7%) vs baseline: +4.5% ✅ import_many_cachedTime: ✅ 122.216µs (SLO: <130.000µs -6.0%) vs baseline: +1.1% Memory: ✅ 37.926MB (SLO: <38.500MB 🟡 -1.5%) vs baseline: +5.3% ✅ import_many_stdlibTime: ✅ 1.644ms (SLO: <1.750ms -6.1%) vs baseline: +0.7% Memory: ✅ 37.639MB (SLO: <38.500MB -2.2%) vs baseline: +5.0% ✅ import_many_stdlib_cachedTime: ✅ 0.979ms (SLO: <1.100ms 📉 -11.0%) vs baseline: -0.5% Memory: ✅ 38.226MB (SLO: <38.500MB 🟡 -0.7%) vs baseline: +6.6% ✅ import_many_unknownTime: ✅ 834.482µs (SLO: <890.000µs -6.2%) vs baseline: +1.1% Memory: ✅ 37.648MB (SLO: <38.500MB -2.2%) vs baseline: +5.2% ✅ import_many_unknown_cachedTime: ✅ 791.293µs (SLO: <870.000µs -9.0%) vs baseline: -0.3% Memory: ✅ 37.923MB (SLO: <38.500MB 🟡 -1.5%) vs baseline: +4.0% ✅ import_oneTime: ✅ 19.693µs (SLO: <30.000µs 📉 -34.4%) vs baseline: -0.3% Memory: ✅ 37.855MB (SLO: <39.000MB -2.9%) vs baseline: +4.4% ✅ import_one_cacheTime: ✅ 6.321µs (SLO: <10.000µs 📉 -36.8%) vs baseline: -0.8% Memory: ✅ 37.655MB (SLO: <38.500MB -2.2%) vs baseline: +3.7% ✅ import_one_stdlibTime: ✅ 18.737µs (SLO: <20.000µs -6.3%) vs baseline: ~same Memory: ✅ 37.591MB (SLO: <38.500MB -2.4%) vs baseline: +3.9% ✅ import_one_stdlib_cacheTime: ✅ 6.240µs (SLO: <10.000µs 📉 -37.6%) vs baseline: -0.4% Memory: ✅ 37.945MB (SLO: <38.500MB 🟡 -1.4%) vs baseline: +4.6% ✅ import_one_unknownTime: ✅ 45.473µs (SLO: <50.000µs -9.1%) vs baseline: +0.2% Memory: ✅ 37.452MB (SLO: <38.500MB -2.7%) vs baseline: +3.4% ✅ import_one_unknown_cacheTime: ✅ 6.260µs (SLO: <10.000µs 📉 -37.4%) vs baseline: ~same Memory: ✅ 37.985MB (SLO: <38.500MB 🟡 -1.3%) vs baseline: +4.7%
|
630c830 to
14cf902
Compare
…tion to azure_functions integration
14cf902 to
63ea5a8
Compare
01eb7e2 to
c38a2f6
Compare
pablomartinezbernardo
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking good from the logic and span structure point of view, leaving the tracer specifics for others to review
wantsui
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it possible to add a test/snapshot that demonstrate an error occurring?
Co-authored-by: wantsui <wan.tsui@datadoghq.com>
Co-authored-by: wantsui <wan.tsui@datadoghq.com>
brettlangdon
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nits, otherwise lgtm from apm-core-python
…4922) ## Description For `azure_eventhubs`, `azure_functions`, `azure_servicebus` integrations: - Add typing to util functions - call _finish_span directly instead of using the ctx.span context manager - recommend import ddtrace.auto instead of patch ## Testing Uses existing test coverage, no additional testing needed ## Risks <!-- Note any risks associated with this change, or "None" if no risks --> ## Additional Notes Follows comments made in #14636 when adding `azure_eventhubs` integration
This PR adds tracing support for producers in the
azure-eventhubpackage and event hubs triggers in theazure-functionspackage.Instrumentation details:
createspans when messages are added to the batchcreateproducer spans will link to both thesendproducer span that sends the batch as well as thereceiveconsumer span that receives the batch or individual messages of the batchcreateproducer spans and its links withDD_TRACE_AZURE_EVENTHUBS_BATCH_LINKS_ENABLEDcreateproducer spans can be children of asendproducer span if the send operation creates a batch internally and adds events to it. This occurs when a list of events is passed tosend_batchbuffered_modeis enabledAdditional Notes:
send_event,send_batch)single,list,batch)Checklist
Reviewer Checklist