-
Notifications
You must be signed in to change notification settings - Fork 317
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[MLOB-2233] feat(openai, llmobs): patch _thenUnwrap
of chat completion api promise instances
#5253
Conversation
Overall package sizeSelf size: 8.68 MB Dependency sizes| name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.4.0 | 29.44 MB | 29.44 MB | | @datadog/native-appsec | 8.4.0 | 19.25 MB | 19.26 MB | | @datadog/native-iast-taint-tracking | 3.3.0 | 13.77 MB | 13.78 MB | | @datadog/pprof | 5.5.1 | 9.79 MB | 10.17 MB | | protobufjs | 7.2.5 | 2.77 MB | 5.16 MB | | @datadog/native-iast-rewriter | 2.8.0 | 2.6 MB | 2.74 MB | | @opentelemetry/core | 1.14.0 | 872.87 kB | 1.47 MB | | @datadog/native-metrics | 3.1.0 | 1.06 MB | 1.46 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | import-in-the-middle | 1.11.2 | 112.74 kB | 835.4 kB | | source-map | 0.7.4 | 226 kB | 226 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | lru-cache | 7.18.3 | 133.92 kB | 133.92 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.0 | 109.9 kB | 109.9 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 5.3.2 | 53.63 kB | 53.63 kB | | shell-quote | 1.8.1 | 44.96 kB | 44.96 kB | | istanbul-lib-coverage | 3.2.0 | 29.34 kB | 29.34 kB | | rfdc | 1.3.1 | 25.21 kB | 25.21 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | dc-polyfill | 0.1.4 | 23.1 kB | 23.1 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | semifies | 1.0.0 | 15.84 kB | 15.84 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | ttl-set | 1.0.0 | 4.61 kB | 9.69 kB | | path-to-regexp | 0.1.12 | 6.6 kB | 6.6 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | module-details-from-path | 1.0.3 | 4.47 kB | 4.47 kB |🤖 This report was automatically generated by heaviest-objects-in-the-universe |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #5253 +/- ##
==========================================
+ Coverage 81.09% 81.27% +0.18%
==========================================
Files 480 487 +7
Lines 21438 21709 +271
==========================================
+ Hits 17386 17645 +259
- Misses 4052 4064 +12 ☔ View full report in Codecov by Sentry. |
Datadog ReportBranch report: ✅ 0 Failed, 627 Passed, 0 Skipped, 8m 46.82s Total Time |
What does this PR do?
Closes #5233
Patches the
APIPromise
instance's_thenUnwrap
returned forchat.completion
calls.This is done through instance patching, as opposed to prototype patching, which is normal for this OpenAI instrumentation, but is becoming a bit hectic. Even though this support is temporary, we will want to rework how this instrumentation is done in general. More notes below.
Motivation
Temporary tracing support for the
client.beta.chat.completions
endpoint by request. This is a one-off, and this support will be removed once the functionality on the beta client (mainly structured output) is on the main client itself.This is the only
beta
method we will support tracing for.Additional Notes
We can't just patch
apiProm._thenUnwrap
and call it a day because_thenUnwrap
returns a newAPIPromise
, and we a) cannot return a normalPromise
in it's place, so we b) instead opt to patch theparse
method on that newAPIPromise
.This instrumentation really needs a refactor. We're doing a lot of nested instance patching that could be done better by maybe patching the base
APIPromise
along with theAPIClient
to hoist the base API information. I will try to do this in a follow-up PR, but for now, I believe just modifying the current implementation is the best approach.