Clement/eng 2181 otel exporter #157
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
clementsirieix
force-pushed
the
clement/eng-2181-otel-exporter
branch
from
51602d3 to
95cdb03
Compare
clementsirieix
commented
Dec 16, 2024
| @@ -119,6 +128,13 @@ def instrument_llamaindex(self): | |||
|
|
|||
| instrument_llamaindex(self.to_sync()) | |||
|
|
|||
| def initialize(self): | |||
| with redirect_stdout(io.StringIO()): | |||
There was a problem hiding this comment.
Traceloop is verbose so this is aimed to prevent explicit print statement.
| root_run = active_root_run_var.get() | ||
| parent = active_steps_var.get()[-1] if active_steps_var.get() else None | ||
|
|
||
| Traceloop.set_association_properties( |
There was a problem hiding this comment.
This is not an upsert so we need to maintain explicit calls.
|
|
||
| Traceloop.set_association_properties( | ||
| { | ||
| "literal.thread_id": str(thread.id) if thread else "None", |
There was a problem hiding this comment.
Can only store strings
literalai/exporter.py
Outdated
Comment on lines
59
to
66
| # # TODO: Add generation promptid | ||
| # # TODO: Add generation variables | ||
| # # TODO: Check missing variables | ||
| # # TODO: ttFirstToken | ||
| # # TODO: duration | ||
| # # TODO: tokenThroughputInSeconds | ||
| # # TODO: Add tools | ||
| # # TODO: error check with gemini error |
There was a problem hiding this comment.
Haven't found a proper way to handle those as of now:
- promptid and variables could be set manually?
- missing variables like
frequency_penaltyare just not tracked by Traceloop so they do not appear in the span attributes, could only found basic support in their issue. - ttFirstToken, duration and tokenThroughputInSeconds could not find a clear alternative
- tools to test
There was a problem hiding this comment.
Could not find a way to handle gemini and ttFirstToken
clementsirieix
force-pushed
the
clement/eng-2181-otel-exporter
branch
from
deca886 to
e09d261
Compare
clementsirieix
force-pushed
the
clement/eng-2181-otel-exporter
branch
from
327ffa3 to
1cdace9
Compare
| @@ -549,6 +566,21 @@ def __enter__(self) -> Step: | |||
|
|
|||
| if active_root_run_var.get() is None and self.step_type == "run": | |||
| active_root_run_var.set(self.step) | |||
| Traceloop.set_association_properties( | |||
There was a problem hiding this comment.
if otel is not initialized this wont be an issue?
There was a problem hiding this comment.
Yes it can be safely called even without initialization 👍
|
As discussed yesterday:
|
tica3467
approved these changes
Jan 3, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
initializemethod to the clientset_propertiesmethod for customizationHow to test?
Test deprecations
Tests
LITERAL_API_URL="http://localhost:3000" LITERAL_API_KEY="my-initial-api-key" python3 examples/streaming.pyLITERAL_API_URL="http://localhost:3000" LITERAL_API_KEY="my-initial-api-key" python3 examples/langchain_variable.pyLITERAL_API_URL="http://localhost:3000" LITERAL_API_KEY="my-initial-api-key" python3 examples/multimodal.pyLITERAL_API_URL="http://localhost:3000" LITERAL_API_KEY="my-initial-api-key" python3 examples/llamaindex_workflow.py