feat: add observability plugin system #227
Open
+285
−156
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
See #203 for background; summary of that issue is that we'd like to have a somewhat more abstracted plugin system for observability/tracing so that we can have our own house-built telemetry plugin. I ultimately opted for the relatively lightweight option of just copying Langfuse's API for this PR; I wasn't confident enough in a refactor that was any more different.
In particular, the substantive changes made here are:
profiles.yaml
,pyproject
-based entrypoint discovery for plugins, etc.observe_wrapper
decorator into a method that calls all registered observerslangfuse_wrapper.py
code intoobservers/langfuse.py
; I didn't really touch the code in there much. Also makes that plugin a defaultReview commit-by-commit for the logical flow here.
Testing
I also did some manual testing, with a couple minor hacks to get around not actually being able to test fully end-to-end with Langfuse (the environment I'm using didn't make that easy+fast so I just did this instead).
disabling actually authenticating Langfuse
and instead just logging to a file for visibility
, as well as this profile:
we can see that Goose runs correctly, with startup messages showing that it's correctly (trying to) initialize the Langfuse client:
and we correctly log method calls.
--tracing
flag, goose runs without errors as expected: