Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add observability plugin system #227

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

ajgray-stripe
Copy link

Description

See #203 for background; summary of that issue is that we'd like to have a somewhat more abstracted plugin system for observability/tracing so that we can have our own house-built telemetry plugin. I ultimately opted for the relatively lightweight option of just copying Langfuse's API for this PR; I wasn't confident enough in a refactor that was any more different.

In particular, the substantive changes made here are:

  • Creates an "observer" plugin system modeled off the existing plugin systems in this repo
  • That includes talking with profiles.yaml, pyproject-based entrypoint discovery for plugins, etc.
  • Pulling out the langfuse's integration observe_wrapper decorator into a method that calls all registered observers
  • Mostly moves current langfuse_wrapper.py code into observers/langfuse.py; I didn't really touch the code in there much. Also makes that plugin a default
    Review commit-by-commit for the logical flow here.

Testing

  • I wrote a few unit tests. I checked that both goosewide and exchange tests all passed locally.

I also did some manual testing, with a couple minor hacks to get around not actually being able to test fully end-to-end with Langfuse (the environment I'm using didn't make that easy+fast so I just did this instead).

  • With the following temporary patches on top of this branch:
    image
    disabling actually authenticating Langfuse

image
and instead just logging to a file for visibility

, as well as this profile:

image

we can see that Goose runs correctly, with startup messages showing that it's correctly (trying to) initialize the Langfuse client:

image

and we correctly log method calls.

image

  • Without the --tracing flag, goose runs without errors as expected:

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant