Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add support for tags with OpenAI Instrumentation #63

Merged
merged 2 commits into from
Apr 26, 2024

Conversation

Granipouss
Copy link

@Granipouss Granipouss commented Apr 25, 2024

https://linear.app/chainlit/issue/ENG-1145/openai-instrumentation-with-metadata-and-tags

In the wrapper we have to remove extra arguments because OpenAI throws if we don't.

Documentation incoming...


This change is Reviewable

Copy link

linear bot commented Apr 25, 2024

ENG-1145 OpenAI Instrumentation with metadata and tags

import os
from literalai import LiteralClient
from openai import OpenAI

openai_client = OpenAI(
    api_key=)

literal_client = LiteralClient(
    api_key=)
literal_client.instrument_openai()

query = "What's 2+2?"
messages = [{"role": "user", "content": query}]
res = openai_client.chat.completions.create(
    messages=messages,
    model="gpt-4",
    extra_query={
        "user-category": "2"
    },
    #extra_body
    #extra_headers
)
print(res)

This is important for ppl only using the instrumentation. How do they pass in the user, metadata or tags?
Is this possible? Is this also an advantage for having an AI proxy?

DoD:

  • Recommendation on how to tackle this use case.

Copy link
Contributor

@willydouhard willydouhard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 2 of 2 files at r1, all commit messages.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @Granipouss)


literalai/instrumentation/openai.py line 192 at r1 (raw file):

                    step.input = {"content": generation.prompt}
                
                step.tags = kwargs.get("literal_tags", [])

I think the tags should go the generation and not the step


literalai/instrumentation/openai.py line 218 at r1 (raw file):

                    step.input = {"content": generation.prompt}

                step.tags = kwargs.get("literal_tags", [])

same here

@Granipouss Granipouss changed the title feat: Add support for tags & metadata with OpenAI Instrumentation feat: Add support for tags with OpenAI Instrumentation Apr 26, 2024
@Granipouss
Copy link
Author

Copy link
Contributor

@willydouhard willydouhard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r2, all commit messages.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @Granipouss)

Copy link
Contributor

@willydouhard willydouhard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @Granipouss)

Copy link
Author

@Granipouss Granipouss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @willydouhard)


literalai/instrumentation/openai.py line 218 at r1 (raw file):

Previously, willydouhard (Willy Douhard) wrote…

same here

Done.

@Granipouss Granipouss merged commit f18453a into main Apr 26, 2024
2 of 3 checks passed
@Granipouss Granipouss deleted the brendan/eng-1145-tags-metadata branch April 26, 2024 17:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants