Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add stream_options property to get token usage. #180

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

dominic-deantonio
Copy link

@dominic-deantonio dominic-deantonio commented Jun 7, 2024

Added the ability to receive token usage based on this object in the OpenAI documentation:

https://platform.openai.com/docs/api-reference/chat/streaming

Added an optional parameter called streamOptions to the createStream interface which passes the aforementioned body parameter to the chat endpoint which enables receiving the optional usage statistics.

Added tests as well which succeeded during my tests. Some of the new tests might be considered unnecessary and I will be happy to change them if necessary.

Unrelated to the changes I made - I will note the following expects failed so I commented them out during my testing - they don't seem related my code changes:

    expect(streamEvent.choices.first.delta.content,
          isA<List<OpenAIChatCompletionChoiceMessageContentItemModel>?>());
      expect(streamEvent.choices.first.delta.content?.first?.text,
          isA<String?>());

Kept getting bad state: no element - presumably because we are using the .first getter here.

Thanks!

This PR is a solution to this issue: #155 (comment)

@imhuwq
Copy link

imhuwq commented Jun 22, 2024

Good job, look for this feature too, and I think this PR is what I want.
When will this be merged and released?

@dominic-deantonio
Copy link
Author

dominic-deantonio commented Jun 22, 2024

Yeah no response from the maintainer so far so not sure what to do. I'm just using my own fork for now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants