Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update media type and JSON handling in OpenAISpec #360

Merged
merged 3 commits into from
Nov 11, 2024

Conversation

fabigr8
Copy link
Contributor

@fabigr8 fabigr8 commented Nov 11, 2024

To comply with actual OpenAI API streaming responses i changed the content type and the json handling in the streaming implementation of openAI-spec see #359

Before submitting
  • Was this discussed/agreed via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs? --> Not applicable
  • Did you write any new necessary tests? --> not necessary

How does this PR impact the user?
As a user, I want to use LitServe to connect to third-party libraries or applications (such as Open WebUI) via openAI streaming API. This PR changes the LitServe OpenAI-spec to fully comply with the OAI streaming API and solve existiing connection issues.

What does this PR do?

Fixes #359

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in GitHub issues there's a high chance it will not be merged.

Did you have fun?

YES 💯

Copy link
Collaborator

@aniketmaurya aniketmaurya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thanks for the PR @fabigr8! Added a small comment.

src/litserve/specs/openai.py Outdated Show resolved Hide resolved
src/litserve/specs/openai.py Outdated Show resolved Hide resolved
Copy link

codecov bot commented Nov 11, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96%. Comparing base (719e037) to head (54177b4).
Report is 1 commits behind head on main.

Additional details and impacted files
@@         Coverage Diff         @@
##           main   #360   +/-   ##
===================================
  Coverage    96%    96%           
===================================
  Files        22     22           
  Lines      1408   1408           
===================================
  Hits       1345   1345           
  Misses       63     63           

fabigr8 and others added 2 commits November 11, 2024 15:50
Co-authored-by: Aniket Maurya <theaniketmaurya@gmail.com>
Co-authored-by: Aniket Maurya <theaniketmaurya@gmail.com>
@aniketmaurya aniketmaurya merged commit e23469c into Lightning-AI:main Nov 11, 2024
19 of 20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bug: OpenAI-Spec should send text/event-stream
3 participants