Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🔧 #195 #196: Set router ctx in stream chunks & handle end of stream in case of some errors #203

Merged
merged 4 commits into from
Apr 16, 2024

Conversation

roma-glushko
Copy link
Member

@roma-glushko roma-glushko commented Apr 15, 2024

Description

Changes

  • Passed RouterID and ModelID information in the chat stream messages
  • Introduced a new ChatStreamMessage type that joins both chunk and error messages. Removed unneeded context from provider chatStream structs
  • defined a set of possible error codes during chat streaming
  • started simplifying logging by using context-based loggers
  • Introduced finish_reason on the error schema

Testing

glide-chat-stream-demo.mov

…nd error message types & removed all unneeded context from stream objects & defined a set of possible error codes
Copy link

codecov bot commented Apr 15, 2024

Codecov Report

Attention: Patch coverage is 73.27586% with 31 lines in your changes are missing coverage. Please review.

Project coverage is 67.19%. Comparing base (9a0ee6d) to head (017b181).
Report is 1 commits behind head on develop.

Files Patch % Lines
pkg/routers/router.go 66.66% 17 Missing ⚠️
pkg/api/http/handlers.go 0.00% 5 Missing ⚠️
pkg/providers/openai/chat.go 25.00% 3 Missing ⚠️
pkg/telemetry/telemetry.go 0.00% 3 Missing ⚠️
pkg/providers/cohere/chat.go 0.00% 1 Missing ⚠️
pkg/providers/cohere/finish_reason.go 0.00% 1 Missing ⚠️
pkg/providers/openai/finish_reasons.go 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #203      +/-   ##
===========================================
- Coverage    67.34%   67.19%   -0.15%     
===========================================
  Files           78       78              
  Lines         3586     3573      -13     
===========================================
- Hits          2415     2401      -14     
- Misses        1043     1045       +2     
+ Partials       128      127       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@roma-glushko roma-glushko merged commit 4a9735c into develop Apr 16, 2024
10 checks passed
@roma-glushko roma-glushko deleted the 195-set-router-ctx-in-stream-chunks branch April 16, 2024 19:03
roma-glushko added a commit to EinStack/glide-py that referenced this pull request Apr 16, 2024
- adjusted all stream schemas to reflect changes in EinStack/glide#203
- raise client error if stream was interrupted by error
- introduced finish_reason filed on the error schema to indicate fatal/terminal errors
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

🔧 Signal end of chat stream in case of error 🔧 Set router context for stream response chunks
1 participant