Skip to content

Conversation

yanxi0830
Copy link
Contributor

@yanxi0830 yanxi0830 commented Nov 19, 2024

TL;DR

  • fix streaming for inference
methods:
      chat_completion: 
        type: http
        endpoint: post /alpha/inference/chat-completion
        streaming:
          skip: [ kotlin ]
          param_discriminator: stream
      completion: 
        type: http
        endpoint: post /alpha/inference/completion
        streaming:
          skip: [ kotlin ]
          param_discriminator: stream
      embeddings: post /alpha/inference/embeddings
python -m examples.inference.client localhost 5000

@yanxi0830 yanxi0830 marked this pull request as ready for review November 19, 2024 19:23
@yanxi0830 yanxi0830 merged commit af1b35c into main Nov 19, 2024
3 checks passed
@yanxi0830 yanxi0830 deleted the stainless_sync_oIDwVx branch November 19, 2024 19:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants