Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

server : handle content array in chat API #8449

Merged
merged 2 commits into from
Jul 12, 2024
Merged

server : handle content array in chat API #8449

merged 2 commits into from
Jul 12, 2024

Conversation

ggerganov
Copy link
Owner

@ggerganov ggerganov commented Jul 12, 2024 •
edited
Loading

fix #8367

Testing:

make -j && ./llama-server --hf-repo "Qwen/Qwen2-0.5B-Instruct-GGUF" --hf-file qwen2-0_5b-instruct-q2_k.gguf

curl \
  --request POST --url http://localhost:8080/v1/chat/completions \
  --header "Content-Type: application/json" \
  --data '{"temperature": 0.0, "messages": [ { "role": "system", "content": "You are a helpful assistant." }, { "role": "user", "content": [ { "type": "text", "text": "tell me a " }, { "type": "text", "text": "riddle" } ] } ] }'

ngxson
ngxson reviewed Jul 12, 2024
View reviewed changes
examples/server/utils.hpp Outdated Show resolved Hide resolved
@ggerganov @ngxson
Update examples/server/utils.hpp
8428a98 
Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>
@ggerganov ggerganov merged commit 4e24cff into master Jul 12, 2024
54 checks passed
@ggerganov ggerganov deleted the gg/server-chat-content branch July 12, 2024 11:48
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Jul 13, 2024
@ggerganov @ngxson
server : handle content array in chat API (ggerganov#8449)
cd3db81 
* server : handle content array in chat API

* Update examples/server/utils.hpp

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>

---------

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Jul 13, 2024
@ggerganov @ngxson
server : handle content array in chat API (ggerganov#8449)
07d457b 
* server : handle content array in chat API

* Update examples/server/utils.hpp

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>

---------

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>
Nexesenex pushed a commit to Nexesenex/croco.cpp that referenced this pull request Jul 15, 2024
@ggerganov @ngxson @Nexesenex
server : handle content array in chat API (ggerganov#8449)
3f77bcb 
* server : handle content array in chat API

* Update examples/server/utils.hpp

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>

---------

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ngxson ngxson ngxson approved these changes

Assignees
No one assigned
Labels
examples server
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

server : support content array in OAI chat API
2 participants
@ggerganov @ngxson