Add Command R chat template #6650
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
We're missing test case for this template. Please follow the full procedure from this article to add a new template. |
|
Thanks for submitting this, eagerly awaiting it's approval!! |
…tion to trim whitespaces
Chat template test for command-R is now added. |
|
Works for me! Had to drop my context to 2k on my RTX 4090 to get it working in IQ_4_XS quant. |
ngxson
approved these changes
Apr 14, 2024
There was a problem hiding this comment.
LGTM. Thank you!
The new template is also added to wiki: https://github.com/ggerganov/llama.cpp/wiki/Templates-supported-by-llama_chat_apply_template
tybalex
pushed a commit
to rubra-ai/tools.cpp
that referenced
this pull request
Apr 17, 2024
* Add chat template for command-r model series * Fix indentation * Add chat template test for command-r models and update the implementation to trim whitespaces * Remove debug print
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Added the chat template for c4ai-command-r-v01 and c4ai-command-r-plus in
llama_chat_apply_template_internalso that we can use--chat-template command-roption when running command-r models using llama.cpp's openai api server.The command-r chat template follows this format: <BOS_TOKEN><|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>{system}<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|>{prompt}<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>{response}