Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Code Llama Support and Fix empty system prompt for llama 2 #2326

Merged
merged 6 commits into from
Aug 28, 2023

Conversation

woshiyyya
Copy link
Contributor

@woshiyyya woshiyyya commented Aug 27, 2023

Why are these changes needed?

Integrate codellama/CodeLlama-7b-Instruct-hf into FastChat. (Tested with the 7b model, but should also work for the larger models.)

(Minor fix: Handled the case when system_prompt == "" for SeparatorStyle.LLAMA2.)

Example usage:

# Run with system message
python3 -m fastchat.serve.cli --model-path codellama/CodeLlama-7b-Instruct-hf --conv-system-msg "Provide answers in Javascript"

# Run without system message
python3 -m fastchat.serve.cli --model-path codellama/CodeLlama-7b-Instruct-hf

Reference: Llama 2 learns to code

Related issue number (if applicable)

Closes #2309

Checks

  • I've run format.sh to lint the changes in this PR.
  • I've included any doc changes needed.
  • I've made sure the relevant tests are passing (if applicable).

Signed-off-by: woshiyyya <xiaoyunxuan1998@gmail.com>
Signed-off-by: woshiyyya <xiaoyunxuan1998@gmail.com>
@merrymercy merrymercy force-pushed the main branch 3 times, most recently from bf7aa7e to a81a04c Compare August 28, 2023 01:36
docs/model_support.md Outdated Show resolved Hide resolved
fastchat/conversation.py Outdated Show resolved Hide resolved
fastchat/conversation.py Outdated Show resolved Hide resolved
woshiyyya and others added 3 commits August 28, 2023 01:41
Signed-off-by: woshiyyya <xiaoyunxuan1998@gmail.com>
@merrymercy merrymercy changed the title Add Code Llama Support Add Code Llama Support and Fix empty system prompt for llama 2 Aug 28, 2023
@merrymercy merrymercy merged commit ac9321c into lm-sys:main Aug 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support codellama
2 participants