Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BFCL] Add DeepSeek-V2.5, DeepSeek-Coder-V2-Instruct-0724, DeepSeek-Coder-V2-Lite-Instruct, DeepSeek-V2-Chat-0628, DeepSeek-V2-Lite-Chat #697

Merged
merged 12 commits into from
Nov 26, 2024
Merged

[BFCL] Add DeepSeek-V2.5, DeepSeek-Coder-V2-Instruct-0724, DeepSeek-Coder-V2-Lite-Instruct, DeepSeek-V2-Chat-0628, DeepSeek-V2-Lite-Chat #697

merged 12 commits into from
Nov 26, 2024

Conversation

moonlight1431
Copy link
Contributor

@moonlight1431 moonlight1431 commented Oct 14, 2024 •
edited by HuanzhiMao
Loading

Add the following new models to the leaderboard:

  • deepseek-ai/DeepSeek-V2.5
  • deepseek-ai/DeepSeek-Coder-V2-Instruct-0724
  • deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct
  • deepseek-ai/DeepSeek-V2-Chat-0628
  • deepseek-ai/DeepSeek-V2-Lite-Chat

@moonlight1431
added deepseekv2 handler for DeepSeek-V2-Chat-0628,DeepSeek-V2-Chat, …
50c1222 
…DeepSeek-V2, DeepSeek-V2-Lite, DeepSeek-v2-Lite-Chat
@HuanzhiMao
Copy link
Collaborator

HuanzhiMao commented Oct 14, 2024 •
edited
Loading

Thanks for the PR @moonlight1431!
Could you update the deepseekHandler with DeepSeek-Coder-V2 as well? https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Instruct

@HuanzhiMao HuanzhiMao added the BFCL-New Model Add New Model to BFCL label Oct 14, 2024
@moonlight1431
Copy link
Contributor Author

added the DeepSeek-Coder-V2 to the deepseek handler!!

@ShishirPatil
Copy link
Owner

Thank you @moonlight1431
@HuanzhiMao I think once we fix the merge conflict in our latest handler map, this is good to go?

@HuanzhiMao HuanzhiMao changed the title added deepseekv2 handler for DeepSeek-V2-Chat-0628,DeepSeek-V2-Chat, … [BFCL] Add DeepSeek-V2, DeepSeek-V2-Lite, DeepSeek-Coder-V2-Instruct, DeepSeek-Coder-V2-Lite-Instruct Nov 9, 2024
@HuanzhiMao HuanzhiMao changed the title [BFCL] Add DeepSeek-V2, DeepSeek-V2-Lite, DeepSeek-Coder-V2-Instruct, DeepSeek-Coder-V2-Lite-Instruct [BFCL] Add DeepSeek-V2.5, DeepSeek-Coder-V2-Instruct-0724, DeepSeek-Coder-V2-Lite-Instruct, DeepSeek-V2-Chat-0628, DeepSeek-V2-Lite-Chat Nov 26, 2024
HuanzhiMao
HuanzhiMao approved these changes Nov 26, 2024
View reviewed changes
Copy link
Collaborator

@HuanzhiMao HuanzhiMao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ShishirPatil ShishirPatil merged commit 7d3bf66 into ShishirPatil:main Nov 26, 2024
This was referenced Nov 28, 2024
Merged
Merged
HuanzhiMao added a commit that referenced this pull request Dec 7, 2024
@HuanzhiMao
[BFCL] Leaderboard Update - 2024/12/06 (Checkpoint d7e52e5) (#800)
2dcccc1 
This PR updates the leaderboard to reflect the change in score due to
the following PR merge:

1. #747 
2. #770 
3. #768 
4. #750 
5. #763 
6. #772 
7. #777 
8. #778 
9. #786 
10. #787 
11. #697 
12. #718 
13. #755 
14. #796 
15. #789 
16. #804 
17. #808 
18. #809
19. #811 
20. #810 

Models were evaluated using checkpoint commit d7e52e5.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@HuanzhiMao HuanzhiMao HuanzhiMao approved these changes

Assignees
No one assigned
Labels
BFCL-New Model Add New Model to BFCL
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@moonlight1431 @HuanzhiMao @ShishirPatil