[feat] support o3-mini #6570
[feat] support o3-mini #6570
Conversation
| @@ -61,7 +61,8 @@ RUN if [ -z "${RELEASE_TAG}" ]; then \ | |||
| fi && \ | |||
| wget https://github.com/${RELEASE_ORG}/openvscode-server/releases/download/${RELEASE_TAG}/${RELEASE_TAG}-linux-${arch}.tar.gz && \ | |||
| tar -xzf ${RELEASE_TAG}-linux-${arch}.tar.gz && \ | |||
| mv -f ${RELEASE_TAG}-linux-${arch} ${OPENVSCODE_SERVER_ROOT} && \ | |||
| if [ -d "${OPENVSCODE_SERVER_ROOT}" ]; then rm -rf "${OPENVSCODE_SERVER_ROOT}"; fi && \ | |||
| mv ${RELEASE_TAG}-linux-${arch} ${OPENVSCODE_SERVER_ROOT} && \ | |||
There was a problem hiding this comment.
Is this o3-mini related?
There was a problem hiding this comment.
This is not, but it fixes an issue in main when running evals. We could keep them together in this PR or I can merge this on a separate PR
There was a problem hiding this comment.
Cool, I’m just a passerby who recently tried to add o3-mini support to this project. While working on it, I came across this PR and thought I’d check it out. Keeping a clean Git history is usually helpful for troubleshooting and makes it easier for others to jump in, so I just wanted to share that thought. Of course, it’s totally up to you as the Collaborator. Thanks for reading!
|
o3-mini seems to be missing on missing on |
Yep! We will add it to the verified model list (or even as one of the default model) once eval finishes and turns favorably |
|
Okay official result for o3-mini after 4 runs.. The cost for 4 runs is about 1257.39, which means 314.35 on average per run. So, conclusion: cost slightly less than sonnet and performs slightly weaker than Claude.
|
| @@ -148,17 +165,11 @@ def __init__( | |||
| base_url=self.config.base_url, | |||
| api_version=self.config.api_version, | |||
| custom_llm_provider=self.config.custom_llm_provider, | |||
| max_tokens=self.config.max_output_tokens, | |||
| max_completion_tokens=self.config.max_output_tokens, | |||
There was a problem hiding this comment.
This is necessary for o3-mini, but I wonder, does it work with the others like before? litellm translates max_tokens, but it might be safer to add max_completion_tokens only for reasoning models.
There was a problem hiding this comment.
Thank you for all the evals, this is exciting! Interesting times ahead.
Not sure, but I am a bit concerned that max_completion_tokens is newer and intended for reasoning models. It might be safer to set it only for them for now? I didn't find something clear on litellm.
I looked through LiteLLM Repo, and it seems LiteLLM does translate them all: https://github.com/search?q=repo%3ABerriAI%2Flitellm+max_completion_tokens&type=code&p=2 For anthropic it didn't make any differences at all: Maybe let's merge it and use it for a while to see if issue comes up? |
|
OK let's merge then! |
|
(For sanity check - I build from this branch and try running claude, there's no issue - probably safe to merge now |
End-user friendly description of the problem this fixes or functionality that this introduces
Give a summary of what the PR does, explaining any non-trivial design decisions
Link of any specific issues this addresses
To run this PR locally, use the following command: