feat: Add converter to support outbound calls to OpenAI servers that want tool not function
#619
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cpacker
changed the title
tool not functiontool not function
cpacker
force-pushed
the
enable-tool-naming-on-openai-apis
branch
from
69495d3 to
8e11cad
Compare
|
Force push rebased on 0.2.11 |
|
Deprecated by 0.3.0 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Please describe the purpose of this pull request.
The function naming (
function/function_call/functionrole) is deprecated in OpenAI API and partially (?) deprecated in Azure OpenAI. OpenAI and Azure both still support the old naming methodology, but newer "OpenAI API-compatible" services that also allow function calling (eg Anyscale) don't support the old naming scheme and wanttoolonly.This PR adds a converter function that goes from the old format to the new one. This is done on-the-fly, ie we still store messages in the functions style but we modify it on the way out to the OpenAI API server.
@sarahwooders eventually we probably want to migrate the internal representations to the
tool_callformat, but I'm not sure how important it is for now, since unless we want to have parallel function calling support nothing really changes and these light patches work fine (AFAIK).How to test
Try using MemGPT with a non-OpenAI OpenAI-compatible endpoint that has function calling, but only the
toolversion of function calling (eg Anyscale).Have you tested this PR?
Yes, seems to work OK with Anyscale.