update api to "tools" concept #105
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* main: test tag updates - add more specific tags for enabling/disabling certain live tests fixes for example and CI refactor streamed response handling for anthropic - buffers incomplete streamed data - more tests - refactored decode logic - refactored Utils.handle_stream_fn to have discrete function steps - REMAINING: Refactor ChatGPT decoding to follow the Anthropic approach cleanup Changes from feedback Minor tweaks Add support for Anthropic
- added MessagePart - updated message docs - implemented ChatGPT support for "gpt-4-vision-preview" model passing image data for user messages
- refactored Anthropic to use for_api/1 functions for LangChain structs - added "options" to MessagePart. Anthropic requires media type to be sent with image parts. - Anthropic raises exception on multiple system messages
* main: remove message wrap to list for Anthropic update Utils.fire_callback - allow for a single message. Don't assume a list. - added live full-stack tests to anthropic Revert "mix format" mix format wrap message support sending the auth api key differently - sends it both ways - Azure OpenAI requires an "api-key" header - Standard OpenAI requires auth bearer header
- before refactor for new design
- working on tool_calls message changes - adding "name" to message for assistant and user. New OpenAI feature.
- message structure fixes - test updates - OpenAI API support improved - ToolCall "complete" function - removed early tool_call support from UserContentPart Working on OpenAI support
- rescues errors - wraps return values in :ok/:error tuples - Function.new validates that an Elixir function is assigned and validates it is arity 2
- execute async functions together - execute sync-only functions - return tool_call hallucination errors - updated tests and added new tests
- updated anthropic for parsing to ToolCall structs - still need more tests
- function support tested - function parameters - multiple tool_result messages get collapsed into a single user too_result message - much expanded test coverage
- helps identify incorrect developer usage much quicker
- restoring a conversation should not include empty tool_calls for an assistant
- track behavior differences between LLMs
- test fixes and cleanup
* main: refactored ChatGoogleAI tests to use pattern matching more Add llama-3 chat template
* me-update-api-to-tools: refactored ChatGoogleAI tests to use pattern matching more Add llama-3 chat template
- applies to how Google uses content
- updated tests
This was referenced Apr 27, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
All tests are passing, but I expect there to be some issues.
This is a breaking change for the message structures. It now supports LLMs that take multi-modal inputs. Meaning we can mix text and images together in a prompt and use the LLM to analyze the images.
Adds:
LangChain.Message.ContentPart- used for User messages and multi-modal support. Google's AI assistant can return multiple parts as well.LangChain.Message.ToolCall- an assistant can request multiple tool calls in the same message.LangChain.Message.ToolResult- the system's answer to a ToolCall. It adds anis_errorboolean flag. This an be helpful in the UI, but Anthropic specifically wants it.Changes:
:functionand:function_callare removed. The equivalent of afunction_callis expressed by an:assistantrole making one or more ToolCall requests. The:functionwas the system's answer to a function call. This is now in the:toolrole.:toolwas added. Atoolmessage contains one or moreToolResultmessages.Changes: