Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Replace auto_tool_execution with execute_tools (just step 1) #873

Closed
wants to merge 3 commits into from
Closed

Replace auto_tool_execution with execute_tools (just step 1) #873

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
6e136ad
Rename auto_tools_execute
mattlindsey Nov 12, 2024
9cb01ed
Fix and update README
mattlindsey Nov 12, 2024
0a4eca1
Fix README
mattlindsey Nov 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -512,7 +512,9 @@ assistant.add_message_and_run!(
messages = assistant.messages

# Run the assistant with automatic tool execution
assistant.run(auto_tool_execution: true)
assistant.run()
# OR run the assistant without executing tools
assistant.run(execute_tools: false)

# If you want to stream the response, you can add a response handler
assistant = Langchain::Assistant.new(
Expand All @@ -524,7 +526,7 @@ assistant = Langchain::Assistant.new(
# print(response_chunk.inspect)
end
assistant.add_message(content: "Hello")
assistant.run(auto_tool_execution: true)
assistant.run()
```

Note that streaming is not currently supported for all LLMs.
Expand Down
2 changes: 1 addition & 1 deletion examples/assistant_chat.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ def prompt_for_message
break
end

assistant.add_message_and_run content: user_message, auto_tool_execution: true
assistant.add_message_and_run content: user_message, execute_tools: true
puts assistant.messages.last.content
end
rescue Interrupt
Expand Down
28 changes: 16 additions & 12 deletions lib/langchain/assistant.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -129,17 +129,21 @@ def add_messages(messages:)

# Run the assistant
#
# @param auto_tool_execution [Boolean] Whether or not to automatically run tools
# @param execute_tools [Boolean] Whether or not to automatically run tools
# @return [Array<Langchain::Message>] The messages
def run(auto_tool_execution: false)
def run(execute_tools: false)
if messages.empty?
Langchain.logger.warn("#{self.class} - No messages to process")
@state = :completed
return
end

@state = :in_progress
@state = handle_state until run_finished?(auto_tool_execution)
if !execute_tools
@state = :completed
Copy link
Contributor Author

@mattlindsey mattlindsey Nov 12, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the part that confuses me. Why are we even calling run if we don't want to do anything?
@andreibondarev If you want to handle this issue yourself, that's totally fine. I don't want to clutter up the code.

I think maybe what's needed is to separate `handle_user_or_tool_message' into 2 methods?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to change the logic if we're merely renaming things?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's already a method called execute_tools so it's not that simple.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can rename that private method to _execute_tools then.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For some reason, doing that I get the error below, then I get lost in the details. I'm probably missing something simple.

OpenAI HTTP Error (spotted in ruby-openai 7.1.0): {"error"=>{"message"=>"An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. The following tool_call_ids did not have response messages: call_ZM6cJs8AMnU0JcEdFErekxlj", "type"=>"invalid_request_error", "param"=>"messages.[3].role", "code"=>nil}}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh wait. You get that error also with the current code if you do this sequence, so it's not an issue with the rename:

a.add_message_and_run(content: "What's the weather in Boston?", auto_tool_execution: false)
a.add_message_and_run(content: "What's the weather in Chicago?", auto_tool_execution: true)

That is confusing. Wasn't there something in the README before about running tools manually? Should that sequence really cause an error?

I'll close these PRs and can do the simple rename later today or tomorrow when hopefully it will make more sense to me. LOL.

else
@state = :in_progress
@state = handle_state until run_finished?(execute_tools)
end

messages
end
Expand All @@ -148,25 +152,25 @@ def run(auto_tool_execution: false)
#
# @return [Array<Langchain::Message>] The messages
def run!
run(auto_tool_execution: true)
run(execute_tools: true)
end

# Add a user message and run the assistant
#
# @param content [String] The content of the message
# @param auto_tool_execution [Boolean] Whether or not to automatically run tools
# @param execute_tools [Boolean] Whether or not to automatically run tools
# @return [Array<Langchain::Message>] The messages
def add_message_and_run(content: nil, image_url: nil, auto_tool_execution: false)
def add_message_and_run(content: nil, image_url: nil, execute_tools: false)
add_message(content: content, image_url: image_url, role: "user")
run(auto_tool_execution: auto_tool_execution)
run(execute_tools: execute_tools)
end

# Add a user message and run the assistant with automatic tool execution
#
# @param content [String] The content of the message
# @return [Array<Langchain::Message>] The messages
def add_message_and_run!(content: nil, image_url: nil)
add_message_and_run(content: content, image_url: image_url, auto_tool_execution: true)
add_message_and_run(content: content, image_url: image_url, execute_tools: true)
end

# Submit tool output
Expand Down Expand Up @@ -233,12 +237,12 @@ def validate_tool_choice!(tool_choice)

# Check if the run is finished
#
# @param auto_tool_execution [Boolean] Whether or not to automatically run tools
# @param execute_tools [Boolean] Whether or not to automatically run tools
# @return [Boolean] Whether the run is finished
def run_finished?(auto_tool_execution)
def run_finished?(execute_tools)
finished_states = [:completed, :failed]

requires_manual_action = (@state == :requires_action) && !auto_tool_execution
requires_manual_action = (@state == :requires_action) && !execute_tools
finished_states.include?(@state) || requires_manual_action
end

Expand Down
40 changes: 20 additions & 20 deletions spec/langchain/assistant/assistant_spec.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -223,7 +223,7 @@
}
end

context "when auto_tool_execution is false" do
context "when execute_tools is false" do
before do
allow(subject.llm).to receive(:chat)
.with(
Expand All @@ -241,22 +241,22 @@
end

it "runs the assistant" do
subject.run(auto_tool_execution: false)
subject.run(execute_tools: false)

expect(subject.messages.last.role).to eq("assistant")
expect(subject.messages.last.tool_calls).to eq([raw_openai_response["choices"][0]["message"]["tool_calls"]][0])
end

it "records the used tokens totals" do
subject.run(auto_tool_execution: false)
subject.run(execute_tools: false)

expect(subject.total_tokens).to eq(109)
expect(subject.total_prompt_tokens).to eq(91)
expect(subject.total_completion_tokens).to eq(18)
end
end

context "when auto_tool_execution is true" do
context "when execute_tools is true" do
let(:raw_openai_response2) do
{
"id" => "chatcmpl-96P6eEMDDaiwzRIHJZAliYHQ8ov3q",
Expand Down Expand Up @@ -299,7 +299,7 @@
end

it "runs the assistant and automatically executes tool calls" do
subject.run(auto_tool_execution: true)
subject.run(execute_tools: true)

expect(subject.messages[-2].role).to eq("tool")
expect(subject.messages[-2].content).to eq("4.0")
Expand All @@ -309,7 +309,7 @@
end

it "records the used tokens totals" do
subject.run(auto_tool_execution: true)
subject.run(execute_tools: true)

expect(subject.total_tokens).to eq(134)
expect(subject.total_prompt_tokens).to eq(121)
Expand Down Expand Up @@ -590,7 +590,7 @@
}
end

context "when auto_tool_execution is false" do
context "when execute_tools is false" do
before do
allow(subject.llm).to receive(:chat)
.with(
Expand All @@ -607,22 +607,22 @@
end

it "runs the assistant" do
subject.run(auto_tool_execution: false)
subject.run(execute_tools: false)

expect(subject.messages.last.role).to eq("assistant")
expect(subject.messages.last.tool_calls).to eq([raw_mistralai_response["choices"][0]["message"]["tool_calls"]][0])
end

it "records the used tokens totals" do
subject.run(auto_tool_execution: false)
subject.run(execute_tools: false)

expect(subject.total_tokens).to eq(109)
expect(subject.total_prompt_tokens).to eq(91)
expect(subject.total_completion_tokens).to eq(18)
end
end

context "when auto_tool_execution is true" do
context "when execute_tools is true" do
let(:raw_mistralai_response2) do
{
"id" => "chatcmpl-96P6eEMDDaiwzRIHJZAliYHQ8ov3q",
Expand Down Expand Up @@ -664,7 +664,7 @@
end

it "runs the assistant and automatically executes tool calls" do
subject.run(auto_tool_execution: true)
subject.run(execute_tools: true)

expect(subject.messages[-2].role).to eq("tool")
expect(subject.messages[-2].content).to eq("4.0")
Expand All @@ -674,7 +674,7 @@
end

it "records the used tokens totals" do
subject.run(auto_tool_execution: true)
subject.run(execute_tools: true)

expect(subject.total_tokens).to eq(134)
expect(subject.total_prompt_tokens).to eq(121)
Expand Down Expand Up @@ -938,7 +938,7 @@
}
end

context "when auto_tool_execution is false" do
context "when execute_tools is false" do
before do
allow(subject.llm).to receive(:chat)
.with(
Expand All @@ -952,14 +952,14 @@

it "runs the assistant" do
subject.add_message(role: "user", content: "Please calculate 2+2")
subject.run(auto_tool_execution: false)
subject.run(execute_tools: false)

expect(subject.messages.last.role).to eq("model")
expect(subject.messages.last.tool_calls).to eq([raw_google_gemini_response["candidates"][0]["content"]["parts"]][0])
end
end

context "when auto_tool_execution is true" do
context "when execute_tools is true" do
let(:raw_google_gemini_response2) do
{
"candidates" => [
Expand Down Expand Up @@ -999,7 +999,7 @@
subject.add_message(role: "user", content: "Please calculate 2+2")
subject.add_message(role: "model", tool_calls: raw_google_gemini_response["candidates"][0]["content"]["parts"])

subject.run(auto_tool_execution: true)
subject.run(execute_tools: true)

expect(subject.messages[-2].role).to eq("function")
expect(subject.messages[-2].content).to eq("4.0")
Expand Down Expand Up @@ -1146,7 +1146,7 @@
end
end

context "when auto_tool_execution is false" do
context "when execute_tools is false" do
before do
allow(subject.llm).to receive(:chat)
.with(
Expand All @@ -1160,7 +1160,7 @@

it "runs the assistant" do
subject.add_message(role: "user", content: "Please calculate 2+2")
subject.run(auto_tool_execution: false)
subject.run(execute_tools: false)

expect(subject.messages.last.role).to eq("assistant")
expect(subject.messages.last.tool_calls).to eq([raw_anthropic_response["content"].last])
Expand All @@ -1178,7 +1178,7 @@
end
end

context "when auto_tool_execution is true" do
context "when execute_tools is true" do
let(:raw_anthropic_response2) do
{
"role" => "assistant",
Expand Down Expand Up @@ -1229,7 +1229,7 @@
tool_calls: [raw_anthropic_response["content"].last]
)

subject.run(auto_tool_execution: true)
subject.run(execute_tools: true)

expect(subject.messages[-2].role).to eq("tool_result")
expect(subject.messages[-2].content).to eq("4.0")
Expand Down
Loading