langgraph/tutorials/multi_agent/agent_supervisor/

[giscus[bot]]

langgraph/tutorials/multi_agent/agent_supervisor/

Build language agents as graphs

https://langchain-ai.github.io/langgraph/tutorials/multi_agent/agent_supervisor/

[wsligter]

Question: Why are we passing 2 system messages? Or are we not?
or is this a bug and should the second 'system' role be changed to 'user'?

prompt = ChatPromptTemplate.from_messages(
[
("system", system_prompt),
MessagesPlaceholder(variable_name="messages"),
(
"system",
"Given the conversation above, who should act next?"
" Or should we FINISH? Select one of: {options}",
),
]
).partial(options=str(options), members=", ".join(members))

[MinLee0210]

You are correct, and I have encountered a situation that seems problematic.

In detail, I attempted to change the LLM-core of the supervisor to Gemini. However, it failed to execute properly when two consecutive 'system' prompts were placed in the sequence.

[alexandrebcaruso]

1 - system prompt to instruct the agent what it is and what it does;
2 - human request;
3 - system prompt to look at the human request and decide which agent should act.

that's how it is supposed to work.

[cris-m]

is that even correct. it will throw an error:

/usr/local/lib/python3.10/dist-packages/langchain_anthropic/chat_models.py in _format_messages(messages)
    176         if message.type == "system":
    177             if i != 0:
--> 178                 raise ValueError("System message must be at beginning of message list.")
    179             if isinstance(message.content, list):
    180                 system = [

ValueError: System message must be at beginning of message list.

[jacktang]

function_def = {
    "name": "route",
    "description": "Select the next role.",
    "parameters": {
        "title": "routeSchema",
        "type": "object",
        "properties": {
            "next": {
                "title": "Next",
                "anyOf": [
                    {"enum": options},
                ],
            }
        },
        "required": ["next"],
    },
}

supervisor_chain = (
    prompt
    | llm.bind_functions(functions=[function_def], function_call="route")
    | JsonOutputFunctionsParser()
)

Is there any simple implementation for route?

[ToxicHu]

hello, have you found the implementation? i have the same question

[marcammann]

I think the idea is that there is no implementation, only a schema. The JsonOutputFunctionsParser is then used to get the argument that was passed into the function. I think technically this should be do-able by just requesting a structured output response format from the chain, i.e. { "next": "..." } instead of going through a function call?

[vaibhavp4]

How is the agent invoked with state without providing specific keys?

[woodswift]

Printing that state value in mid of run shows it is just like what get_state() returns. But this is still a valid question requiring clarification.

[intern162]

Can someone please tell me how/where is the supervisor giving the relevant prompt to the agents? Like who is telling the agent what to do?

[gtnpromtior]

each agent have it's own prompt.

meta_campaigns_agent = create_react_agent(
    llm_mini,
    tools=initialize_tools_v2(),
    state_modifier=meta_prompt,
)

[alysonhower]

laughing in "escreva um relatório sobre pikas"

[Dumplingisabeast]

In the following, create_agent mentions about "agent_scratchpad" but i can't find it mentioned anywhere in the subsequent code. Can someone help me understand how it works? Is it necessary?

def create_agent(llm: ChatOpenAI, tools: list, system_prompt: str):
# Each worker node will be given a name and some tools.
prompt = ChatPromptTemplate.from_messages(
[
(
"system",
system_prompt,
),
MessagesPlaceholder(variable_name="messages"),
MessagesPlaceholder(variable_name="agent_scratchpad"),
]
)
agent = create_openai_tools_agent(llm, tools, prompt)
executor = AgentExecutor(agent=agent, tools=tools)
return executor

[intern162]

It is necessary for using create_openai_tools_agent, without that it is not necessary to use one

[Smone5]

I had a problem executing the Python code portion of this. Changing this code helped me:

This executes code locally, which can be unsafe

#python_repl_tool = PythonREPLTool()

from langchain_experimental.utilities import PythonREPL
from langchain_core.tools import tool
repl = PythonREPL()
@tool
def python_repl(code: Annotated[str, "The python coder to execute python tasks"],):
"""Use this to execute python code. If you want to see the output of a value,
you should print if out with 'print(....)'. This is visiable to the user."""

try:
    result= repl.run(code)
except Exception as e:
    return f"Failed to execute. Error: {repr(e)}"
result_str = f"Successfully executed:\n```python\n{code}\n```\nStdout: {result}"
return (
    result_str + "\n\nIf you have completed all tasks, respond with FINAL ANSWER."
)

[grantwag]

Could this example please be updated to not use the legacy create_openai_tools_agent() and AgentExecutor().

[westernspion]

For those of you here using AzureChatOpenAI - parallel_tool_calls is unsupported and is slated for release in Sept'24 Azure/azure-rest-api-specs#29545

Otherwise, you'll run into this on any complex query that triggers your supervisory agent to call >1 sub-agent in parallel

BadRequestError: Error code: 400 - {'error': {'message': "An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. The following tool_call_ids did not have response messages: call_...`

Anyone with a solution to disable this, or better, handle parallel tool calls, would be excellent.

Otherwise, the only alternative is to rearchitect the overall workflow in such a way that it can't call more than 1 tool at a time.

[alexandrebcaruso]

openai.BadRequestError: Error code: 400 - {'object': 'error', 'message': "[{'type': 'extra_forbidden', 'loc': ('body', 'function_call'), 'msg': 'Extra inputs are not permitted', 'input': {'name': 'route'}}, {'type': 'extra_forbidden', 'loc': ('body', 'functions'), 'msg': 'Extra inputs are not permitted', 'input': [{'name': 'route', 'description': 'Select the next role.', 'parameters': {'title': 'routeSchema', 'type': 'object', 'properties': {'next': {'title': 'Next', 'anyOf': [{'enum': ['FINISH', 'Ageant 1', 'Agent 2']}]}}, 'required': ['next']}}]}]", 'type': 'BadRequestError', 'param': None, 'code': 400}

I'm using Lamma 3 with openai and I have a custom base_url. I have single agents with tools running fine (with create_sql_agents).

the problem:
multi-agents using bind_tools or bind_functions (following exactly what this tutorial shows).

thanks!

[alexandrebcaruso]

the issue is probably with the custom base URL. somehow it doesn't accept "functions" or "tools" parameters, although it is just a wrapper around openai.

I don't get this issue when I use local lamma.cpp with llama 3.1 model.

[waywooKwong]

" bind_functions " are only supported in ChatOpenAI,
bind_tools are supported in different models,
llama3.1 make sense because llama3.1 has been fine-tuned for tool use,
as well as llama3-groq-tool-use

these two models are given in the tutorial "Tool calling" part, i find it updates after llama3.1 coming out.
https://python.langchain.com/v0.2/docs/integrations/chat/ollama/

[woodswift]

Could you provide an example to show the supervisor can manage two agents all together? The current examples only shows that supervisor can invoke one of them but not prove they can collaborate together. Thanks!

[alexandrebcaruso]

I think the idea of the implementation on this tutorial is to prove the agents can collaborate via a supervisor. did you check the previous tutorial? is about exactly that: multi-agent direct collaboration. hope it helps!

[waywooKwong]

In router-chain building,
"function_def" and then "bind_functions" are only supported in ChatOpenAI, so far I haven't checked out any other wrapper which contains this function.
But i find that "bind_tools" are supported by different models in different way.
I suggest import model through "from langchain_community import ChatOllama(or your model)" and checkout if the wrapper contains bind_tools

[alexandrebcaruso]

Llama.cpp with Llama 3.1 supports bind_tools

[alexandrebcaruso]

it seems that the code was updated, and now the supervisor_chain is wrapped in a function def:

def supervisor_agent(state):
    supervisor_chain = (
        prompt
        | llm.with_structured_output(routeResponse)
    )
    return supervisor_chain.invoke(state)

I guess the node definition should also be udpated?
workflow.add_node("supervisor", supervisor_chain)

[RafaelRViana]

This function is not being used in no other part of the code. I don't know why it was created, maybe it is used in another example.

If you remove function and use code (without invoke) outside function scope. This example works well.
supervisor_chain = supervisor_chain = (
prompt
| llm.with_structured_output(routeResponse)
)
workflow.add_node("supervisor", supervisor_chain)

[TaisukeIto]

It seems there's an inconsistency due to a version update of the LangGraph package or something.
The program in this tutorial doesn't work.
Message integration doesn't seem to work properly.

[TaisukeIto]

Sorry, I posted to the wrong place.
It's the Customer Support Bot that's not working.

[avfranco-br]

At least for me, in this code def supervisor_agent(state): supervisor_chain = ( prompt | llm.with_structured_output(routeResponse) ) return supervisor_chain.invoke(state), returning supervisor_chain.invoke(state) was causing my the graph enter in a recursion loop. Removing the .invoke(state) it worked.

[inoue0426]

I downloaded this file and try with Ollama with below code.

# llm = ChatOpenAI(
#     api_key="ollama",
#     model="llama3.1",
#     base_url="http://localhost:11434/v1",
# )

from langchain_ollama import ChatOllama
llm = ChatOllama(model='llama3.1')

Then I got below error.

File ~/miniconda3/envs/multi/lib/python3.11/site-packages/langgraph/utils/runnable.py:159, in RunnableCallable.invoke(self, input, config, **kwargs)
    157     context = copy_context()
    158     context.run(_set_config_context, child_config)
--> 159     ret = context.run(self.func, input, **kwargs)
    160 except BaseException as e:
    161     run_manager.on_chain_error(e)

Cell In[5], line 8, in <lambda>(x)
      6 conditional_map = {k: k for k in members}
      7 conditional_map["FINISH"] = END
----> 8 workflow.add_conditional_edges("supervisor", lambda x: x["next"], conditional_map)
      9 # Finally, add entrypoint
     10 workflow.add_edge(START, "supervisor")

KeyError: 'next'

How can I fix this?

Full log

--------------------------------------------------------------------------- KeyError Traceback (most recent call last) Cell In[15], line 1 ----> 1 for s in graph.stream( 2 { 3 "messages": [ 4 HumanMessage(content="Code hello world and print it to the terminal") 5 ] 6 } 7 ): 8 if "__end__" not in s: 9 print(s)

File ~/miniconda3/envs/multi/lib/python3.11/site-packages/langgraph/pregel/init.py:1278, in Pregel.stream(self, input, config, stream_mode, output_keys, interrupt_before, interrupt_after, debug, subgraphs)
1267 # Similarly to Bulk Synchronous Parallel / Pregel model
1268 # computation proceeds in steps, while there are channel updates
1269 # channel updates from step N are only visible in step N+1
1270 # channels are guaranteed to be immutable for the duration of the step,
1271 # with channel updates applied only at the transition between steps
1272 while loop.tick(
1273 input_keys=self.input_channels,
1274 interrupt_before=interrupt_before_,
1275 interrupt_after=interrupt_after_,
1276 manager=run_manager,
1277 ):
-> 1278 for _ in runner.tick(
1279 loop.tasks.values(),
1280 timeout=self.step_timeout,
1281 retry_policy=self.retry_policy,
1282 get_waiter=get_waiter,
1283 ):
1284 # emit output
1285 yield from output()
1286 # emit output

File ~/miniconda3/envs/multi/lib/python3.11/site-packages/langgraph/pregel/runner.py:52, in PregelRunner.tick(self, tasks, reraise, timeout, retry_policy, get_waiter)
50 t = tasks[0]
51 try:
---> 52 run_with_retry(t, retry_policy)
53 self.commit(t, None)
54 except Exception as exc:

File ~/miniconda3/envs/multi/lib/python3.11/site-packages/langgraph/pregel/retry.py:29, in run_with_retry(task, retry_policy)
27 task.writes.clear()
28 # run the task
---> 29 task.proc.invoke(task.input, config)
30 # if successful, end
31 break

File ~/miniconda3/envs/multi/lib/python3.11/site-packages/langgraph/utils/runnable.py:387, in RunnableSeq.invoke(self, input, config, **kwargs)
385 input = context.run(step.invoke, input, config, **kwargs)
386 else:
--> 387 input = context.run(step.invoke, input, config)
388 # finish the root run
389 except BaseException as e:

File ~/miniconda3/envs/multi/lib/python3.11/site-packages/langgraph/utils/runnable.py:167, in RunnableCallable.invoke(self, input, config, **kwargs)
165 else:
166 context.run(_set_config_context, config)
--> 167 ret = context.run(self.func, input, **kwargs)
168 if isinstance(ret, Runnable) and self.recurse:
169 return ret.invoke(input, config)

File ~/miniconda3/envs/multi/lib/python3.11/site-packages/langgraph/graph/graph.py:94, in Branch._route(self, input, config, reader, writer)
92 else:
93 value = input
---> 94 result = self.path.invoke(value, config)
95 return self._finish(writer, input, result, config)

File ~/miniconda3/envs/multi/lib/python3.11/site-packages/langgraph/utils/runnable.py:159, in RunnableCallable.invoke(self, input, config, **kwargs)
157 context = copy_context()
158 context.run(_set_config_context, child_config)
--> 159 ret = context.run(self.func, input, **kwargs)
160 except BaseException as e:
161 run_manager.on_chain_error(e)

Cell In[11], line 8, in (x)
6 conditional_map = {k: k for k in members}
7 conditional_map["FINISH"] = END
----> 8 workflow.add_conditional_edges("supervisor", lambda x: x["next"], conditional_map)
9 # Finally, add entrypoint
10 workflow.add_edge(START, "supervisor")

KeyError: 'next'

[100101010]

Hello, i got the same error, and i revised the routeResponse function as follows. Subsequently, it works.

class routeResponse(BaseModel):
    # next: Literal[*options]
    next: Literal["Researcher", "Coder", "FINISH"]

[kmillanrfractal]

Were are the prompts for the other nodes? how do they know what is their purpose?

[gtnpromtior]

you have to give it to them......
meta_campaigns_agent = create_react_agent(
llm_mini,
tools=initialize_tools_v2(),
state_modifier=meta_prompt, --------- HERE
)

[cris-m]

I am stuck in infinite loop in this section. is the following correct? I see two system in the same prompt

prompt = ChatPromptTemplate.from_messages(
    [
        ("system", system_prompt),
        MessagesPlaceholder(variable_name="messages"),
        (
            "system",
            "Given the conversation above, who should act next?"
            " Or should we FINISH? Select one of: {options}",
        ),
    ]
).partial(options=str(options), members=", ".join(members))

I think it must be like this:

prompt = ChatPromptTemplate.from_messages(
    [
        ("system", system_prompt),
        MessagesPlaceholder(variable_name="messages"),
        (
            "human",
            "Given the conversation above, who should act next?"
            " Or should we FINISH? Select one of: {options}",
        ),
    ]
).partial(options=str(options), members=", ".join(members))

supervisor unable to return next it always None:

{'supervisor': {'next': 'Coder'}}
----
{'Coder': {'messages': [HumanMessage(content='Here is the code to print "Hello World" to the terminal:\n\n```python\nprint("Hello World")\n```', additional_kwargs={}, response_metadata={}, name='Coder')]}}
----
{'supervisor': None}
----
{'Coder': {'messages': [HumanMessage(content='Here is the code to print "Hello World" to the terminal:\n\n```python\nprint("Hello World")\n```\n\nAnd here is the output:\n\n```\nHello World\n```', additional_kwargs={}, response_metadata={}, name='Coder')]}}
----
{'supervisor': None}
----
{'Coder': {'messages': [HumanMessage(content='Here is the code to print "Hello World" to the terminal:\n\n```python\nprint("Hello World")\n```\n\nAnd here is the output:\n\n```\nHello World\n```', additional_kwargs={}, response_metadata={}, name='Coder')]}}
----
{'supervisor': None}
----

[MinLee0210]

I am facing the same situation. I change the code for each agent so that after it executes its task, it goes directly to the END state.

In my case, after an agent completed its task, it went back to the supervisor. Instead of ending up at the END state, the supervisor viewed the content, then routed the content to the agent.

[gtnpromtior]

can you show me this please ?

[yangqiqing]

How to add memory to Supervisor. I tried graph = workflow.compile(checkpointer=memory) but no work.