LangGraph supervisor keeps calling same node even when FINISH

[shaikhn1750]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.

Example Code

Here is my System Prompt:

system_prompt = (
    " You are a supervisor tasked with managing a conversation between the"
    " following workers:  {members}. Given the following user request,"
    " respond with the worker to act next. Each worker will perform a"
    " task and respond with their results."
    " If you or any of the other {members} have the final answer or deliverable"
    " prefix your response with FINISH: so you know it is time to stop. Once any of the {members} respond"
    " with a final answer or deliverable return the response to the user and stop the execution"
    )

And here is the response.

Please enter your question: hello
{'supervisor': {'next': 'Conversation'}}
----
{'Conversation': {'messages': [HumanMessage(content='FINISH: Hello there! How can I assist you today?', name='Conversation')]}}
----
{'supervisor': {'next': 'Conversation'}}
----
{'Conversation': {'messages': [HumanMessage(content="FINISH: How's your day going?", name='Conversation')]}}
----
{'supervisor': {'next': 'FINISH'}}
----

Process finished with exit code 0

Error Message and Stack Trace (if applicable)

No response

Description

Hello,
I have a conversational node which is used to respond with a normal conversation.
Even when the message is prefixed as finish the supervisor keeps calling the node again.

System Info

System Information

OS: Darwin
OS Version: Darwin Kernel Version 22.3.0: Mon Jan 30 20:39:46 PST 2023; root:xnu-8792.81.3~2/RELEASE_ARM64_T6020
Python Version: 3.11.6 (v3.11.6:8b6ee5ba3b, Oct 2 2023, 11:18:21) [Clang 13.0.0 (clang-1300.0.29.30)]

Package Information

langchain_core: 0.1.18
langchain: 0.1.5
langchain_community: 0.0.17
langchain_experimental: 0.0.50
langchain_openai: 0.0.5
langchainhub: 0.1.14
langgraph: 0.0.21

Packages not installed (Not Necessarily a Problem)

The following packages were not found:

langserve

[hinthornw]

Could you share al angsmith trace of this behavior to help me debug?

[shaikhn1750]

@hinthornw here is the trace.

{
  "__end__": {
    "messages": [
      {
        "content": "hi",
        "additional_kwargs": {},
        "type": "human",
        "example": false
      },
      {
        "content": "FINISH: Hello! How can I assist you today?",
        "additional_kwargs": {},
        "type": "human",
        "example": false,
        "name": "Conversation"
      },
      {
        "content": "FINISH: Hello! How can I assist you today?",
        "additional_kwargs": {},
        "type": "human",
        "example": false,
        "name": "Conversation"
      }
    ],
    "next": "FINISH"
  }
}

for a simple hi the supervisor called the conversational node twice.

[shaikhn1750]

Another trace :

Here the python tool is called twice at the end.
Any help would be appreciated @hinthornw. :)

INPUT:
get tld & its count from premium inventory table, use different colors to visualise the data.

{
  "__end__": {
    "messages": [
      {
        "content": "hi",
        "additional_kwargs": {},
        "type": "human",
        "example": false
      },
      {
        "content": "FINISH: Hello! How can I assist you today?",
        "additional_kwargs": {},
        "type": "human",
        "example": false,
        "name": "Conversation"
      },
      {
        "content": "how are you",
        "additional_kwargs": {},
        "type": "human",
        "example": false
      },
      {
        "content": "get tld & its count from premium inventory table, use different colors to visualise the data",
        "additional_kwargs": {},
        "type": "human",
        "example": false
      },
      {
        "content": "```sql\nSELECT tld, COUNT(*) AS tld_count FROM premium_db.premium_inventory GROUP BY tld;\n```\nPlease note that as an advanced text to SQL converter, I only generate SQL queries. Visualization of the data with different colors would be handled by a data visualization tool or software after executing the SQL query in a database environment.",
        "additional_kwargs": {},
        "type": "human",
        "example": false,
        "name": "text_to_sql"
      },
      {
        "content": "FINISH: The data from the `premium_inventory` table has been visualized in a bar chart with different colors for each TLD. Each bar represents a TLD (Top-Level Domain), and the height of the bar indicates the count of that specific TLD in the premium inventory data. The colors are assigned using a paired colormap to distinguish between different TLDs. The x-axis labels, which are the TLDs, have been rotated for better readability.",
        "additional_kwargs": {},
        "type": "human",
        "example": false,
        "name": "sql_to_python"
      },
      {
        "content": "FINISH: The data from the `premium_inventory` table has been visualized in a bar chart with different colors for each TLD. Each bar represents a TLD (Top-Level Domain), and the height of the bar indicates the count of that specific TLD in the premium inventory data. The colors are assigned using a paired colormap to distinguish between different TLDs. The x-axis labels, which are the TLDs, have been rotated for better readability.",
        "additional_kwargs": {},
        "type": "human",
        "example": false,
        "name": "sql_to_python"
      }
    ],
    "next": "FINISH"
  }
}

[shaikhn1750]

@hinthornw in the example https://github.com/langchain-ai/langgraph/blob/main/examples/multi_agent/hierarchical_agent_teams.ipynb you can see the Research team is called 9 times with the same output and then other node is called. can't we remove this redundant calling??

[bonillam]

I have a similar issue:
-i created a custom tool,
-created a new agent, assigned the custom tool to it
-the supervisor agent only calls the researcher and coder agents. The custom worker is only called if I specifically include a instruction (in the prompt) to ask the custom agent.

@hinthornw, using graphs for agent orchestration is a nice concept. Also great videos. Hoping to connect with you. Will DM via LinkedIn

[madina1203]

We are working on a project which incorporates LangGraph multi agents and supervisor workflow. There is still a problem of Supervisor calling the same agent even if this agent has already performed its task. I have already tried to modify the prompt, but I wonder if there is a way to modify the workflow to limit the number of interactions between supervisor and other agents? This would be very helpful.

[shaikhn1750]

@madina1203 you can add a conditional edge like the following

def should_continue(state):
       # This function decides
       messages = state["messages"]
       last_message = messages[-1]

       if "FINISH:" in last_message.content:
           return "FINISH"
       else:
           return state["next"]

   workflow.add_conditional_edges("supervisor", should_continue, conditional_map)

And modify the prompt to add prefix FINISH: in the response.
This worked for me.

[dhanashri3107]

Facing the similar issue with multi- agent collaboration. Router calls the same node even when the recusion limit is set to 150. @madina1203 what is conditional_map in above case ?

[hinthornw]

One way to have them end more reliably is to include reminders/instructions in responses after a tool is invoked. Like after a tool is used, return the tool result and append "If all tasks are complete, reply with "FINISHED", etc. Or could factor finishing out into an explicit function call

[maurovz]

Experiencing the same thing.. Any updates on this?

[leobenkel]

Same here