Agent not using documents in answering question

[Richie97]

I have a graph with a few agents, one of which has a retriever tool to fetch meeting transcripts. When asked for meeting details, the retriever finds a set of documents, but those documents aren't used to answer the questions.
Where I make the agents:

def create_agent(
    llm: ChatOpenAI,
    tools: list,
    system_prompt: str,
) -> str:
    """Create a function-calling agent and add it to the graph."""
    system_prompt += "\nWork autonomously according to your specialty, using the tools available to you."
    " Do not ask for clarification."
    " Your other team members (and other teams) will collaborate with you with their own specialties."
    " You are chosen for a reason! You are one of the following team members: {team_members}."
    prompt = ChatPromptTemplate.from_messages(
        [
            (
                "system",
                system_prompt,
            ),
            MessagesPlaceholder(variable_name="messages"),
            MessagesPlaceholder(variable_name="agent_scratchpad"),
        ]
    )
    agent = create_openai_tools_agent(llm, tools, prompt)
    executor = AgentExecutor(agent=agent, tools=tools)
    return executor

Actual Agent:

self.agent = create_agent(
            llm,
            [meeting_notes_tool],
            """You are a Meeting Agent able to query meeting transcripts with the retriever tool. These meetings are in a db, 
            and contain notes and next steps for our meetings. Querys should be fully formed questions.
             Using those documents, you can determine answers to meeting based questions.""",
        )

This will correctly call the tool and the tool gets back some set of documents, but those documents are never used after they are fetched, the agent will just say "I couldn't find any information regarding the last call in the meeting transcripts. It's possible that the meeting may not have been recorded or documented in the system. If you have any other questions or need information on a different topic, feel free to ask!" even though I can see the documents in the tool call.

Is there something I am missing?

The graph is created similar to the ones in the docs:

 # Top-level graph state
class State(TypedDict):
    messages: Annotated[List[BaseMessage], operator.add]    
    next: str


# Define the graph.
super_graph = StateGraph(State)
# add nodes and all that.