feat: ReAct agent chat engine with context generator

[parhammmm]

This enables tool calling within a conversation + RAG enabled.

example usage:

import { FunctionTool, ContextReActAgentChatEngine } from "llamaindex";

export const NewsTool = FunctionTool.from(
  async ({ query }: { query: string }) => {
    // search
    return results
  },
  {
    name: "news_search",
    description:
      "Use this function to perform a web search for the latest news from around the web",
    parameters: {
      type: "object",
      properties: {
        query: {
          type: "string",
          description:
            "Search query that will be used to get the latest news from the web, ask for specific queries",
        },
      },
      required: ["query"],
    },
  },
);

export const SearchTool = FunctionTool.from(
  async ({ query }: { query: string }) => {
    // search
    return results
  },
  {
    name: "web_search",
    description:
      "Use this function to perform a web search when the user asks for infomation that is likely to only be found on the web and not in your knowledge",
    parameters: {
      type: "object",
      properties: {
        query: {
          type: "string",
          description:
            "Search query that will be used to get information from the web using a search engine",
        },
      },
      required: ["query"],
    },
  },
);

export const ScrapeTool = FunctionTool.from(
  async ({ url }: { url: string }) => {
    // ...scrape
    return scraped
  },
  {
    name: "web_scraper",
    description:
      "Use this function to scrape content from a webpage in order to read it. This will give you the text need to further investigate a topic. Use it after searching the web or news",
    parameters: {
      type: "object",
      properties: {
        url: {
          type: "string",
          description: "A url in the format of https://example.com/foo/bar",
        },
      },
      required: ["url"],
    },
  },
);


const engine = new ContextReActAgentChatEngine({
  chatModel,
  tools: [NewsTool, SearchTool, ScrapeTool], // News and/or Search is done first and scrape is used for further research
  retriever,
});

const response = await engine.chat({
  message,
  chatHistory: [...],
});

[changeset-bot]

🦋 Changeset detected

Latest commit: 255f1e7

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 10 packages

Name	Type
llamaindex	Patch
docs	Patch
@llamaindex/community	Patch
@llamaindex/experimental	Patch
@llamaindex/cloudflare-worker-agent-test	Patch
@llamaindex/next-agent-test	Patch
@llamaindex/nextjs-edge-runtime-test	Patch
@llamaindex/waku-query-engine-test	Patch
@llamaindex/autotool-01-node-example	Patch
@llamaindex/autotool-02-next-example	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
llama-index-ts-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jun 4, 2024 3:11pm

[himself65]

could you please rebase the code?

[marcusschiesser]

@parhammmm did you compare this approach with using a QueryEngineTool that is wrapping the RAG? It has a couple of advantages:

• you can use it with every agent chat engine (not only ReAct)
• you can add multiple QueryEngineTools and the agent can decide which to use

[parhammmm]

@marcusschiesser I actually did and have moved to using that approach, I made the engine for it on my local I can make a PR

Might be worth while not to merge in this PR because of it

[parhammmm]

@marcusschiesser having said that it does have a use case, when you need things that should always be called e.g. attachments lookups and you don't want to fiddle with the prompting of the agent. Will still make the AgentChatEngine pr shortly

@himself65 rebased

[marcusschiesser]

@parhammmm how about using then a design where you just plugin the contextGenerator into the existing chat engines?

[parhammmm]

@marcusschiesser we can just go ahead and abandon this PR in favour of #909 ?

[marcusschiesser]

@parhammmm just reviewed #909 - but the use case of this PR is to add a contextGenerator to the agents, isn't?

[parhammmm]

@marcusschiesser @himself65 closing this in favour of the recent PRs