Skip to content

Commit

Permalink
Update concepts with more tool content, fix titles (#6123)
Browse files Browse the repository at this point in the history
  • Loading branch information
jacoblee93 authored Jul 18, 2024
1 parent f9952a9 commit 4655788
Show file tree
Hide file tree
Showing 11 changed files with 118 additions and 32 deletions.
2 changes: 2 additions & 0 deletions docs/core_docs/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,8 @@ docs/how_to/custom_chat.md
docs/how_to/custom_chat.mdx
docs/how_to/custom_callbacks.md
docs/how_to/custom_callbacks.mdx
docs/how_to/convert_runnable_to_tool.md
docs/how_to/convert_runnable_to_tool.mdx
docs/how_to/code_splitter.md
docs/how_to/code_splitter.mdx
docs/how_to/chatbots_tools.md
Expand Down
115 changes: 99 additions & 16 deletions docs/core_docs/docs/concepts.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -271,13 +271,18 @@ This property returns an array of objects. Each object has the following keys:

This represents a system message, which tells the model how to behave. Not every model provider supports this.

#### FunctionMessage
#### ToolMessage

This represents the result of a function call. In addition to `role` and `content`, this message has a `name` parameter which conveys the name of the function that was called to produce this result.
This represents the result of a tool call. In addition to `role` and `content`, this message has:

#### ToolMessage
- a `tool_call_id` field which conveys the id of the call to the tool that was called to produce this result.
- an `artifact` field which can be used to pass along arbitrary artifacts of the tool execution which are useful to track but which should not be sent to the model.

This represents the result of a tool call. This is distinct from a FunctionMessage in order to match OpenAI's `function` and `tool` message types. In addition to `role` and `content`, this message has a `tool_call_id` parameter which conveys the id of the call to the tool that was called to produce this result.
#### (Legacy) FunctionMessage

This is a legacy message type, corresponding to OpenAI's legacy function-calling API. ToolMessage should be used instead to correspond to the updated tool-calling API.

This represents the result of a function call. In addition to `role` and `content`, this message has a `name` parameter which conveys the name of the function that was called to produce this result.

### Prompt templates

Expand Down Expand Up @@ -540,26 +545,104 @@ For specifics on how to use retrievers, see the [relevant how-to guides here](/d

<span data-heading-keywords="tool,tools"></span>

Tools are interfaces that an agent, chain, or LLM can use to interact with the world.
They combine a few things:
Tools are utilities designed to be called by a model: their inputs are designed to be generated by models, and their outputs are designed to be passed back to models.
Tools are needed whenever you want a model to control parts of your code or call out to external APIs.

A tool consists of:

1. The name of the tool.
2. A description of what the tool does.
3. A JSON schema defining the inputs to the tool.
4. A function.

When a tool is bound to a model, the name, description and JSON schema are provided as context to the model.

Given a list of tools and a set of instructions, a model can request to call one or more tools with specific inputs.
Once the chosen tools are invoked, the results can be passed back to the model so that it can complete whatever task
it's performing.

#### Tool inputs

A tool can take arbitrary arguments as input. At runtime, these arguments can be passed in either:

1. As just the raw arguments.
2. As a `ToolCall`, which contains the arguments along with other metadata like the tool call ID.

```ts
const tool = ...
const llmWithTools = llm.bindTools([tool]);
const aiMessage = await llmWithTools.invoke("do xyz...");

// AIMessage(tool_calls=[ToolCall(...), ...], ...)

const toolCall = aiMessage.tool_calls[0];

// ToolCall(args={...}, id=..., ...)

// 1. pass in args directly
await tool.invoke(toolCall.args);

// 2. pass in the whole ToolCall
await tool.invoke(toolCall);
```

A tool also has access to the `RunnableConfig` that's passed into whatever chain the tool is a part of. This allows you to write tool logic that can be parameterized by the chain config.

```ts
const config = { configurable: { tool_param_foo: ... }};
await tool.invoke(toolCall, config);
```

See the how-to guide for [passing in configs here](/docs/how_to/tool_configure/).

1. The name of the tool
2. A description of what the tool is
3. JSON schema of what the inputs to the tool are
4. The function to call
5. Whether the result of a tool should be returned directly to the user
#### Tool outputs

It is useful to have all this information because this information can be used to build action-taking systems! The name, description, and JSON schema can be used to prompt the LLM so it knows how to specify what action to take, and then the function to call is equivalent to taking that action.
The format of a tool's output depends on the format of the input. If a tool is called:

The simpler the input to a tool is, the easier it is for an LLM to be able to use it.
Many agents will only work with tools that have a single string input.
1. With a dict of its arguments then it will produce an arbitrary output that we assume can be passed to a model as the `ToolMessage.content` field,
2. A `ToolCall` then it will produce a `ToolMessage(content=..., ...)` where the tool output has already been assigned to the `ToolMessage.content` field.

Importantly, the name, description, and JSON schema (if used) are all used in the prompt. Therefore, it is really important that they are clear and describe exactly how the tool should be used. You may need to change the default name, description, or JSON schema if the LLM is not understanding how to use the tool.
```ts
// 1. pass in args directly

For specifics on how to use tools, see the [relevant how-to guides here](/docs/how_to/#tools).
tool.invoke(toolCall.args);
// -> "tool result foobar..."

// 2. pass in the whole ToolCall

await tool.invoke(toolCall);
// -> ToolMessage(content="tool result foobar...", tool_call_id=..., name="tool_name")
```

A tool can also be defined to include an artifact when invoked with a `ToolCall`. An artifact is some element of the
tool's execution which is useful to return but shouldn't be sent to the model. The artifact can _only_ be returned
when the tool input is a `ToolCall`:

```ts
await toolWithArtifact.invoke(toolCall);
// -> ToolMessage(content="tool result foobar...", tool_call_id=..., name="tool_name", artifact=...).
```

Learn about [`ToolMessage.artifact` here](/docs/concepts/#toolmessage) and about [defining tools that return artifacts here](/docs/how_to/tool_artifacts/).

#### Best practices

When designing tools to be used by a model, it is important to keep in mind that:

- Chat models that have explicit [tool-calling APIs](/docs/concepts/#functiontool-calling) will be better at tool calling than non-fine-tuned models.
- Models will perform better if the tools have well-chosen names, descriptions, and JSON schemas. This another form of prompt engineering.
- Simple, narrowly scoped tools are easier for models to use than complex tools.

#### Related

For specifics on how to use tools, see the [tools how-to guides](/docs/how_to/#tools).

To use a pre-built tool, see the [tool integration docs](/docs/integrations/tools/).

### Toolkits

<span data-heading-keywords="toolkit,toolkits"></span>

Toolkits are collections of tools that are designed to be used together for specific tasks. They have convenient loading methods.

All Toolkits expose a `getTools` method which returns an array of tools.
Expand Down
2 changes: 1 addition & 1 deletion docs/core_docs/docs/how_to/convert_runnable_to_tool.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"id": "9a8bceb3-95bd-4496-bb9e-57655136e070",
"metadata": {},
"source": [
"# How to use Runnables as Tools\n",
"# How to convert Runnables to Tools\n",
"\n",
"```{=mdx}\n",
"\n",
Expand Down
2 changes: 1 addition & 1 deletion docs/core_docs/docs/how_to/custom_tools.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
"id": "5436020b",
"metadata": {},
"source": [
"# How to create custom Tools\n",
"# How to create Tools\n",
"\n",
":::info Prerequisites\n",
"\n",
Expand Down
16 changes: 8 additions & 8 deletions docs/core_docs/docs/how_to/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -160,17 +160,17 @@ Indexing is the process of keeping your vectorstore in-sync with the underlying

LangChain [Tools](/docs/concepts/#tools) contain a description of the tool (to pass to the language model) as well as the implementation of the function to call.

- [How to: create custom tools](/docs/how_to/custom_tools)
- [How to: use built-in tools and built-in toolkits](/docs/how_to/tools_builtin)
- [How to: convert Runnables to tools](/docs/how_to/convert_runnable_to_tool)
- [How to: use a chat model to call tools](/docs/how_to/tool_calling/)
- [How to: pass tool results back to model](/docs/how_to/tool_results_pass_to_model/)
- [How to: add ad-hoc tool calling capability to LLMs and Chat Models](/docs/how_to/tools_prompting)
- [How to: create tools](/docs/how_to/custom_tools)
- [How to: use built-in tools and toolkits](/docs/how_to/tools_builtin)
- [How to: use chat models to call tools](/docs/how_to/tool_calling/)
- [How to: pass tool outputs to chat models](/docs/how_to/tool_results_pass_to_model/)
- [How to: pass run time values to tools](/docs/how_to/tool_runtime)
- [How to: handle errors when calling tools](/docs/how_to/tools_error)
- [How to: handle tool errors](/docs/how_to/tools_error)
- [How to: access the `RunnableConfig` object within a custom tool](/docs/how_to/tool_configure)
- [How to: stream events from child runs within a custom tool](/docs/how_to/tool_stream_events)
- [How to: return extra artifacts from a custom tool](/docs/how_to/tool_artifacts)
- [How to: return artifacts from a tool](/docs/how_to/tool_artifacts)
- [How to: convert Runnables to tools](/docs/how_to/convert_runnable_to_tool)
- [How to: add ad-hoc tool calling capability to models](/docs/how_to/tools_prompting)

### Agents

Expand Down
3 changes: 2 additions & 1 deletion docs/core_docs/docs/how_to/tool_artifacts.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,13 @@
"id": "503e36ae-ca62-4f8a-880c-4fe78ff5df93",
"metadata": {},
"source": [
"# How to return extra artifacts from a tool\n",
"# How to return artifacts from a tool\n",
"\n",
"```{=mdx}\n",
":::info Prerequisites\n",
"This guide assumes familiarity with the following concepts:\n",
"\n",
"- [ToolMessage](/docs/concepts/#toolmessage)\n",
"- [Tools](/docs/concepts/#tools)\n",
"- [Tool calling](/docs/concepts/#functiontool-calling)\n",
"\n",
Expand Down
2 changes: 1 addition & 1 deletion docs/core_docs/docs/how_to/tool_calling.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# How to use a chat model to call tools\n",
"# How to use chat models to call tools\n",
"\n",
":::info Prerequisites\n",
"\n",
Expand Down
2 changes: 1 addition & 1 deletion docs/core_docs/docs/how_to/tool_configure.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# How to access the RunnableConfig object within a custom tool\n",
"# How to access the RunnableConfig from a tool\n",
"\n",
"```{=mdx}\n",
":::info Prerequisites\n",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# How to pass tool outputs to the model\n",
"# How to pass tool outputs to chat models\n",
"\n",
"```{=mdx}\n",
":::info Prerequisites\n",
Expand Down
2 changes: 1 addition & 1 deletion docs/core_docs/docs/how_to/tool_runtime.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# How to pass run time values to a tool\n",
"# How to pass run time values to tools\n",
"\n",
"```{=mdx}\n",
":::info Prerequisites\n",
Expand Down
2 changes: 1 addition & 1 deletion docs/core_docs/docs/how_to/tool_stream_events.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# How to stream events from child runs within a custom tool\n",
"# How to stream events from a tool\n",
"\n",
"```{=mdx}\n",
":::info Prerequisites\n",
Expand Down

0 comments on commit 4655788

Please sign in to comment.