Merge branch 'main' into brace/remove-message

deno.json

@@ -9,7 +9,7 @@
     "@langchain/textsplitters": "npm:@langchain/textsplitters",
     "@langchain/google-vertexai-web": "npm:@langchain/google-vertexai-web",
     "@langchain/mistralai": "npm:@langchain/mistralai",
-    "@langchain/core/": "npm:/@langchain/core@latest/",
+    "@langchain/core/": "npm:/@langchain/core@0.2.16/",
     "@langchain/pinecone": "npm:@langchain/pinecone",
     "@langchain/google-common": "npm:@langchain/google-common",
     "@langchain/langgraph": "npm:/@langchain/langgraph@0.0.21",

docs/core_docs/docs/concepts.mdx

@@ -16,6 +16,7 @@ import useBaseUrl from "@docusaurus/useBaseUrl";
     dark: useBaseUrl("/svg/langchain_stack_062024_dark.svg"),
   }}
   title="LangChain Framework Overview"
+  style={{ width: "100%" }}
 />
 
 ### `@langchain/core`

docs/core_docs/docs/how_to/custom_tools.ipynb

@@ -42,7 +42,7 @@
    "source": [
     "## `tool` function\n",
     "\n",
-    ":::note\n",
+    ":::caution Compatibility\n",
     "Only available in `@langchain/core` version 0.2.7 and above.\n",
     ":::\n",
     "\n",
@@ -63,11 +63,14 @@
    "metadata": {},
    "outputs": [
     {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "The sum of 1 and 2 is 3\n"
-     ]
+     "data": {
+      "text/plain": [
+       "\u001b[32m\"The sum of 1 and 2 is 3\"\u001b[39m"
+      ]
+     },
+     "execution_count": 1,
+     "metadata": {},
+     "output_type": "execute_result"
     }
    ],
    "source": [
@@ -102,7 +105,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": 2,
    "id": "833dda4a",
    "metadata": {},
    "outputs": [
@@ -112,7 +115,7 @@
        "\u001b[32m\"72\"\u001b[39m"
       ]
      },
-     "execution_count": 1,
+     "execution_count": 2,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -143,12 +146,12 @@
    "source": [
     "## `DynamicTool`\n",
     "\n",
-    "For older agents that require tools which accept only a single input, you can pass the relevant parameters to the [`DynamicTool`](https://v02.api.js.langchain.com/classes/langchain_core_tools.DynamicTool.html) class. This is useful when working with older agents that only support tools that accept a single input. In this case, no schema is required:"
+    "For older agents that require tools which accept only a single input, you can pass the relevant parameters to the [`DynamicTool`](https://api.js.langchain.com/classes/langchain_core_tools.DynamicTool.html) class. This is useful when working with older agents that only support tools that accept a single input. In this case, no schema is required:"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": 3,
    "id": "b0ce7de8",
    "metadata": {},
    "outputs": [
@@ -158,7 +161,7 @@
        "\u001b[32m\"LangChain\"\u001b[39m"
       ]
      },
-     "execution_count": 2,
+     "execution_count": 3,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -177,16 +180,162 @@
     "await searchTool.invoke(\"foo\");"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "d4093dea",
+   "metadata": {},
+   "source": [
+    "# Returning artifacts of Tool execution\n",
+    "\n",
+    "Sometimes there are artifacts of a tool's execution that we want to make accessible to downstream components in our chain or agent, but that we don't want to expose to the model itself. For example if a tool returns custom objects like Documents, we may want to pass some view or metadata about this output to the model without passing the raw output to the model. At the same time, we may want to be able to access this full output elsewhere, for example in downstream tools.\n",
+    "\n",
+    "The Tool and `ToolMessage` interfaces make it possible to distinguish between the parts of the tool output meant for the model (`ToolMessage.content`) and those parts which are meant for use outside the model (`ToolMessage.artifact`).\n",
+    "\n",
+    ":::caution Compatibility\n",
+    "This functionality was added in `@langchain/core>=0.2.16`. Please make sure your package is up to date.\n",
+    ":::\n",
+    "\n",
+    "If you want your tool to distinguish between message content and other artifacts, we need to do three things:\n",
+    "\n",
+    "- Set the `response_format` parameter to `\"content_and_artifact\"` when defining the tool.\n",
+    "- Make sure that we return a tuple of `[content, artifact]`.\n",
+    "- Call the tool with a a [`ToolCall`](https://api.js.langchain.com/types/langchain_core_messages_tool.ToolCall.html)    (like the ones generated by tool-calling models) rather than with the required schema directly.\n",
+    "\n",
+    "Here's an example of what this looks like. First, create a new tool:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "ecf15c35",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import { z } from \"zod\";\n",
+    "import { tool } from \"@langchain/core/tools\";\n",
+    "\n",
+    "const randomIntToolSchema = z.object({\n",
+    "  min: z.number(),\n",
+    "  max: z.number(),\n",
+    "  size: z.number(),\n",
+    "});\n",
+    "\n",
+    "const generateRandomInts = tool(async ({ min, max, size }) => {\n",
+    "  const array: number[] = [];\n",
+    "  for (let i = 0; i < size; i++) {\n",
+    "    array.push(Math.floor(Math.random() * (max - min + 1)) + min);\n",
+    "  }\n",
+    "  return [\n",
+    "    `Successfully generated array of ${size} random ints in [${min}, ${max}].`,\n",
+    "    array,\n",
+    "  ];\n",
+    "}, {\n",
+    "  name: \"generateRandomInts\",\n",
+    "  description: \"Generate size random ints in the range [min, max].\",\n",
+    "  schema: randomIntToolSchema,\n",
+    "  responseFormat: \"content_and_artifact\",\n",
+    "});"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "5775e686",
+   "metadata": {},
+   "source": [
+    "If you invoke our tool directly with the tool arguments, you'll get back just the `content` part of the output:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "ecbde6de",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "\u001b[32m\"Successfully generated array of 10 random ints in [0, 9].\"\u001b[39m"
+      ]
+     },
+     "execution_count": 5,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "await generateRandomInts.invoke({ min: 0, max: 9, size: 10 });"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6299ef60",
+   "metadata": {},
+   "source": [
+    "But if you invoke our tool with a `ToolCall`, you'll get back a ToolMessage that contains both the content and artifact generated by the `Tool`:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "05209573",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "ToolMessage {\n",
+       "  lc_serializable: \u001b[33mtrue\u001b[39m,\n",
+       "  lc_kwargs: {\n",
+       "    content: \u001b[32m\"Successfully generated array of 10 random ints in [0, 9].\"\u001b[39m,\n",
+       "    artifact: [\n",
+       "      \u001b[33m7\u001b[39m, \u001b[33m7\u001b[39m, \u001b[33m1\u001b[39m, \u001b[33m4\u001b[39m, \u001b[33m8\u001b[39m,\n",
+       "      \u001b[33m4\u001b[39m, \u001b[33m8\u001b[39m, \u001b[33m3\u001b[39m, \u001b[33m0\u001b[39m, \u001b[33m9\u001b[39m\n",
+       "    ],\n",
+       "    tool_call_id: \u001b[32m\"123\"\u001b[39m,\n",
+       "    name: \u001b[32m\"generateRandomInts\"\u001b[39m,\n",
+       "    additional_kwargs: {},\n",
+       "    response_metadata: {}\n",
+       "  },\n",
+       "  lc_namespace: [ \u001b[32m\"langchain_core\"\u001b[39m, \u001b[32m\"messages\"\u001b[39m ],\n",
+       "  content: \u001b[32m\"Successfully generated array of 10 random ints in [0, 9].\"\u001b[39m,\n",
+       "  name: \u001b[32m\"generateRandomInts\"\u001b[39m,\n",
+       "  additional_kwargs: {},\n",
+       "  response_metadata: {},\n",
+       "  id: \u001b[90mundefined\u001b[39m,\n",
+       "  tool_call_id: \u001b[32m\"123\"\u001b[39m,\n",
+       "  artifact: [\n",
+       "    \u001b[33m7\u001b[39m, \u001b[33m7\u001b[39m, \u001b[33m1\u001b[39m, \u001b[33m4\u001b[39m, \u001b[33m8\u001b[39m,\n",
+       "    \u001b[33m4\u001b[39m, \u001b[33m8\u001b[39m, \u001b[33m3\u001b[39m, \u001b[33m0\u001b[39m, \u001b[33m9\u001b[39m\n",
+       "  ]\n",
+       "}"
+      ]
+     },
+     "execution_count": 6,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "await generateRandomInts.invoke({\n",
+    "  name: \"generateRandomInts\",\n",
+    "  args: { min: 0, max: 9, size: 10 },\n",
+    "  id: \"123\", // required\n",
+    "  type: \"tool_call\",\n",
+    "});"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "8eceaf09",
    "metadata": {},
    "source": [
-    "## Next steps\n",
+    "## Related\n",
     "\n",
     "You've now seen a few ways to create custom tools in LangChain.\n",
     "\n",
-    "Next, you might be interested in learning [how to use a chat model to call tools](/docs/how_to/tool_calling/)."
+    "Next, you might be interested in learning [how to use a chat model to call tools](/docs/how_to/tool_calling/).\n",
+    "\n",
+    "You can also check out how to create your own [custom versions of other modules](/docs/how_to/#custom)."
    ]
   }
  ],
@@ -197,15 +346,12 @@
    "name": "deno"
   },
   "language_info": {
-   "codemirror_mode": {
-    "mode": "typescript",
-    "name": "javascript",
-    "typescript": true
-   },
    "file_extension": ".ts",
-   "mimetype": "text/typescript",
+   "mimetype": "text/x.typescript",
    "name": "typescript",
-   "version": "3.7.2"
+   "nb_converter": "script",
+   "pygments_lexer": "typescript",
+   "version": "5.3.3"
   },
   "vscode": {
    "interpreter": {

docs/core_docs/docs/how_to/index.mdx

@@ -162,6 +162,7 @@ LangChain [Tools](/docs/concepts/#tools) contain a description of the tool (to p
 - [How to: use built-in tools and built-in toolkits](/docs/how_to/tools_builtin)
 - [How to: use a chat model to call tools](/docs/how_to/tool_calling/)
 - [How to: add ad-hoc tool calling capability to LLMs and Chat Models](/docs/how_to/tools_prompting)
+- [How to: return extra artifacts from a custom tool](/docs/how_to/custom_tools/#returning-artifacts-of-tool-execution)
 
 ### Agents
 

docs/core_docs/docs/introduction.mdx

@@ -23,6 +23,7 @@ import useBaseUrl from "@docusaurus/useBaseUrl";
     dark: useBaseUrl("/svg/langchain_stack_062024_dark.svg"),
   }}
   title="LangChain Framework Overview"
+  style={{ width: "100%" }}
 />
 
 Concretely, the framework consists of the following open-source libraries:

environment_tests/test-exports-cjs/package.json

@@ -26,7 +26,7 @@
     "@langchain/core": "workspace:*",
     "@langchain/openai": "workspace:*",
     "@tsconfig/recommended": "^1.0.2",
-    "@xenova/transformers": "^2.5.4",
+    "@xenova/transformers": "^2.17.2",
     "d3-dsv": "2",
     "hnswlib-node": "^3.0.0",
     "langchain": "workspace:*",

environment_tests/test-exports-esm/package.json

@@ -27,7 +27,7 @@
     "@langchain/core": "workspace:*",
     "@langchain/openai": "workspace:*",
     "@tsconfig/recommended": "^1.0.2",
-    "@xenova/transformers": "^2.5.4",
+    "@xenova/transformers": "^2.17.2",
     "d3-dsv": "2",
     "hnswlib-node": "^3.0.0",
     "langchain": "workspace:*",

langchain-core/.gitignore

@@ -10,6 +10,14 @@ callbacks/base.cjs
 callbacks/base.js
 callbacks/base.d.ts
 callbacks/base.d.cts
+callbacks/dispatch.cjs
+callbacks/dispatch.js
+callbacks/dispatch.d.ts
+callbacks/dispatch.d.cts
+callbacks/dispatch/web.cjs
+callbacks/dispatch/web.js
+callbacks/dispatch/web.d.ts
+callbacks/dispatch/web.d.cts
 callbacks/manager.cjs
 callbacks/manager.js
 callbacks/manager.d.ts

langchain-core/langchain.config.js

@@ -15,6 +15,8 @@ export const config = {
     agents: "agents",
     caches: "caches/base",
     "callbacks/base": "callbacks/base",
+    "callbacks/dispatch": "callbacks/dispatch/index",
+    "callbacks/dispatch/web": "callbacks/dispatch/web",
     "callbacks/manager": "callbacks/manager",
     "callbacks/promises": "callbacks/promises",
     chat_history: "chat_history",

langchain-core/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@langchain/core",
-  "version": "0.2.15",
+  "version": "0.2.16",
   "description": "Core LangChain.js abstractions and schemas",
   "type": "module",
   "engines": {
@@ -123,6 +123,24 @@
       "import": "./callbacks/base.js",
       "require": "./callbacks/base.cjs"
     },
+    "./callbacks/dispatch": {
+      "types": {
+        "import": "./callbacks/dispatch.d.ts",
+        "require": "./callbacks/dispatch.d.cts",
+        "default": "./callbacks/dispatch.d.ts"
+      },
+      "import": "./callbacks/dispatch.js",
+      "require": "./callbacks/dispatch.cjs"
+    },
+    "./callbacks/dispatch/web": {
+      "types": {
+        "import": "./callbacks/dispatch/web.d.ts",
+        "require": "./callbacks/dispatch/web.d.cts",
+        "default": "./callbacks/dispatch/web.d.ts"
+      },
+      "import": "./callbacks/dispatch/web.js",
+      "require": "./callbacks/dispatch/web.cjs"
+    },
     "./callbacks/manager": {
       "types": {
         "import": "./callbacks/manager.d.ts",
@@ -607,6 +625,14 @@
     "callbacks/base.js",
     "callbacks/base.d.ts",
     "callbacks/base.d.cts",
+    "callbacks/dispatch.cjs",
+    "callbacks/dispatch.js",
+    "callbacks/dispatch.d.ts",
+    "callbacks/dispatch.d.cts",
+    "callbacks/dispatch/web.cjs",
+    "callbacks/dispatch/web.js",
+    "callbacks/dispatch/web.d.ts",
+    "callbacks/dispatch/web.d.cts",
     "callbacks/manager.cjs",
     "callbacks/manager.js",
     "callbacks/manager.d.ts",