From 22a16e30609ae4d4a12d864c5676ced8823ce7c6 Mon Sep 17 00:00:00 2001 From: kevin666aa Date: Mon, 23 Dec 2024 04:29:00 -0500 Subject: [PATCH 1/5] revise autogen --- README.md | 10 +++++----- docs/v1/integrations/autogen.mdx | 10 +++++----- examples/README.md | 2 +- 3 files changed, 11 insertions(+), 11 deletions(-) diff --git a/README.md b/README.md index 61269a4c5..ed143ab67 100644 --- a/README.md +++ b/README.md @@ -58,7 +58,7 @@ AgentOps helps developers build, evaluate, and monitor AI agents. From prototype | ๐Ÿ’ธ **LLM Cost Management** | Track spend with LLM foundation model providers | | ๐Ÿงช **Agent Benchmarking** | Test your agents against 1,000+ evals | | ๐Ÿ” **Compliance and Security** | Detect common prompt injection and data exfiltration exploits | -| ๐Ÿค **Framework Integrations** | Native Integrations with CrewAI, AutoGen, Camel AI, & LangChain | +| ๐Ÿค **Framework Integrations** | Native Integrations with CrewAI, AG2(AutoGen), Camel AI, & LangChain | ## Quick Start โŒจ๏ธ @@ -171,11 +171,11 @@ pip install 'crewai[agentops]' - [AgentOps integration example](https://docs.agentops.ai/v1/integrations/crewai) - [Official CrewAI documentation](https://docs.crewai.com/how-to/AgentOps-Observability) -### AutoGen ๐Ÿค– -With only two lines of code, add full observability and monitoring to Autogen agents. Set an `AGENTOPS_API_KEY` in your environment and call `agentops.init()` +### AG2 ๐Ÿค– +With only two lines of code, add full observability and monitoring to AG2 (formerly AutoGen) agents. Set an `AGENTOPS_API_KEY` in your environment and call `agentops.init()` -- [Autogen Observability Example](https://microsoft.github.io/autogen/docs/notebooks/agentchat_agentops) -- [Autogen - AgentOps Documentation](https://microsoft.github.io/autogen/docs/ecosystem/agentops) +- [AG2 Observability Example](https://docs.ag2.ai/notebooks/agentchat_agentops) +- [AG2 - AgentOps Documentation](https://docs.ag2.ai/docs/ecosystem/agentops) ### Camel AI ๐Ÿช diff --git a/docs/v1/integrations/autogen.mdx b/docs/v1/integrations/autogen.mdx index a9cfc412e..85cea76a1 100644 --- a/docs/v1/integrations/autogen.mdx +++ b/docs/v1/integrations/autogen.mdx @@ -1,14 +1,14 @@ --- -title: Autogen -description: "Track your Autogen agents with AgentOps" +title: AG2 +description: "Track your AG2 agents with AgentOps" --- import CodeTooltip from '/snippets/add-code-tooltip.mdx' import EnvTooltip from '/snippets/add-env-tooltip.mdx' -AgentOps and Autogen teamed up to make monitoring Autogen agents dead simple. Autogen has comprehensive [documentation](https://microsoft.github.io/autogen/docs) available as well as a great [quickstart](https://microsoft.github.io/autogen/docs/Getting-Started). +AgentOps and AG2 (Formerly AutoGen) teamed up to make monitoring AG2 agents dead simple. AG2 has comprehensive [documentation](https://docs.ag2.ai/docs/Home) available as well as a great [quickstart](https://docs.ag2.ai/docs/Getting-Started). -## Adding AgentOps to Autogen agents +## Adding AgentOps to AG2 agents @@ -21,7 +21,7 @@ AgentOps and Autogen teamed up to make monitoring Autogen agents dead simple. Au ``` - + ```bash pip pip install pyautogen diff --git a/examples/README.md b/examples/README.md index 03c7f4528..6dca55bcd 100644 --- a/examples/README.md +++ b/examples/README.md @@ -23,7 +23,7 @@ At a high level, AgentOps gives you the ability to monitor LLM calls, costs, lat ## Integrations - [AI21](./ai21_examples/ai21_examples.ipynb) - [Anthropic](./anthropic_examples/)- -- [Autogen](./autogen_examples/) +- [AG2](./autogen_examples/) - [Cohere](./cohere_examples/cohere_example.ipynb) - [Crew.ai](./crew_examples/) - [Groq](./multi_agent_groq_example.ipynb) From 39602b9a86f569bd80c4515c4a1fcd786010635a Mon Sep 17 00:00:00 2001 From: kevin666aa Date: Mon, 23 Dec 2024 04:42:05 -0500 Subject: [PATCH 2/5] update ag2 --- CONTRIBUTING.md | 2 +- docs/v1/examples/examples.mdx | 4 +- docs/v1/usage/sdk-reference.mdx | 2 +- examples/autogen_examples/AgentChat.ipynb | 2 +- examples/autogen_examples/MathAgent.ipynb | 473 +++++++++++----------- 5 files changed, 241 insertions(+), 242 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 1db574e52..0b66b8d86 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -2,7 +2,7 @@ Thanks for checking out AgentOps. We're building tools to help developers like you make AI agents that actually work reliably. If you've ever tried to build an agent system, you know the pain - they're a nightmare to debug, impossible to monitor, and when something goes wrong... good luck figuring out why. -We created AgentOps to solve these headaches, and we'd love your help making it even better. Our SDK hooks into all the major Python frameworks (AutoGen, CrewAI, LangChain) and LLM providers (OpenAI, Anthropic, Cohere, etc.) to give you visibility into what your agents are actually doing. +We created AgentOps to solve these headaches, and we'd love your help making it even better. Our SDK hooks into all the major Python frameworks (AG2, CrewAI, LangChain) and LLM providers (OpenAI, Anthropic, Cohere, etc.) to give you visibility into what your agents are actually doing. ## How You Can Help diff --git a/docs/v1/examples/examples.mdx b/docs/v1/examples/examples.mdx index 7dbea88fb..90bd7e494 100644 --- a/docs/v1/examples/examples.mdx +++ b/docs/v1/examples/examples.mdx @@ -33,8 +33,8 @@ mode: "wide" Track observations from Claude, Haiku and Sonnet series of models - - AutoGen multi-agent conversible workflow with tool usage + } iconType="image" href="/v1/integrations/autogen"> + AG2 (Formerly AutoGen) multi-agent conversible workflow with tool usage } iconType="image" href="/v1/examples/camel"> diff --git a/docs/v1/usage/sdk-reference.mdx b/docs/v1/usage/sdk-reference.mdx index 62b11c9fc..1e5934026 100644 --- a/docs/v1/usage/sdk-reference.mdx +++ b/docs/v1/usage/sdk-reference.mdx @@ -114,7 +114,7 @@ Set the parent API key which has visibility over projects it is a parent of. ### `stop_instrumenting()` Stops instrumenting LLM calls. This is typically used by agent frameworks (i.e., [CrewAI](/v1/integrations/crewai) and -[autogen](/v1/integrations/autogen)) to stop using AgentOps' auto-instrumentation of LLM libraries such as OpenAI. This +[ag2](/v1/integrations/autogen)) to stop using AgentOps' auto-instrumentation of LLM libraries such as OpenAI. This allows these frameworks to use their own instrumenting or callback handler. diff --git a/examples/autogen_examples/AgentChat.ipynb b/examples/autogen_examples/AgentChat.ipynb index 85fd49910..7de27face 100644 --- a/examples/autogen_examples/AgentChat.ipynb +++ b/examples/autogen_examples/AgentChat.ipynb @@ -6,7 +6,7 @@ "id": "bb6538d8-2a5d-4a99-b2c1-7130963e4f7b", "metadata": {}, "source": [ - "# AutoGen Chat Example\n", + "# AG2 Chat Example\n", "\n", "\n", "AgentOps automatically configures itself when it's initialized meaning your agent run data will be tracked and logged to your AgentOps dashboard right away." diff --git a/examples/autogen_examples/MathAgent.ipynb b/examples/autogen_examples/MathAgent.ipynb index 1f1a44a65..e3a094ebd 100644 --- a/examples/autogen_examples/MathAgent.ipynb +++ b/examples/autogen_examples/MathAgent.ipynb @@ -1,238 +1,237 @@ { - "cells": [ - { - "attachments": {}, - "cell_type": "markdown", - "id": "bb6538d8-2a5d-4a99-b2c1-7130963e4f7b", - "metadata": {}, - "source": [ - "# AutoGen Tool Example\n", - "\n", - "\n", - "To get started, you'll need to install the AgentOps package and [set an API key](app.agentops.ai).\n", - "\n", - "AgentOps automatically configures itself when it's initialized meaning your agent run data will be tracked and logged to your AgentOps account right away." - ] - }, - { - "cell_type": "markdown", - "id": "083244fa", - "metadata": {}, - "source": [ - "First let's install the required packages" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "9c8104ad", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -U pyautogen\n", - "%pip install -U agentops\n", - "%pip install -U python-dotenv" - ] - }, - { - "cell_type": "markdown", - "id": "cc44e459", - "metadata": {}, - "source": [ - "Then import them" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "7672f591", - "metadata": {}, - "outputs": [], - "source": [ - "from autogen import ConversableAgent\n", - "from typing import Annotated, Literal\n", - "from autogen import ConversableAgent, register_function\n", - "import agentops\n", - "import os\n", - "from dotenv import load_dotenv\n", - "from IPython.core.error import (\n", - " StdinNotImplementedError,\n", - ") # only needed by AgentOps testing automation" - ] - }, - { - "cell_type": "markdown", - "id": "24f8bd70", - "metadata": {}, - "source": [ - "Next, we'll set our API keys. There are several ways to do this, the code below is just the most foolproof way for the purposes of this notebook. It accounts for both users who use environment variables and those who just want to set the API Key here in this notebook.\n", - "\n", - "[Get an AgentOps API key](https://agentops.ai/settings/projects)\n", - "\n", - "1. Create an environment variable in a .env file or other method. By default, the AgentOps `init()` function will look for an environment variable named `AGENTOPS_API_KEY`. Or...\n", - "\n", - "2. Replace `` below and pass in the optional `api_key` parameter to the AgentOps `init(api_key=...)` function. Remember not to commit your API key to a public repo!" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "9eeaef34", - "metadata": {}, - "outputs": [], - "source": [ - "load_dotenv()\n", - "OPENAI_API_KEY = os.getenv(\"OPENAI_API_KEY\") or \"\"\n", - "AGENTOPS_API_KEY = os.getenv(\"AGENTOPS_API_KEY\") or \"\"" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "d93f2339-7b99-4cf1-9232-c24faba49c7b", - "metadata": {}, - "outputs": [], - "source": [ - "agentops.init(AGENTOPS_API_KEY, default_tags=[\"autogen-tool-example\"])\n", - "\n", - "print(\"AgentOps is now running. You can view your session in the link above\")" - ] - }, - { - "cell_type": "markdown", - "id": "7858f0f6-9aca-4cdb-a514-9fbf7e353d50", - "metadata": {}, - "source": [ - "AutoGen will now start automatically tracking\n", - "\n", - "* LLM prompts and completions\n", - "* Token usage and costs\n", - "* Agent names and actions\n", - "* Correspondence between agents\n", - "* Tool usage\n", - "* Errors" - ] - }, - { - "cell_type": "markdown", - "id": "dc592637", - "metadata": {}, - "source": [ - "# Tool Example\n", - "AgentOps tracks when Autogen agents use tools. You can find more information on this example in [tool-use.ipynb](https://github.com/microsoft/autogen/blob/main/website/docs/tutorial/tool-use.ipynb)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "9e4dfe37-85e0-4035-a314-3459c6e378c4", - "metadata": {}, - "outputs": [], - "source": [ - "# Define model, openai api key, tags, etc in the agent configuration\n", - "config_list = [\n", - " {\n", - " \"model\": \"gpt-4-turbo\",\n", - " \"api_key\": OPENAI_API_KEY,\n", - " \"tags\": [\"mathagent-example\", \"tool\"],\n", - " }\n", - "]\n", - "\n", - "Operator = Literal[\"+\", \"-\", \"*\", \"/\"]\n", - "\n", - "\n", - "def calculator(a: int, b: int, operator: Annotated[Operator, \"operator\"]) -> int:\n", - " if operator == \"+\":\n", - " return a + b\n", - " elif operator == \"-\":\n", - " return a - b\n", - " elif operator == \"*\":\n", - " return a * b\n", - " elif operator == \"/\":\n", - " return int(a / b)\n", - " else:\n", - " raise ValueError(\"Invalid operator\")\n", - "\n", - "\n", - "# Create the agent that uses the LLM.\n", - "assistant = ConversableAgent(\n", - " name=\"Assistant\",\n", - " system_message=\"You are a helpful AI assistant. \"\n", - " \"You can help with simple calculations. \"\n", - " \"Return 'TERMINATE' when the task is done.\",\n", - " llm_config={\"config_list\": config_list},\n", - ")\n", - "\n", - "# The user proxy agent is used for interacting with the assistant agent\n", - "# and executes tool calls.\n", - "user_proxy = ConversableAgent(\n", - " name=\"User\",\n", - " llm_config=False,\n", - " is_termination_msg=lambda msg: msg.get(\"content\") is not None\n", - " and \"TERMINATE\" in msg[\"content\"],\n", - " human_input_mode=\"NEVER\",\n", - ")\n", - "\n", - "assistant.register_for_llm(name=\"calculator\", description=\"A simple calculator\")(\n", - " calculator\n", - ")\n", - "user_proxy.register_for_execution(name=\"calculator\")(calculator)\n", - "\n", - "# Register the calculator function to the two agents.\n", - "register_function(\n", - " calculator,\n", - " caller=assistant, # The assistant agent can suggest calls to the calculator.\n", - " executor=user_proxy, # The user proxy agent can execute the calculator calls.\n", - " name=\"calculator\", # By default, the function name is used as the tool name.\n", - " description=\"A simple calculator\", # A description of the tool.\n", - ")\n", - "\n", - "# Let the assistant start the conversation. It will end when the user types \"exit\".\n", - "try:\n", - " user_proxy.initiate_chat(\n", - " assistant, message=\"What is (1423 - 123) / 3 + (32 + 23) * 5?\"\n", - " )\n", - "except StdinNotImplementedError:\n", - " # This is only necessary for AgentOps testing automation which is headless and will not have user input\n", - " print(\"Stdin not implemented. Skipping initiate_chat\")\n", - " agentops.end_session(\"Indeterminate\")\n", - "\n", - "agentops.end_session(\"Success\")" - ] - }, - { - "cell_type": "markdown", - "id": "f67b0305-1247-489e-b1b0-829127af76d3", - "metadata": {}, - "source": [ - "You can see your run in action at [app.agentops.ai](app.agentops.ai). In this example, the AgentOps dashboard will show:\n", - "\n", - "* Agents talking to each other\n", - "* Each use of the `calculator` tool\n", - "* Each call to OpenAI for LLM use" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.5" - } - }, - "nbformat": 4, - "nbformat_minor": 5 - } - \ No newline at end of file + "cells": [ + { + "attachments": {}, + "cell_type": "markdown", + "id": "bb6538d8-2a5d-4a99-b2c1-7130963e4f7b", + "metadata": {}, + "source": [ + "# AG2 Tool Example\n", + "\n", + "\n", + "To get started, you'll need to install the AgentOps package and [set an API key](app.agentops.ai).\n", + "\n", + "AgentOps automatically configures itself when it's initialized meaning your agent run data will be tracked and logged to your AgentOps account right away." + ] + }, + { + "cell_type": "markdown", + "id": "083244fa", + "metadata": {}, + "source": [ + "First let's install the required packages" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9c8104ad", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -U pyautogen\n", + "%pip install -U agentops\n", + "%pip install -U python-dotenv" + ] + }, + { + "cell_type": "markdown", + "id": "cc44e459", + "metadata": {}, + "source": [ + "Then import them" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7672f591", + "metadata": {}, + "outputs": [], + "source": [ + "from autogen import ConversableAgent\n", + "from typing import Annotated, Literal\n", + "from autogen import ConversableAgent, register_function\n", + "import agentops\n", + "import os\n", + "from dotenv import load_dotenv\n", + "from IPython.core.error import (\n", + " StdinNotImplementedError,\n", + ") # only needed by AgentOps testing automation" + ] + }, + { + "cell_type": "markdown", + "id": "24f8bd70", + "metadata": {}, + "source": [ + "Next, we'll set our API keys. There are several ways to do this, the code below is just the most foolproof way for the purposes of this notebook. It accounts for both users who use environment variables and those who just want to set the API Key here in this notebook.\n", + "\n", + "[Get an AgentOps API key](https://agentops.ai/settings/projects)\n", + "\n", + "1. Create an environment variable in a .env file or other method. By default, the AgentOps `init()` function will look for an environment variable named `AGENTOPS_API_KEY`. Or...\n", + "\n", + "2. Replace `` below and pass in the optional `api_key` parameter to the AgentOps `init(api_key=...)` function. Remember not to commit your API key to a public repo!" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9eeaef34", + "metadata": {}, + "outputs": [], + "source": [ + "load_dotenv()\n", + "OPENAI_API_KEY = os.getenv(\"OPENAI_API_KEY\") or \"\"\n", + "AGENTOPS_API_KEY = os.getenv(\"AGENTOPS_API_KEY\") or \"\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d93f2339-7b99-4cf1-9232-c24faba49c7b", + "metadata": {}, + "outputs": [], + "source": [ + "agentops.init(AGENTOPS_API_KEY, default_tags=[\"autogen-tool-example\"])\n", + "\n", + "print(\"AgentOps is now running. You can view your session in the link above\")" + ] + }, + { + "cell_type": "markdown", + "id": "7858f0f6-9aca-4cdb-a514-9fbf7e353d50", + "metadata": {}, + "source": [ + "AutoGen will now start automatically tracking\n", + "\n", + "* LLM prompts and completions\n", + "* Token usage and costs\n", + "* Agent names and actions\n", + "* Correspondence between agents\n", + "* Tool usage\n", + "* Errors" + ] + }, + { + "cell_type": "markdown", + "id": "dc592637", + "metadata": {}, + "source": [ + "# Tool Example\n", + "AgentOps tracks when AG2 agents use tools. You can find more information on this example in [tool-use.ipynb](https://docs.ag2.ai/docs/tutorial/tool-use#tool-use)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9e4dfe37-85e0-4035-a314-3459c6e378c4", + "metadata": {}, + "outputs": [], + "source": [ + "# Define model, openai api key, tags, etc in the agent configuration\n", + "config_list = [\n", + " {\n", + " \"model\": \"gpt-4-turbo\",\n", + " \"api_key\": OPENAI_API_KEY,\n", + " \"tags\": [\"mathagent-example\", \"tool\"],\n", + " }\n", + "]\n", + "\n", + "Operator = Literal[\"+\", \"-\", \"*\", \"/\"]\n", + "\n", + "\n", + "def calculator(a: int, b: int, operator: Annotated[Operator, \"operator\"]) -> int:\n", + " if operator == \"+\":\n", + " return a + b\n", + " elif operator == \"-\":\n", + " return a - b\n", + " elif operator == \"*\":\n", + " return a * b\n", + " elif operator == \"/\":\n", + " return int(a / b)\n", + " else:\n", + " raise ValueError(\"Invalid operator\")\n", + "\n", + "\n", + "# Create the agent that uses the LLM.\n", + "assistant = ConversableAgent(\n", + " name=\"Assistant\",\n", + " system_message=\"You are a helpful AI assistant. \"\n", + " \"You can help with simple calculations. \"\n", + " \"Return 'TERMINATE' when the task is done.\",\n", + " llm_config={\"config_list\": config_list},\n", + ")\n", + "\n", + "# The user proxy agent is used for interacting with the assistant agent\n", + "# and executes tool calls.\n", + "user_proxy = ConversableAgent(\n", + " name=\"User\",\n", + " llm_config=False,\n", + " is_termination_msg=lambda msg: msg.get(\"content\") is not None\n", + " and \"TERMINATE\" in msg[\"content\"],\n", + " human_input_mode=\"NEVER\",\n", + ")\n", + "\n", + "assistant.register_for_llm(name=\"calculator\", description=\"A simple calculator\")(\n", + " calculator\n", + ")\n", + "user_proxy.register_for_execution(name=\"calculator\")(calculator)\n", + "\n", + "# Register the calculator function to the two agents.\n", + "register_function(\n", + " calculator,\n", + " caller=assistant, # The assistant agent can suggest calls to the calculator.\n", + " executor=user_proxy, # The user proxy agent can execute the calculator calls.\n", + " name=\"calculator\", # By default, the function name is used as the tool name.\n", + " description=\"A simple calculator\", # A description of the tool.\n", + ")\n", + "\n", + "# Let the assistant start the conversation. It will end when the user types \"exit\".\n", + "try:\n", + " user_proxy.initiate_chat(\n", + " assistant, message=\"What is (1423 - 123) / 3 + (32 + 23) * 5?\"\n", + " )\n", + "except StdinNotImplementedError:\n", + " # This is only necessary for AgentOps testing automation which is headless and will not have user input\n", + " print(\"Stdin not implemented. Skipping initiate_chat\")\n", + " agentops.end_session(\"Indeterminate\")\n", + "\n", + "agentops.end_session(\"Success\")" + ] + }, + { + "cell_type": "markdown", + "id": "f67b0305-1247-489e-b1b0-829127af76d3", + "metadata": {}, + "source": [ + "You can see your run in action at [app.agentops.ai](app.agentops.ai). In this example, the AgentOps dashboard will show:\n", + "\n", + "* Agents talking to each other\n", + "* Each use of the `calculator` tool\n", + "* Each call to OpenAI for LLM use" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.5" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} From 9c41ffbed8b4e46a18e1f9a7f29619f022e8158d Mon Sep 17 00:00:00 2001 From: kevin666aa Date: Mon, 23 Dec 2024 04:46:24 -0500 Subject: [PATCH 3/5] update --- examples/demos/agentchat_agentops.ipynb | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/examples/demos/agentchat_agentops.ipynb b/examples/demos/agentchat_agentops.ipynb index 2aa7a84e7..4fb5dd697 100644 --- a/examples/demos/agentchat_agentops.ipynb +++ b/examples/demos/agentchat_agentops.ipynb @@ -39,7 +39,7 @@ "id": "38182a5296dceb34", "metadata": {}, "source": [ - "## Adding AgentOps to an existing Autogen service.\n", + "## Adding AgentOps to an existing AG2 service.\n", "To get started, you'll need to install the AgentOps package and set an API key.\n", "\n", "AgentOps automatically configures itself when it's initialized meaning your agent run data will be tracked and logged to your AgentOps account right away." @@ -151,7 +151,7 @@ "id": "4dd8f461ccd9cbef", "metadata": {}, "source": [ - "Autogen will now start automatically tracking\n", + "AG2 will now start automatically tracking\n", "- LLM prompts and completions\n", "- Token usage and costs\n", "- Agent names and actions\n", @@ -218,7 +218,7 @@ "metadata": {}, "source": [ "# Tool Example\n", - "AgentOps also tracks when Autogen agents use tools. You can find more information on this example in [tool-use.ipynb](https://github.com/microsoft/autogen/blob/main/website/docs/tutorial/tool-use.ipynb)" + "AgentOps also tracks when AG2 agents use tools. You can find more information on this example in [tool-use.ipynb](https://docs.ag2.ai/docs/tutorial/tool-use#tool-use)" ] }, { From 284675787ace69ee44ceb61091a4424ff9399099 Mon Sep 17 00:00:00 2001 From: kevin666aa Date: Mon, 23 Dec 2024 04:54:30 -0500 Subject: [PATCH 4/5] update --- examples/autogen_examples/MathAgent.ipynb | 472 +++++++++++----------- 1 file changed, 236 insertions(+), 236 deletions(-) diff --git a/examples/autogen_examples/MathAgent.ipynb b/examples/autogen_examples/MathAgent.ipynb index e3a094ebd..124017bc2 100644 --- a/examples/autogen_examples/MathAgent.ipynb +++ b/examples/autogen_examples/MathAgent.ipynb @@ -1,237 +1,237 @@ { - "cells": [ - { - "attachments": {}, - "cell_type": "markdown", - "id": "bb6538d8-2a5d-4a99-b2c1-7130963e4f7b", - "metadata": {}, - "source": [ - "# AG2 Tool Example\n", - "\n", - "\n", - "To get started, you'll need to install the AgentOps package and [set an API key](app.agentops.ai).\n", - "\n", - "AgentOps automatically configures itself when it's initialized meaning your agent run data will be tracked and logged to your AgentOps account right away." - ] - }, - { - "cell_type": "markdown", - "id": "083244fa", - "metadata": {}, - "source": [ - "First let's install the required packages" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "9c8104ad", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -U pyautogen\n", - "%pip install -U agentops\n", - "%pip install -U python-dotenv" - ] - }, - { - "cell_type": "markdown", - "id": "cc44e459", - "metadata": {}, - "source": [ - "Then import them" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "7672f591", - "metadata": {}, - "outputs": [], - "source": [ - "from autogen import ConversableAgent\n", - "from typing import Annotated, Literal\n", - "from autogen import ConversableAgent, register_function\n", - "import agentops\n", - "import os\n", - "from dotenv import load_dotenv\n", - "from IPython.core.error import (\n", - " StdinNotImplementedError,\n", - ") # only needed by AgentOps testing automation" - ] - }, - { - "cell_type": "markdown", - "id": "24f8bd70", - "metadata": {}, - "source": [ - "Next, we'll set our API keys. There are several ways to do this, the code below is just the most foolproof way for the purposes of this notebook. It accounts for both users who use environment variables and those who just want to set the API Key here in this notebook.\n", - "\n", - "[Get an AgentOps API key](https://agentops.ai/settings/projects)\n", - "\n", - "1. Create an environment variable in a .env file or other method. By default, the AgentOps `init()` function will look for an environment variable named `AGENTOPS_API_KEY`. Or...\n", - "\n", - "2. Replace `` below and pass in the optional `api_key` parameter to the AgentOps `init(api_key=...)` function. Remember not to commit your API key to a public repo!" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "9eeaef34", - "metadata": {}, - "outputs": [], - "source": [ - "load_dotenv()\n", - "OPENAI_API_KEY = os.getenv(\"OPENAI_API_KEY\") or \"\"\n", - "AGENTOPS_API_KEY = os.getenv(\"AGENTOPS_API_KEY\") or \"\"" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "d93f2339-7b99-4cf1-9232-c24faba49c7b", - "metadata": {}, - "outputs": [], - "source": [ - "agentops.init(AGENTOPS_API_KEY, default_tags=[\"autogen-tool-example\"])\n", - "\n", - "print(\"AgentOps is now running. You can view your session in the link above\")" - ] - }, - { - "cell_type": "markdown", - "id": "7858f0f6-9aca-4cdb-a514-9fbf7e353d50", - "metadata": {}, - "source": [ - "AutoGen will now start automatically tracking\n", - "\n", - "* LLM prompts and completions\n", - "* Token usage and costs\n", - "* Agent names and actions\n", - "* Correspondence between agents\n", - "* Tool usage\n", - "* Errors" - ] - }, - { - "cell_type": "markdown", - "id": "dc592637", - "metadata": {}, - "source": [ - "# Tool Example\n", - "AgentOps tracks when AG2 agents use tools. You can find more information on this example in [tool-use.ipynb](https://docs.ag2.ai/docs/tutorial/tool-use#tool-use)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "9e4dfe37-85e0-4035-a314-3459c6e378c4", - "metadata": {}, - "outputs": [], - "source": [ - "# Define model, openai api key, tags, etc in the agent configuration\n", - "config_list = [\n", - " {\n", - " \"model\": \"gpt-4-turbo\",\n", - " \"api_key\": OPENAI_API_KEY,\n", - " \"tags\": [\"mathagent-example\", \"tool\"],\n", - " }\n", - "]\n", - "\n", - "Operator = Literal[\"+\", \"-\", \"*\", \"/\"]\n", - "\n", - "\n", - "def calculator(a: int, b: int, operator: Annotated[Operator, \"operator\"]) -> int:\n", - " if operator == \"+\":\n", - " return a + b\n", - " elif operator == \"-\":\n", - " return a - b\n", - " elif operator == \"*\":\n", - " return a * b\n", - " elif operator == \"/\":\n", - " return int(a / b)\n", - " else:\n", - " raise ValueError(\"Invalid operator\")\n", - "\n", - "\n", - "# Create the agent that uses the LLM.\n", - "assistant = ConversableAgent(\n", - " name=\"Assistant\",\n", - " system_message=\"You are a helpful AI assistant. \"\n", - " \"You can help with simple calculations. \"\n", - " \"Return 'TERMINATE' when the task is done.\",\n", - " llm_config={\"config_list\": config_list},\n", - ")\n", - "\n", - "# The user proxy agent is used for interacting with the assistant agent\n", - "# and executes tool calls.\n", - "user_proxy = ConversableAgent(\n", - " name=\"User\",\n", - " llm_config=False,\n", - " is_termination_msg=lambda msg: msg.get(\"content\") is not None\n", - " and \"TERMINATE\" in msg[\"content\"],\n", - " human_input_mode=\"NEVER\",\n", - ")\n", - "\n", - "assistant.register_for_llm(name=\"calculator\", description=\"A simple calculator\")(\n", - " calculator\n", - ")\n", - "user_proxy.register_for_execution(name=\"calculator\")(calculator)\n", - "\n", - "# Register the calculator function to the two agents.\n", - "register_function(\n", - " calculator,\n", - " caller=assistant, # The assistant agent can suggest calls to the calculator.\n", - " executor=user_proxy, # The user proxy agent can execute the calculator calls.\n", - " name=\"calculator\", # By default, the function name is used as the tool name.\n", - " description=\"A simple calculator\", # A description of the tool.\n", - ")\n", - "\n", - "# Let the assistant start the conversation. It will end when the user types \"exit\".\n", - "try:\n", - " user_proxy.initiate_chat(\n", - " assistant, message=\"What is (1423 - 123) / 3 + (32 + 23) * 5?\"\n", - " )\n", - "except StdinNotImplementedError:\n", - " # This is only necessary for AgentOps testing automation which is headless and will not have user input\n", - " print(\"Stdin not implemented. Skipping initiate_chat\")\n", - " agentops.end_session(\"Indeterminate\")\n", - "\n", - "agentops.end_session(\"Success\")" - ] - }, - { - "cell_type": "markdown", - "id": "f67b0305-1247-489e-b1b0-829127af76d3", - "metadata": {}, - "source": [ - "You can see your run in action at [app.agentops.ai](app.agentops.ai). In this example, the AgentOps dashboard will show:\n", - "\n", - "* Agents talking to each other\n", - "* Each use of the `calculator` tool\n", - "* Each call to OpenAI for LLM use" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.5" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} + "cells": [ + { + "attachments": {}, + "cell_type": "markdown", + "id": "bb6538d8-2a5d-4a99-b2c1-7130963e4f7b", + "metadata": {}, + "source": [ + "# AG2 Tool Example\n", + "\n", + "\n", + "To get started, you'll need to install the AgentOps package and [set an API key](app.agentops.ai).\n", + "\n", + "AgentOps automatically configures itself when it's initialized meaning your agent run data will be tracked and logged to your AgentOps account right away." + ] + }, + { + "cell_type": "markdown", + "id": "083244fa", + "metadata": {}, + "source": [ + "First let's install the required packages" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9c8104ad", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -U pyautogen\n", + "%pip install -U agentops\n", + "%pip install -U python-dotenv" + ] + }, + { + "cell_type": "markdown", + "id": "cc44e459", + "metadata": {}, + "source": [ + "Then import them" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7672f591", + "metadata": {}, + "outputs": [], + "source": [ + "from autogen import ConversableAgent\n", + "from typing import Annotated, Literal\n", + "from autogen import ConversableAgent, register_function\n", + "import agentops\n", + "import os\n", + "from dotenv import load_dotenv\n", + "from IPython.core.error import (\n", + " StdinNotImplementedError,\n", + ") # only needed by AgentOps testing automation" + ] + }, + { + "cell_type": "markdown", + "id": "24f8bd70", + "metadata": {}, + "source": [ + "Next, we'll set our API keys. There are several ways to do this, the code below is just the most foolproof way for the purposes of this notebook. It accounts for both users who use environment variables and those who just want to set the API Key here in this notebook.\n", + "\n", + "[Get an AgentOps API key](https://agentops.ai/settings/projects)\n", + "\n", + "1. Create an environment variable in a .env file or other method. By default, the AgentOps `init()` function will look for an environment variable named `AGENTOPS_API_KEY`. Or...\n", + "\n", + "2. Replace `` below and pass in the optional `api_key` parameter to the AgentOps `init(api_key=...)` function. Remember not to commit your API key to a public repo!" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9eeaef34", + "metadata": {}, + "outputs": [], + "source": [ + "load_dotenv()\n", + "OPENAI_API_KEY = os.getenv(\"OPENAI_API_KEY\") or \"\"\n", + "AGENTOPS_API_KEY = os.getenv(\"AGENTOPS_API_KEY\") or \"\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d93f2339-7b99-4cf1-9232-c24faba49c7b", + "metadata": {}, + "outputs": [], + "source": [ + "agentops.init(AGENTOPS_API_KEY, default_tags=[\"autogen-tool-example\"])\n", + "\n", + "print(\"AgentOps is now running. You can view your session in the link above\")" + ] + }, + { + "cell_type": "markdown", + "id": "7858f0f6-9aca-4cdb-a514-9fbf7e353d50", + "metadata": {}, + "source": [ + "AG2 will now start automatically tracking\n", + "\n", + "* LLM prompts and completions\n", + "* Token usage and costs\n", + "* Agent names and actions\n", + "* Correspondence between agents\n", + "* Tool usage\n", + "* Errors" + ] + }, + { + "cell_type": "markdown", + "id": "dc592637", + "metadata": {}, + "source": [ + "# Tool Example\n", + "AgentOps tracks when AG2 agents use tools. You can find more information on this example in [tool-use.ipynb](https://docs.ag2.ai/docs/tutorial/tool-use#tool-use)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9e4dfe37-85e0-4035-a314-3459c6e378c4", + "metadata": {}, + "outputs": [], + "source": [ + "# Define model, openai api key, tags, etc in the agent configuration\n", + "config_list = [\n", + " {\n", + " \"model\": \"gpt-4-turbo\",\n", + " \"api_key\": OPENAI_API_KEY,\n", + " \"tags\": [\"mathagent-example\", \"tool\"],\n", + " }\n", + "]\n", + "\n", + "Operator = Literal[\"+\", \"-\", \"*\", \"/\"]\n", + "\n", + "\n", + "def calculator(a: int, b: int, operator: Annotated[Operator, \"operator\"]) -> int:\n", + " if operator == \"+\":\n", + " return a + b\n", + " elif operator == \"-\":\n", + " return a - b\n", + " elif operator == \"*\":\n", + " return a * b\n", + " elif operator == \"/\":\n", + " return int(a / b)\n", + " else:\n", + " raise ValueError(\"Invalid operator\")\n", + "\n", + "\n", + "# Create the agent that uses the LLM.\n", + "assistant = ConversableAgent(\n", + " name=\"Assistant\",\n", + " system_message=\"You are a helpful AI assistant. \"\n", + " \"You can help with simple calculations. \"\n", + " \"Return 'TERMINATE' when the task is done.\",\n", + " llm_config={\"config_list\": config_list},\n", + ")\n", + "\n", + "# The user proxy agent is used for interacting with the assistant agent\n", + "# and executes tool calls.\n", + "user_proxy = ConversableAgent(\n", + " name=\"User\",\n", + " llm_config=False,\n", + " is_termination_msg=lambda msg: msg.get(\"content\") is not None\n", + " and \"TERMINATE\" in msg[\"content\"],\n", + " human_input_mode=\"NEVER\",\n", + ")\n", + "\n", + "assistant.register_for_llm(name=\"calculator\", description=\"A simple calculator\")(\n", + " calculator\n", + ")\n", + "user_proxy.register_for_execution(name=\"calculator\")(calculator)\n", + "\n", + "# Register the calculator function to the two agents.\n", + "register_function(\n", + " calculator,\n", + " caller=assistant, # The assistant agent can suggest calls to the calculator.\n", + " executor=user_proxy, # The user proxy agent can execute the calculator calls.\n", + " name=\"calculator\", # By default, the function name is used as the tool name.\n", + " description=\"A simple calculator\", # A description of the tool.\n", + ")\n", + "\n", + "# Let the assistant start the conversation. It will end when the user types \"exit\".\n", + "try:\n", + " user_proxy.initiate_chat(\n", + " assistant, message=\"What is (1423 - 123) / 3 + (32 + 23) * 5?\"\n", + " )\n", + "except StdinNotImplementedError:\n", + " # This is only necessary for AgentOps testing automation which is headless and will not have user input\n", + " print(\"Stdin not implemented. Skipping initiate_chat\")\n", + " agentops.end_session(\"Indeterminate\")\n", + "\n", + "agentops.end_session(\"Success\")" + ] + }, + { + "cell_type": "markdown", + "id": "f67b0305-1247-489e-b1b0-829127af76d3", + "metadata": {}, + "source": [ + "You can see your run in action at [app.agentops.ai](app.agentops.ai). In this example, the AgentOps dashboard will show:\n", + "\n", + "* Agents talking to each other\n", + "* Each use of the `calculator` tool\n", + "* Each call to OpenAI for LLM use" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.5" + } + }, + "nbformat": 4, + "nbformat_minor": 5 + } From 887521143552a6c19c986d363a419ea3131d70cf Mon Sep 17 00:00:00 2001 From: kevin666aa Date: Wed, 8 Jan 2025 08:47:03 -0500 Subject: [PATCH 5/5] update --- docs/images/external/autogen/ag2_blue.svg | 35 +++++++++++++++++++++++ docs/v1/examples/examples.mdx | 2 +- docs/v1/usage/sdk-reference.mdx | 2 +- 3 files changed, 37 insertions(+), 2 deletions(-) create mode 100644 docs/images/external/autogen/ag2_blue.svg diff --git a/docs/images/external/autogen/ag2_blue.svg b/docs/images/external/autogen/ag2_blue.svg new file mode 100644 index 000000000..28043ea47 --- /dev/null +++ b/docs/images/external/autogen/ag2_blue.svg @@ -0,0 +1,35 @@ + + + + + + + + + + diff --git a/docs/v1/examples/examples.mdx b/docs/v1/examples/examples.mdx index 90bd7e494..d5f73774f 100644 --- a/docs/v1/examples/examples.mdx +++ b/docs/v1/examples/examples.mdx @@ -33,7 +33,7 @@ mode: "wide" Track observations from Claude, Haiku and Sonnet series of models - } iconType="image" href="/v1/integrations/autogen"> + } iconType="image" href="/v1/integrations/autogen"> AG2 (Formerly AutoGen) multi-agent conversible workflow with tool usage diff --git a/docs/v1/usage/sdk-reference.mdx b/docs/v1/usage/sdk-reference.mdx index 1e5934026..ae0690c3d 100644 --- a/docs/v1/usage/sdk-reference.mdx +++ b/docs/v1/usage/sdk-reference.mdx @@ -114,7 +114,7 @@ Set the parent API key which has visibility over projects it is a parent of. ### `stop_instrumenting()` Stops instrumenting LLM calls. This is typically used by agent frameworks (i.e., [CrewAI](/v1/integrations/crewai) and -[ag2](/v1/integrations/autogen)) to stop using AgentOps' auto-instrumentation of LLM libraries such as OpenAI. This +[AG2](/v1/integrations/autogen)) to stop using AgentOps' auto-instrumentation of LLM libraries such as OpenAI. This allows these frameworks to use their own instrumenting or callback handler.