[Autobuild] Introducing Captain Agent #21
Conversation
Merge branch 'main' into meta
LeoLjl
requested review from
sonichi,
qingyun-wu,
skzhang1,
yiranwu0 and
marklysze
|
@LeoLjl could you please check the failing docs/checks CI? @qingyun-wu could you please set up the openai CI? |
There was a problem hiding this comment.
My suggestion of breaking down the PRs:
- doc + blogpost + test. Out-of-the-box usage of CaptainAgent. Shortest code example. No customization.
- Working code for the documented functionality in (1) which passes the test.
- Add configurable options in doc + blogpost + test.
- Working code for the documented functionality in (3) which passes the test.
We can keep this PR as a draft for having an overall understanding. Then create new PRs following (1)-(4). (1) and (2) are minimal requirements for initial release. We'll need focused discussion for each PR, and each PR will depend on previous ones.
What do you think?
| We provide an agent library and a tool library for Captain Agent to choose from when building the team. In the following section, we demonstrate how to use the libraries. | ||
|
|
||
| # Using Agent Library | ||
| To use captain agent, we need to instantiate a CaptainAgent and the corresponding CaptainUserProxy, then initiate a chat. To we just need to add a line of configuration to the settings. |
There was a problem hiding this comment.
It's not clear why we need to use two special agent classes here. CaptainAgent is easy to understand. Why is a special CaptainUserProxy is needed? A user is supposed to use CaptainAgent together with any other agents, in my understanding.
| # Using Agent Library | ||
| To use captain agent, we need to instantiate a CaptainAgent and the corresponding CaptainUserProxy, then initiate a chat. To we just need to add a line of configuration to the settings. | ||
|
|
||
| ```python |
There was a problem hiding this comment.
The sample code is too long. Let's think about a shortest example possible. We'd like a user to simply import CaptainAgent and use it as a stronger alternative of AssistantAgent out of the box, without any extra configuration. They can customize some options, but let's use different examples: one example for out-of-the-box use, and another example to explain customization options.
| "name": "WebServing_Expert", | ||
| "description": "A helpful assistant with access to a web browser. Ask them to perform web searches, open pages, navigate to Wikipedia, answer questions from pages, and or generate summaries.", | ||
| "system_message": "", | ||
| "model_path": "autogen/agentchat/contrib/web_surfer/WebSurferAgent", |
There was a problem hiding this comment.
"model_path" is the field name but the value is the path for an agent class.
| @@ -0,0 +1,73 @@ | |||
| # Details of all the available configurations | |||
| Captain Agent requires `nested_mode_config` for configuration. Below is an example, we will break it down and provide a detailed explanation. | |||
There was a problem hiding this comment.
Can we stage the PRs? In the first PR, don't expose any tunable configuration. In the next PR, we can carefully discuss what configuration makes sense to be tunable. The reason is that once a config is added to user-facing API, removing it is a technical dept. Unless we are sure a config is good to keep, we can add it later.
| ## ``autobuild_init_config`` | ||
| This section is used to configure the initial setup of autobuild. | ||
|
|
||
| ### ``config_file_or_env`` |
There was a problem hiding this comment.
This is an example of config that I don't think should appear. We can review them in a separate PR if we stage the PRs.
Co-authored-by: Chi Wang <4250911+sonichi@users.noreply.github.com>
Of course. However, I am not sure if we can get it delivered on time given the tight scedule if we break it down and merge separately. |
|
@LeoLjl for the interface, let's just do sth like this captain_agent = CaptainAgent(
name="captain_agent",
llm_config=llm_config,
agent_lib="xxxx",
tool_lib="xxxx",
code_execution_config={"use_docker": False, "work_dir": "groupchat"},
) |
This PR introduces CaptainAgent.
CaptainAgent is an agent that can break down tasks and dynamically call AutoBuild as a tool to solve the subtasks. In one tool call, the CaptainAgent will provide the building task, execution task and group name. The building task is used to assemble group of experts, the execution task is what the experts need to solve.
We propose a RAG-like pipeline for expert group generation. Given a building task, we perform similarity search in agent library and find topk candidates, then let LLM to select the best fit. If no expert is suitable for the task, new experts will be generated. Tools will be retrieved and binded to the experts.
Below is the roadmap for merging it.
Why are these changes needed?
Related issue number
Checks