Skip to content

Commit

Permalink
improvement: refactoring the sample project in agentUniverse.
Browse files Browse the repository at this point in the history
  • Loading branch information
EdwinInAu committed Oct 30, 2024
1 parent 8ad3703 commit 063fae1
Show file tree
Hide file tree
Showing 295 changed files with 438 additions and 486 deletions.
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,6 @@ cython_debug/
.DS_Store

*.sqlite3
*.db

*.pyc
__pycache__/
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ pip install magent-ui ruamel.yaml

**One-click Run**

Run the [product_application.py](sample_standard_app/app/bootstrap/product_application.py) file located in sample_standard_app/app/bootstrap for a one-click start.
Run the [product_application.py](sample_standard_app/boostrap/platform/product_application.py) file located in sample_standard_app/bootstrap for a one-click start.

For more details, refer to [Quick Start for Product Platform](./docs/guidebook/en/10_1_1_Product%20Platform%20Quick%20Start.md) and the [Advanced Guide](./docs/guidebook/en/10_1_2_Product_Platform_Advancement_Guide.md).

Expand Down
2 changes: 1 addition & 1 deletion README_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ pip install magent-ui ruamel.yaml

**一键运行**

运行sample_standard_app/app/boostrap下的[product_application.py](sample_standard_app/app/bootstrap/product_application.py)文件,一键启动。
运行sample_standard_app/boostrap下的[product_application.py](sample_standard_app/boostrap/platform/product_application.py)文件,一键启动。

更多详情参考 [产品化平台快速开始](./docs/guidebook/zh/10_1_1_产品化平台快速开始.md)
[产品化平台进阶指南](./10_1_2_产品化平台进阶指南.md)
Expand Down
2 changes: 1 addition & 1 deletion agentuniverse/agent_serve/web/dal/request_library.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ def __init__(self, configer: Configer = None):
if system_db_uri and system_db_uri.strip():
pass
else:
db_path = get_project_root_path() / 'DB' / 'agent_universe.db'
db_path = get_project_root_path() / 'intelligence' / 'db' / 'agent_universe.db'
db_path.parent.mkdir(parents=True, exist_ok=True)
system_db_uri = f'sqlite:///{db_path}'

Expand Down
12 changes: 9 additions & 3 deletions agentuniverse/base/agentuniverse.py
Original file line number Diff line number Diff line change
Expand Up @@ -59,9 +59,9 @@ def start(self, config_path: str = None, core_mode: bool = False):
# get default config path
project_root_path = get_project_root_path()
sys.path.append(str(project_root_path.parent))
app_path = project_root_path / 'app'
if app_path.exists():
sys.path.append(str(app_path))
# TODO
self._add_to_sys_path(project_root_path, ['intelligence', 'app'])

if not config_path:
config_path = project_root_path / 'config' / 'config.toml'
config_path = str(config_path)
Expand Down Expand Up @@ -288,3 +288,9 @@ def __dynamic_import_and_init(self, class_path: str, configer: Configer):
module = importlib.import_module(module_path)
cls = getattr(module, class_name)
cls(configer)

def _add_to_sys_path(self, root_path, sub_dirs):
for sub_dir in sub_dirs:
app_path = root_path / sub_dir
if app_path.exists():
sys.path.append(str(app_path))
14 changes: 8 additions & 6 deletions agentuniverse_product/agentuniverse_product.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,9 +41,7 @@ def start(self, config_path: str = None):
# get default config path
project_root_path = get_project_root_path()
sys.path.append(str(project_root_path.parent))
app_path = project_root_path / 'app'
if app_path.exists():
sys.path.append(str(app_path))
self._add_to_sys_path(project_root_path, ['platform', 'app'])
if not config_path:
config_path = project_root_path / 'config' / 'config.toml'
config_path = str(config_path)
Expand All @@ -62,16 +60,14 @@ def start(self, config_path: str = None):
# start the product ui server
config = configer.value.get('MAGENT_UI', {})
try:
from magent_ui import launch
from magent_ui import launch
except ImportError as e:
print(e)
raise ImportError(
"Could not start product server provided by magent-ui."
" Please install it with `pip install magent-ui ruamel.yaml`."
)
launch(**config)



def __init_product_tables(self):
"""Initialize the product tables including session and message tables."""
Expand Down Expand Up @@ -119,3 +115,9 @@ def __register(self, product_configer_list: list[ComponentConfiger]):
continue
product_instance.component_config_path = product_configer.configer.path
ProductManager().register(product_instance.get_instance_code(), product_instance)

def _add_to_sys_path(self, root_path, sub_dirs):
for sub_dir in sub_dirs:
app_path = root_path / sub_dir
if app_path.exists():
sys.path.append(str(app_path))
4 changes: 2 additions & 2 deletions agentuniverse_product/base/constant/llm_constant.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@
'demo_llm': ['gpt-3.5-turbo', 'gpt-4o', 'gpt-4-turbo', 'gpt-4o-mini', 'gpt-4'],
'default_openai_llm': ['gpt-3.5-turbo', 'gpt-4o', 'gpt-4-turbo', 'gpt-4o-mini', 'gpt-4'],
'qwen_llm': ['qwen-max', 'qwen-long', 'qwen-turbo', 'qwen2-72b-instruct', 'qwen2-7b-instruct', 'qwen1.5-110b-chat',
'qwen1.5-72b-chat', 'qwen1.5-7b-chat'],
'qwen1.5-72b-chat', 'qwen1.5-7b-chat', 'qwen2.5-72b-instruct'],
'default_qwen_llm': ['qwen-max', 'qwen-long', 'qwen-turbo', 'qwen2-72b-instruct', 'qwen2-7b-instruct',
'qwen1.5-110b-chat', 'qwen1.5-72b-chat', 'qwen1.5-7b-chat'],
'qwen1.5-110b-chat', 'qwen1.5-72b-chat', 'qwen1.5-7b-chat', 'qwen2.5-72b-instruct'],
'wenxin_llm': ['ERNIE-Speed-AppBuilder-8K-0516', 'ERNIE-Lite-8K-0725', 'ERNIE-Speed-128K', 'ERNIE-3.5-128K',
'ERNIE-3.5-8K-0701', 'ERNIE-4.0-8K-0613', 'ERNIE-4.0-8K-Preview', 'ERNIE-3.5-8K-Preview',
'ERNIE-Tiny-8K', 'ERNIE-4.0-8K-Latest', 'ERNIE-4.0-Turbo-8K'],
Expand Down
4 changes: 2 additions & 2 deletions agentuniverse_product/service/agent_service/agent_service.py
Original file line number Diff line number Diff line change
Expand Up @@ -280,14 +280,14 @@ def create_agent(agent_dto: AgentDTO) -> str:

# write product YAML file
product_file_name = f"{agent_dto.id}_product"
product_file_path = os.path.join('...', 'platform', 'difizen', 'product', 'agent', f"{product_file_name}.yaml")
product_file_path = os.path.join('..', '..', 'platform', 'difizen', 'product', 'agent', f"{product_file_name}.yaml")
write_yaml_file(product_file_path, product_config_data)

# assemble agent config data
agent_config_data = assemble_agent_config_data(agent_dto)

# write agent YAML file
agent_file_path = os.path.join('..,', 'intelligence', 'agentic', 'agent', f"{agent_dto.id}.yaml")
agent_file_path = os.path.join('..', '..', 'intelligence', 'agentic', 'agent', f"{agent_dto.id}.yaml")
write_yaml_file(agent_file_path, agent_config_data)

# register product and agent instance
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ def create_knowledge(knowledge_dto: KnowledgeDTO) -> str:
knowledge_id = knowledge_dto.id

product_file_name = f"{knowledge_id}_product"
product_file_path = os.path.join("...", "platform", "difizen", "product", "knowledge",
product_file_path = os.path.join("..", "..", "platform", "difizen", "product", "knowledge",
f"{product_file_name}.yaml")

product_config_data = assemble_knowledge_product_config_data(knowledge_dto)
Expand All @@ -74,7 +74,7 @@ def create_knowledge(knowledge_dto: KnowledgeDTO) -> str:
except Exception as e:
raise e

knowledge_file_path = os.path.join("...", "intelligence", "agentic", "knowledge", f"{knowledge_id}.yaml")
knowledge_file_path = os.path.join("..", "..", "intelligence", "agentic", "knowledge", f"{knowledge_id}.yaml")

knowledge_config = assemble_knowledge_config(knowledge_dto)

Expand Down Expand Up @@ -170,11 +170,11 @@ def upload_knowledge_file(knowledge_id: str, file) -> bool:
bool: True if the file is uploaded successfully, False otherwise.
"""

upload_file_path = os.path.join("..", "resources")
upload_file_path = os.path.join("..", "..", "platform", "difizen", "resources")

try:
if not os.path.exists(upload_file_path):
os.makedirs(os.path.dirname(upload_file_path), exist_ok=True)
os.makedirs(upload_file_path, exist_ok=True)

file_location = os.path.join(upload_file_path, file.filename)

Expand All @@ -185,7 +185,7 @@ def upload_knowledge_file(knowledge_id: str, file) -> bool:
print(f"upload file failed: {e}")
raise e

knowledge_store_file_path = os.path.join("...", "intelligence", "agentic", "knowledge", "store",
knowledge_store_file_path = os.path.join("..", "..", "intelligence", "agentic", "knowledge", "store",
f"{knowledge_id}_store.yaml")
knowledge_store_name = f"{knowledge_id}_chroma_store"

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ def create_plugin_with_openapi(plugin_dto: PluginDTO) -> str:
product_config_data = assemble_plugin_product_config_data(plugin_dto, tool_id_list)
# write product YAML file
product_file_name = f"{plugin_dto.id}_product"
product_file_path = os.path.join('...', 'platform', 'difizen', 'product', 'plugin', f"{product_file_name}.yaml")
product_file_path = os.path.join('..', '..', 'platform', 'difizen', 'product', 'plugin', f"{product_file_name}.yaml")
write_yaml_file(product_file_path, product_config_data)
register_product(product_file_path)
return plugin_dto.id
4 changes: 2 additions & 2 deletions agentuniverse_product/service/tool_service/tool_service.py
Original file line number Diff line number Diff line change
Expand Up @@ -80,14 +80,14 @@ def create_tool(tool_dto: ToolDTO) -> str:

# write product YAML file
product_file_name = f"{tool_dto.id}_product"
product_file_path = os.path.join('...', 'platform', 'difizen', 'product', 'tool', f"{product_file_name}.yaml")
product_file_path = os.path.join('..', '..', 'platform', 'difizen', 'product', 'tool', f"{product_file_name}.yaml")
write_yaml_file(product_file_path, product_config_data)

# assemble tool config data
tool_config_data = assemble_api_tool_config_data(tool_dto)

# write tool YAML file
tool_file_path = os.path.join('...', 'intelligence', 'agentic', 'tool', f"{tool_dto.id}.yaml")
tool_file_path = os.path.join('..', '..', 'intelligence', 'agentic', 'tool', f"{tool_dto.id}.yaml")
write_yaml_file(tool_file_path, tool_config_data)

# register product and tool instance
Expand Down
12 changes: 5 additions & 7 deletions agentuniverse_product/service/util/agent_util.py
Original file line number Diff line number Diff line change
Expand Up @@ -99,15 +99,12 @@ def assemble_agent_config_data(agent_dto: AgentDTO) -> Dict:
'name': agent_dto.id,
'description': agent_dto.description
},
'profile': {},
'plan': {
'planner': {
'name': agent_dto.planner.id,
'workflow_id': agent_dto.planner.workflow_id
}
'profile': {
},
'action': {}
}
if agent_dto.planner.workflow_id:
agent_config_data['profile']['workflow_id'] = agent_dto.planner.workflow_id

if agent_dto.llm:
llm = LLMManager().get_instance_obj(agent_dto.llm.id)
Expand Down Expand Up @@ -213,11 +210,12 @@ def get_planner_dto(agent: Agent) -> PlannerDTO | None:
planner_name = planner.get('name') or get_default_planner_name(agent)
if not planner_name:
return None
workflow_id = planner.get('workflow_id') if planner.get('workflow_id') else agent_model.profile.get('workflow_id')
members = None
if planner_name == 'peer_planner':
members = assemble_peer_planner_members(planner, agent_model)
return PlannerDTO(nickname='', id=planner_name, members=members,
workflow_id=planner.get('workflow_id'))
workflow_id=workflow_id)


def assemble_peer_planner_members(planner: dict, agent_model: AgentModel) -> list[AgentDTO]:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ def create_workflow(workflow_dto: WorkflowDTO) -> str:
}
# TODO db/
# write workflow YAML file
workflow_file_path = os.path.join('...', 'platform', 'difizen', 'workflow', f"{workflow_id}.yaml")
workflow_file_path = os.path.join('..', '..', 'platform', 'difizen', 'workflow', f"{workflow_id}.yaml")
write_yaml_file(workflow_file_path, workflow_config)
# register workflow instance
register_workflow(workflow_file_path)
Expand Down
4 changes: 2 additions & 2 deletions docs/guidebook/en/10_1_1_Product Platform Quick Start.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ In this section, we will show you how to:

## Environment and Application Engineering Preparation
### Application Engineering Preparation
We have placed the **product module samples** in the agentUniverse’s sample_standard_app project. You can view them [here](../../../sample_standard_app/app/core/product). This part can be configured in the background through YAML, and of course, these functions can be automatically created and managed through the product page.
We have placed the **product module samples** in the agentUniverse’s sample_standard_app project. You can view them [here](../../../sample_standard_app/platform/difizen/product). This part can be configured in the background through YAML, and of course, these functions can be automatically created and managed through the product page.

### Installing Dependencies
**Using pip**
Expand All @@ -33,7 +33,7 @@ Of course, when using the agent, you need to pre-configure the various LLM model

## Using the agentUniverse Product Platform
### Starting the Product Service
Run the [product_application](../../../sample_standard_app/app/bootstrap/product_application.py) file located in `sample_standard_app/app/bootstrap` to start with one click.
Run the [product_application](../../../sample_standard_app/boostrap/platform/product_application.py) file located in `sample_standard_app/bootstrap` to start with one click.

![img.png](../_picture/product_start.png)

Expand Down
4 changes: 2 additions & 2 deletions docs/guidebook/en/1_3_Quick_Start.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ action:
- 'google_search_tool'
metadata:
type: 'AGENT'
module: 'sample_standard_app.app.core.agent.rag_agent_case.demo_rag_agent'
module: 'sample_standard_app.intelligence.agentic.agent.agent_case.rag_agent_case.demo_rag_agent'
class: 'DemoRagAgent'
```
In `xx_agent_case_a.yaml`, we define the configuration of a `DemoRagAgent`. The `info` part is the basic information setting of the agent, the `profile` part contains the setting of the agent using the llm model, the `plan` part contains the behavior planning setting of the agent (determining the working mode of the agent), the `action` part contains the setting of the tools and knowledge that the agent can use, and the `metadata` part contains the metadata setting of the agent object.
Expand Down Expand Up @@ -158,7 +158,7 @@ plan:
reviewing: 'demo_reviewing_agent'
metadata:
type: 'AGENT'
module: 'sample_standard_app.app.core.agent.peer_agent_case.demo_peer_agent'
module: 'sample_standard_app.intelligence.agentic.agent.agent_case.peer_agent_case.demo_peer_agent'
class: 'DemoPeerAgent'
```
In `xx_agent_case_b.yaml`, we define the configuration of a `DemoPeerAgent`. The `info` part is the basic information setting of the agent, the `plan` part contains the behavior planning setting of the agent (determining the working mode of the agent), and the `metadata` part contains the metadata setting of the agent object.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -78,15 +78,15 @@ As demonstrated by the recommended directory structure for the project, the proj
# Perform a full component scan and registration for all the paths under this list.
default = ['sample_standard_app.app.core']
# Scan and register agent components for all paths under this list, with priority over the default.
agent = ['sample_standard_app.app.core.agent']
agent = ['sample_standard_app.intelligence.agentic.agent']
# Scan and register agent components for all paths under this list, with priority over the default.
knowledge = ['sample_standard_app.app.core.knowledge']
# Scan and register knowledge components for all paths under this list, with priority over the default.
llm = ['sample_standard_app.app.core.llm']
# Scan and register llm components for all paths under this list, with priority over the default.
planner = ['sample_standard_app.app.core.planner']
# Scan and register planner components for all paths under this list, with priority over the default.
tool = ['sample_standard_app.app.core.tool']
tool = ['sample_standard_app.intelligence.agentic.tool']
# Scan and register memory components for all paths under this list, with priority over the default.
memory = ['sample_standard_app.app.core.memory']
# Scan and register service components for all paths under this list, with priority over the default.
Expand Down
10 changes: 5 additions & 5 deletions docs/guidebook/en/2_2_1_Agent_Create_And_Use.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,12 +101,12 @@ action:
# - 'knowledge_a'
metadata:
type: 'AGENT'
module: 'sample_standard_app.app.core.agent.rag_agent_case.demo_rag_agent'
module: 'sample_standard_app.intelligence.agentic.agent.agent_case.rag_agent_case.demo_rag_agent'
class: 'DemoRagAgent'
```
The above is an actual example of an agent configuration. In addition to the standard configuration items introduced above, the observant among you may have noticed variables in the prompt like `{background}` and `{input}`. This is a very practical prompt replacement feature, which we will explain in the section [How to dynamically adjust settings based on user input](#How to dynamically adjust settings based on user input).

You can find more agent configuration YAML examples in our sample project under the path `sample_standard_app.app.core.agent`.
You can find more agent configuration YAML examples in our sample project under the path `sample_standard_app.intelligence.agentic.agent`.

In addition to this, agentUniverse does not restrict users from extending the YAML configuration content for agents. You can create any custom configuration key according to your own requirements, but please be careful not to duplicate the default configuration keywords mentioned above.

Expand Down Expand Up @@ -251,7 +251,7 @@ Taking the configuration of the example project as a reference, it would be as f
```yaml
[CORE_PACKAGE]
# Scan and register agent components for all paths under this list, with priority over the default.
agent = ['sample_standard_app.app.core.agent']
agent = ['sample_standard_app.intelligence.agentic.agent']
```
## Other techniques for agent development
Expand All @@ -264,15 +264,15 @@ In the [Creating Agent Domain Behavior Definitions](#Creating Agent Domain Behav
In [An actual example of an agent configuration](#An actual example of an agent configuration.) section of this document, the prompt includes variables like `{background}`,`{input}`, etc. This feature is the prompt variable template replacement function, aimed at dynamically influencing the prompt based on the user's input. One only needs to define the text using `{variable}` format in the agent configuration settings section, and define it in the parse_input's `agent_input` to dynamically replace the corresponding prompt based on the input portion.
For example, in the sample agent `sample_standard_app.app.core.agent.rag_agent_case.demo_rag_agent.py`, there is the following `parse_input` method.
For example, in the sample agent `sample_standard_app.intelligence.agentic.agent.agent_case.rag_agent_case.demo_rag_agent.py`, there is the following `parse_input` method.
```text
def parse_input(self, input_object: InputObject, agent_input: dict) -> dict:
agent_input['input'] = input_object.get_data('input')
return agent_input
```
In its agent settings `sample_standard_app.app.core.agent.rag_agent_case.demo_rag_agent.yaml`, in the `instruction` section, we can see the following configuration.
In its agent settings `sample_standard_app.intelligence.agentic.agent.agent_case.rag_agent_case.demo_rag_agent.yaml`, in the `instruction` section, we can see the following configuration.
```text
instruction: |
Expand Down
Loading

0 comments on commit 063fae1

Please sign in to comment.