Fix/openapi plugin (#311)

lylalala · wangyijunlyy · tuhahaha · zzhangpurdue · commit fd29af468d48 · 2024-02-27T17:07:54.000+08:00
* update * add style_repaint * 恢复 * 恢复 * add style_repaint tool * modify style_repaint * fix error * fix rate limit * improve stability * use dashscope to convert url * modify and better * fix * refact framework: add base class for llm, tool, and agent * refact llm * refact llm for custom * add role_play agent * add unittest for role-play agent * refactor/tool/image_gen * fix pre-commit * add agent_builder agent * revert modelscope_agent/agent.py * update agent and add react prompt * adapt agentbuilder in apps * update style_repaint * update style_repaint * change pipeline plugin tool (#248) * change pipeline plugin tool * Standardized input and output * Standardized input and output * Standardized input and output * refactor ms agent * save system to history * avoid duplicated system in roleplay * builder with history running * test case * update tool * update * modify import method * code interpreter and knowledge * running code interpreter with file * Refactor/citest (#257) * add citest * cache dependency * cache dependency update 1 * cache dependency update 2 * cache dependency update 3 --------- Co-authored-by: Zhicheng Zhang <zhangzhicheng.zzc@alibaba-inc.com> * add modelscope tool (#258) * add modelscope tool * add modelscope tool * add modelscope tool --------- Co-authored-by: Zhicheng Zhang <zczhang.hvac@gmail.com> * unittest-web search web browser * update tool and unittest * Refactor/gradio4 (#260) * Feat/gradio4 (#229) * feat: update gradio 4 * feat: chatbot requirement * feat: update app.py to gradio 4 * fix: gradio 4 of process_configuration * fix: typo * fix: mschatbot * feat: chatbot support copy --------- Co-authored-by: nil.wyw <nil.wyw@alibaba-inc.com> * feat: update gradio 4 * feat: fileupload * feat: MultimodalInput for appBot * fix: ui * feat: MultimodalInput for app.py * feat: Textbox => MultimodalInput * bump version to 0.2.4rc0 * fix: lint * fix: lint * fix: lint * fix: lint * fix: add quotes for src prop (#249) * feat: update gradio 4 * feat: chatbot requirement * feat: update app.py to gradio 4 * fix: gradio 4 of process_configuration * fix: typo * fix: mschatbot * feat: chatbot support copy * feat: fileupload * fix: gradio 4 dataframe update * fix: appBot * fix: dataframe display when config update * fix: src format * fix: files path --------- Co-authored-by: nil.wyw <nil.wyw@alibaba-inc.com> * fix pre-commit * fix submit input. * fix multimodal * fix bugs & support appBot.py * fix pre-commit * fix auto-merge --------- Co-authored-by: wyw <1006629314@qq.com> Co-authored-by: nil.wyw <nil.wyw@alibaba-inc.com> Co-authored-by: Col0ring <1561999073@qq.com> Co-authored-by: wenmeng.zwm <wenmeng.zwm@alibaba-inc.com> Co-authored-by: Col0ring <47329987+Col0ring@users.noreply.github.com> Co-authored-by: suluyan.sly <suluyan.sly@alibaba-inc.com> Co-authored-by: skyline2006 <skyline2006@163.com> * update reame and tool cfg * Refactor/unittest (#264) * update unittest * update requirements * update requirements * update requirements * update requirements * update requirements * update requirements * update requirements * update requirements * update requirements * update requirements * update requirements * update requirements * update ci * update ci * update ci * update ci * update ci * update ci * update ci * update ci * update ci * update ci * merge gradio4 * update ci * update ci * update ci * update ci * pass unit test * merge and pass unit test --------- Co-authored-by: Zhicheng Zhang <zhangzhicheng.zzc@alibaba-inc.com> * merge origin master * fix bugs * fix video-to-image * add qwen-max * fix apps/agentfabric & update release version * apps/agentfabric/user_core.py * fix pre-commit * fix bugs & add video play for code interpreter * comment out translate * Refactor log (#268) * update logger * add dashscope log * update user log * add comment --------- Co-authored-by: ly119399 <ly119399@alibaba-inc.com> * Refactor/knowledgeretrieval (#271) * pass retrieval knowledge * add unit test * update requirements * pass lint * refact name * refact name --------- Co-authored-by: Zhicheng Zhang <zhangzhicheng.zzc@alibaba-inc.com> * add logger (#272) * comment out translate * Refactor log (#268) * update logger * add dashscope log * update user log * add comment --------- Co-authored-by: ly119399 <ly119399@alibaba-inc.com> --------- Co-authored-by: skyline2006 <skyline2006@163.com> Co-authored-by: lylalala <liuyang070424@163.com> Co-authored-by: ly119399 <ly119399@alibaba-inc.com> Co-authored-by: Zhicheng Zhang <zhangzhicheng.zzc@alibaba-inc.com> * pass code unittest * fix publish for missing MODELSCOPE_API_TOKEN alert & rm translation tool * fix bugs * update log * fix bugs * fix instruction * fix tool log * fix bugs * fix pre-commit * Zhipu-glm4 (#276) support zhipu glm-4 model --------- Co-authored-by: skyline2006 <skyline2006@163.com> Co-authored-by: Zhicheng Zhang <zczhang.hvac@gmail.com> * add zhipuai to requirements.txt * make sure .md file is parsed by nltk * Update demo 20240118 (#275) Co-authored-by: Jintao Huang <huangjintao.hjt@alibaba-inc.com> * add zhipuai * update retrieval knowledge logic to not load in agent fabric * update name * fix bugs * fix pre-commit * fix ci * update max_token setting * fix doc list extend method * fix ci * fix ci * add docstr * rm useless agent: function call, react, react_chat * bugfix * bugfix * bugfix: change _detect_tool for role_play * remove uuid related log in ms * fix publish avatar * version 0.3.0rc0 * fix bugs * fix bug * fix audio->video * update demo * bug fixed by comment * lint pass * bug fixed * bug fixed * fix ci (not complete) * add hf tool unit test * Openapi (#287) * feat: openapi * fix pre-commit * fix bugs --------- Co-authored-by: skyline2006 <skyline2006@163.com> Co-authored-by: suluyan <suluyan.sly@alibaba-inc.com> * bug fixed * pass lint * fix openapi bug --------- Co-authored-by: wangyijunlyy <1157637345@qq.com> Co-authored-by: tujianhong.tjh <tujianhong.tjh@alibaba-inc.com> Co-authored-by: suluyan.sly <suluyan.sly@alibaba-inc.com> Co-authored-by: Jianhong Tu <37433392+tuhahaha@users.noreply.github.com> Co-authored-by: mushenL <125954878+mushenL@users.noreply.github.com> Co-authored-by: Zhicheng Zhang <zhangzhicheng.zzc@alibaba-inc.com> Co-authored-by: Zhicheng Zhang <zczhang.hvac@gmail.com> Co-authored-by: suluyana <110878454+suluyana@users.noreply.github.com> Co-authored-by: wyw <1006629314@qq.com> Co-authored-by: nil.wyw <nil.wyw@alibaba-inc.com> Co-authored-by: Col0ring <1561999073@qq.com> Co-authored-by: wenmeng.zwm <wenmeng.zwm@alibaba-inc.com> Co-authored-by: Col0ring <47329987+Col0ring@users.noreply.github.com> Co-authored-by: skyline2006 <skyline2006@163.com> Co-authored-by: ly119399 <ly119399@alibaba-inc.com> Co-authored-by: Jintao Huang <huangjintao.hjt@alibaba-inc.com> (cherry picked from commit 251b56f)
diff --git a/modelscope_agent/agent.py b/modelscope_agent/agent.py
@@ -115,7 +115,14 @@ def _register_tool(self, tool: Union[str, Dict]):
             raise NotImplementedError
         if tool not in self.function_list:
             self.function_list.append(tool)
-            self.function_map[tool_name] = TOOL_REGISTRY[tool_name](tool_cfg)
+            tool_class = TOOL_REGISTRY[tool_name]
+            try:
+                self.function_map[tool_name] = tool_class(tool_cfg)
+            except TypeError:
+                # When using OpenAPI, tool_class is already an instantiated object, not a corresponding class
+                self.function_map[tool_name] = tool_class
+            except Exception as e:
+                raise RuntimeError(e)
 
     def _detect_tool(self, message: Union[str, dict]):
         # use built-in default judgment functions
diff --git a/modelscope_agent/tools/base.py b/modelscope_agent/tools/base.py
@@ -76,10 +76,16 @@ def _build_function(self):
                 },
             }
             for para in self.parameters:
-                function['parameters']['properties'][para['name']] = {
-                    'type': para['type'],
-                    'description': para['description']
+                function_details = {
+                    'type':
+                    para['type'] if 'type' in para else para['schema']['type'],
+                    'description':
+                    para['description']
                 }
+                if 'enum' in para and para['enum'] not in ['', []]:
+                    function_details['enum'] = para['enum']
+                function['parameters']['properties'][
+                    para['name']] = function_details
                 if 'required' in para and para['required']:
                     function['parameters']['required'].append(para['name'])
         else:
diff --git a/modelscope_agent/tools/dashscope_tools/wordart_tool.py b/modelscope_agent/tools/dashscope_tools/wordart_tool.py
@@ -45,7 +45,6 @@ def call(self, params: str, **kwargs) -> str:
         if isinstance(params, str):
             return 'Parameter Error'
         remote_parsed_input = json.dumps(self._remote_parse_input(**params))
-        origin_result = None
         self.token = kwargs.get('token',
                                 os.environ.get('DASHSCOPE_API_KEY', ''))
         assert self.token != '', 'dashscope api token must be acquired with wordart'
diff --git a/modelscope_agent/tools/openapi_plugin.py b/modelscope_agent/tools/openapi_plugin.py
@@ -47,26 +47,17 @@ def __init__(self, cfg, name):
         self.description = self.cfg.get('description',
                                         'This is a api tool that ...')
         self.responses_param = self.cfg.get('responses_param', [])
-        try:
-            all_para = {
-                'name': self.name,
-                'description': self.description,
-                'parameters': self.parameters
-            }
-            self.tool_schema = ToolSchema(**all_para)
-        except ValidationError:
-            raise ValueError(f'Error when parsing parameters of {self.name}')
-        self._str = self.tool_schema.model_dump_json()
-        self._function = self.parse_pydantic_model_to_openai_function(all_para)
+        super().__init__(cfg)
 
     def call(self, params: str, **kwargs):
         if self.url == '':
             raise ValueError(
                 f"Could not use remote call for {self.name} since this tool doesn't have a remote endpoint"
             )
-        #
-        # remote_parsed_input = json.dumps(
-        #     self._remote_parse_input(*args, **kwargs))
+
+        json_params = json.loads(params)
+        remote_parsed_input = json.dumps(
+            self._remote_parse_input(**json_params))
         params = self._verify_args(params)
         if isinstance(params, str):
             return 'Parameter Error'
@@ -80,7 +71,10 @@ def call(self, params: str, **kwargs):
                     print(f'data: {kwargs}')
                     print(f'header: {self.header}')
                     response = requests.request(
-                        'POST', url=self.url, headers=self.header, data=params)
+                        'POST',
+                        url=self.url,
+                        headers=self.header,
+                        data=remote_parsed_input)
 
                     if response.status_code != requests.codes.ok:
                         response.raise_for_status()
@@ -96,6 +90,8 @@ def call(self, params: str, **kwargs):
                     raise ValueError(
                         f'Remote call failed with error code: {e.response.status_code},\
                         error message: {e.response.content.decode("utf-8")}')
+                except Exception as e:
+                    raise ValueError(e)
 
             raise ValueError(
                 'Remote call max retry times exceeded! Please try to use local call.'
@@ -106,8 +102,8 @@ def call(self, params: str, **kwargs):
             new_url = self.url
             matches = re.findall(r'\{(.*?)\}', self.url)
             for match in matches:
-                if match in kwargs:
-                    new_url = new_url.replace('{' + match + '}', kwargs[match])
+                if match in params:
+                    new_url = new_url.replace('{' + match + '}', params[match])
                 else:
                     print(
                         f'The parameter {match} was not generated by the model.'
@@ -209,7 +205,7 @@ def parse_nested_parameters(param_name, param_info, parameters_list, content):
                             inner_param_required,
                             'type':
                             inner_param_type,
-                            'value':
+                            'enum':
                             inner_param_info.get('enum', '')
                         })
         else:
@@ -219,7 +215,7 @@ def parse_nested_parameters(param_name, param_info, parameters_list, content):
                 'description': param_description,
                 'required': param_required,
                 'type': param_type,
-                'value': param_info.get('enum', '')
+                'enum': param_info.get('enum', '')
             })
     except Exception as e:
         raise ValueError(f'{e}:schema结构出错')
@@ -348,7 +344,6 @@ def openapi_schema_convert(schema, auth):
                         }
                     }
             elif method.upper() == 'GET':
-                parameters_list = []
                 parameters_list = details.get('parameters', [])
                 config_entry = {
                     'name': name,
