[Test Case Failure]: ShuttleAIToolModel_test.py::test_agent_exec

[cobycloud]

Test Case

tests/llms/ShuttleAIToolModel_test.py::test_agent_exec

Root Cause

tests/llms/ShuttleAIToolModel_test.py::test_agent_exec 
-------------------------------- live log call ---------------------------------
{'status': 400, 'error': {'message': 'We could not parse the JSON body of your request.', 'type': 'invalid_request_error', 'code': None, 'param': None, 'hint': "This likely means you aren't using your HTTP library correctly. The ShuttleAI API expects a JSON payload, but what was sent was not valid JSON. If you have trouble figuring out how to fix this, please contact us at https://discord.gg/shuttleai"}}
'choices'
FAILED

tests/llms/ShuttleAIToolModel_test.py:45: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/swarmauri/standard/agents/concrete/ToolAgent.py:40: in exec
    self.conversation = self.llm.predict(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = ShuttleAIToolModel(name='shuttle-2-turbo', id='bab67668-b420-4795-bca5-4f6147390cc0', members=[], owner=None, host=Non... 'mistral-7b-instruct-v0.2', 'gemini-1.5-pro-latest', 'gemini-1.0-pro-latest'], api_key='***')
conversation = Conversation(name=None, id='73899cd9-1161-455a-b178-6c4ad4daf2a3', members=[], owner=None, host=None, resource='Conversation', version='0.1.0', type='Conversation')
toolkit = Toolkit(name=None, id='71ee7380-88c0-4888-aa45-1f695f66da69', members=[], owner=None, host=None, resource='Toolkit', v..., resource='Parameter', version='0.1.0', type='integer', description='The right operand', required=True, enum=None)])})
tool_choice = 'auto', temperature = 0.7, max_tokens = 1024, top_p = 1.0
internet = True, raw = False, image = None, citations = True, tone = 'precise'

    def predict(self,
        conversation,
        toolkit=None,
        tool_choice=None,
        temperature=0.7,
        max_tokens=1024,
        top_p=1.0,
        internet=True,
        raw=False,
        image=None,
        citations=True,
        tone='precise'):
    
        formatted_messages = self._format_messages(conversation.history)
    
        if toolkit and not tool_choice:
            tool_choice = "auto"
    
        url = "https://api.shuttleai.app/v1/chat/completions"
        headers = {
            "Authorization": f"***",
            "Content-Type": "application/json",
        }
    
        formatted_messages = self._format_messages(conversation.history)
    
        payload = {
            "model": self.name,
            "messages": formatted_messages,
            "max_tokens": max_tokens,
            "temperature": temperature,
            "top_p": top_p,
            "internet": internet,
            "raw": raw,
            "image": image,
            "tool_choice": tool_choice,
            "tools": self._schema_convert_tools(toolkit.tools),
        }
    
        if self.name in ['gpt-4-bing', 'gpt-4-turbo-bing']:
            payload['tone'] = tone
            payload['citations'] = citations
    
        agent_response = requests.request("POST", url, json=payload, headers=headers)
        logging.info(agent_response.json())
    
        try:
            messages = [formatted_messages[-1], agent_response.json()['choices'][0]['message']['content']]
        except Exception as error:
            logging.warn(error)
>       tool_calls = agent_response.json()['choices'][0]['message'].get('tool_calls', None)
E       KeyError: 'choices'

/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/swarmauri/standard/llms/concrete/ShuttleAIToolModel.py:90: KeyError
------------------------------ Captured log call -------------------------------
INFO     root:ShuttleAIToolModel.py:84 {'status': 400, 'error': {'message': 'We could not parse the JSON body of your request.', 'type': 'invalid_request_error', 'code': None, 'param': None, 'hint': "This likely means you aren't using your HTTP library correctly. The ShuttleAI API expects a JSON payload, but what was sent was not valid JSON. If you have trouble figuring out how to fix this, please contact us at https://discord.gg/shuttleai"}}
WARNING  root:ShuttleAIToolModel.py:89 'choices'

Potential Solutions

TBD

Additional Context (optional)

No response

Type of Test Case

Unit

[RafaelJohn9]

tests/llms/ShuttleAIToolModel_test.py::test_ubc_type PASSED              [ 58%]
tests/llms/ShuttleAIToolModel_test.py::test_serialization PASSED         [ 58%]
tests/llms/ShuttleAIToolModel_test.py::test_default_name PASSED          [ 58%]
tests/llms/ShuttleAIToolModel_test.py::test_agent_exec 
-------------------------------- live log call ---------------------------------
{'id': 'chatcmpl-cdfef69f08784405aedcb2d636a14aba', 'object': 'chat.completion', 'created': 1724159258, 'model': 'shuttle-2.5', 'choices': [{'index': 0, 'message': {'role': 'assistant', 'content': "I apologize, but it seems there was a mistake in the search query. I was asked to perform an addition of 512 and 671, not to search for any specific information. The search results provided are not relevant to the original request. Please let me know if you need assistance with the math problem or if there's anything else I can help you with."}, 'finish_reason': 'stop'}], 'usage': {'prompt_tokens': 39, 'completion_tokens': 77, 'total_charged': 7.34e-05}}
messages: [{'content': 'Add 512+671', 'role': 'user'}, "I apologize, but it seems there was a mistake in the search query. I was asked to perform an addition of 512 and 671, not to search for any specific information. The search results provided are not relevant to the original request. Please let me know if you need assistance with the math problem or if there's anything else I can help you with."]
agent_response: {'id': 'chatcmpl-cdfef69f08784405aedcb2d636a14aba', 'object': 'chat.completion', 'created': 1724159258, 'model': 'shuttle-2.5', 'choices': [{'index': 0, 'message': {'role': 'assistant', 'content': "I apologize, but it seems there was a mistake in the search query. I was asked to perform an addition of 512 and 671, not to search for any specific information. The search results provided are not relevant to the original request. Please let me know if you need assistance with the math problem or if there's anything else I can help you with."}, 'finish_reason': 'stop'}], 'usage': {'prompt_tokens': 39, 'completion_tokens': 77, 'total_charged': 7.34e-05}}
conversation: name=None id='05f1fff5-4366-45a7-93fb-14a277966218' members=[] owner=None host=None resource='Conversation' version='0.1.0' type='Conversation'
I apologize, but it seems there was a mistake in the search query. I was asked to perform an addition of 512 and 671, not to search for any specific information. The search results provided are not relevant to the original request. Please let me know if you need assistance with the math problem or if there's anything else I can help you with.
PASSED   ```