Python - AGUI - CopilotKit - Errors when using AzureAIClient instead of samples using AzureOpenAIChatClient

[nourshaker-msft]

Due to customer requirements, we need to be able to use AzureAIClient instead of AzureOpenAIChatClient:
Passing the following client to simple_agent.py

chat_client = AzureAIClient(
    project_endpoint=project_endpoint,
    credential=credential,
    model_deployment_name="gpt-4.1",
    agent_name="ag-ui-fastapi-server",
    streaming=False,
)

will result in the following error (as soon as you start chatting with it) on the backend:

  + Exception Group Traceback (most recent call last):
  |   File "/opt/conda/lib/python3.12/site-packages/starlette/_utils.py", line 79, in collapse_excgroups
  |     yield
  |   File "/opt/conda/lib/python3.12/site-packages/starlette/responses.py", line 270, in __call__
  |     async with anyio.create_task_group() as task_group:
  |   File "/home/codespace/.local/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 781, in __aexit__
  |     raise BaseExceptionGroup(
  | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "/opt/conda/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py", line 416, in run_asgi
    |     result = await app(  # type: ignore[func-returns-value]
    |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/conda/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
    |     return await self.app(scope, receive, send)
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/conda/lib/python3.12/site-packages/fastapi/applications.py", line 1135, in __call__
    |     await super().__call__(scope, receive, send)
    |   File "/opt/conda/lib/python3.12/site-packages/starlette/applications.py", line 107, in __call__
    |     await self.middleware_stack(scope, receive, send)
    |   File "/opt/conda/lib/python3.12/site-packages/starlette/middleware/errors.py", line 186, in __call__
    |     raise exc
    |   File "/opt/conda/lib/python3.12/site-packages/starlette/middleware/errors.py", line 164, in __call__
    |     await self.app(scope, receive, _send)
    |   File "/opt/conda/lib/python3.12/site-packages/starlette/middleware/cors.py", line 85, in __call__
    |     await self.app(scope, receive, send)
    |   File "/opt/conda/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 63, in __call__
    |     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
    |   File "/opt/conda/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    |     raise exc
    |   File "/opt/conda/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    |     await app(scope, receive, sender)
    |   File "/opt/conda/lib/python3.12/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    |     await self.app(scope, receive, send)
    |   File "/opt/conda/lib/python3.12/site-packages/starlette/routing.py", line 716, in __call__
    |     await self.middleware_stack(scope, receive, send)
    |   File "/opt/conda/lib/python3.12/site-packages/starlette/routing.py", line 736, in app
    |     await route.handle(scope, receive, send)
    |   File "/opt/conda/lib/python3.12/site-packages/starlette/routing.py", line 290, in handle
    |     await self.app(scope, receive, send)
    |   File "/opt/conda/lib/python3.12/site-packages/fastapi/routing.py", line 119, in app
    |     await wrap_app_handling_exceptions(app, request)(scope, receive, send)
    |   File "/opt/conda/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    |     raise exc
    |   File "/opt/conda/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    |     await app(scope, receive, sender)
    |   File "/opt/conda/lib/python3.12/site-packages/fastapi/routing.py", line 106, in app
    |     await response(scope, receive, send)
    |   File "/opt/conda/lib/python3.12/site-packages/starlette/responses.py", line 269, in __call__
    |     with collapse_excgroups():
    |   File "/opt/conda/lib/python3.12/contextlib.py", line 158, in __exit__
    |     self.gen.throw(value)
    |   File "/opt/conda/lib/python3.12/site-packages/starlette/_utils.py", line 85, in collapse_excgroups
    |     raise exc
    |   File "/opt/conda/lib/python3.12/site-packages/starlette/responses.py", line 273, in wrap
    |     await func()
    |   File "/opt/conda/lib/python3.12/site-packages/starlette/responses.py", line 253, in stream_response
    |     async for chunk in self.body_iterator:
    |   File "/opt/conda/lib/python3.12/site-packages/agent_framework_ag_ui/_endpoint.py", line 73, in event_generator
    |     async for event in wrapped_agent.run_agent(input_data):
    |   File "/opt/conda/lib/python3.12/site-packages/agent_framework_ag_ui/_agent.py", line 176, in run_agent
    |     async for event in orchestrator.run(context):
    |   File "/opt/conda/lib/python3.12/site-packages/agent_framework_ag_ui/_orchestrators.py", line 444, in run
    |     async for update in context.agent.run_stream(messages_to_run, **run_kwargs):
    |   File "/opt/conda/lib/python3.12/site-packages/agent_framework/observability.py", line 1408, in trace_run_streaming
    |     async for streaming_agent_response in run_streaming_func(self, messages=messages, thread=thread, **kwargs):
    |   File "/opt/conda/lib/python3.12/site-packages/agent_framework/_agents.py", line 1033, in run_stream
    |     async for update in self.chat_client.get_streaming_response(
    |   File "/opt/conda/lib/python3.12/site-packages/agent_framework/_tools.py", line 2007, in streaming_function_invocation_wrapper
    |     async for update in func(self, messages=prepped_messages, **filtered_kwargs):
    |   File "/opt/conda/lib/python3.12/site-packages/agent_framework/observability.py", line 1160, in trace_get_streaming_response
    |     async for update in func(self, messages=messages, **kwargs):
    |   File "/opt/conda/lib/python3.12/site-packages/agent_framework/_middleware.py", line 1419, in _stream_generator
    |     async for update in original_get_streaming_response(self, messages, **kwargs):
    |   File "/opt/conda/lib/python3.12/site-packages/agent_framework/_clients.py", line 676, in get_streaming_response
    |     async for update in self._inner_get_streaming_response(
    |   File "/opt/conda/lib/python3.12/site-packages/agent_framework/openai/_responses_client.py", line 127, in _inner_get_streaming_response
    |     run_options = await self._prepare_options(messages, chat_options, **kwargs)
    |                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/conda/lib/python3.12/site-packages/agent_framework_azure_ai/_client.py", line 384, in _prepare_options
    |     agent_reference = await self._get_agent_reference_or_create(run_options, instructions)
    |                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/conda/lib/python3.12/site-packages/agent_framework_azure_ai/_client.py", line 357, in _get_agent_reference_or_create
    |     created_agent = await self.project_client.agents.create_version(
    |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/conda/lib/python3.12/site-packages/azure/core/tracing/decorator_async.py", line 119, in wrapper_use_tracer
    |     return await func(*args, **kwargs)
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/conda/lib/python3.12/site-packages/azure/ai/projects/aio/operations/_operations.py", line 1300, in create_version
    |     pipeline_response: PipelineResponse = await self._client._pipeline.run(  # pylint: disable=protected-access
    |                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/conda/lib/python3.12/site-packages/azure/core/pipeline/_base_async.py", line 229, in run
    |     return await first_node.send(pipeline_request)
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/conda/lib/python3.12/site-packages/azure/core/pipeline/_base_async.py", line 77, in send
    |     response = await self.next.send(request)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/conda/lib/python3.12/site-packages/azure/core/pipeline/_base_async.py", line 77, in send
    |     response = await self.next.send(request)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/conda/lib/python3.12/site-packages/azure/core/pipeline/_base_async.py", line 77, in send
    |     response = await self.next.send(request)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   [Previous line repeated 1 more time]
    |   File "/opt/conda/lib/python3.12/site-packages/azure/core/pipeline/_base_async.py", line 81, in send
    |     await _await_result(self._policy.on_response, request, response)
    |   File "/opt/conda/lib/python3.12/site-packages/azure/core/pipeline/_tools_async.py", line 54, in await_result
    |     result = func(*args, **kwargs)
    |              ^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/conda/lib/python3.12/site-packages/azure/core/pipeline/policies/_universal.py", line 728, in on_response
    |     response.context[self.CONTEXT_NAME] = self.deserialize_from_http_generics(
    |                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/conda/lib/python3.12/site-packages/azure/core/pipeline/policies/_universal.py", line 690, in deserialize_from_http_generics
    |     return cls.deserialize_from_text(response.text(encoding), mime_type, response=response)
    |                                      ^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/conda/lib/python3.12/site-packages/azure/core/rest/_http_response_impl.py", line 318, in text
    |     self._text = decode_to_text(self.encoding, self.content)
    |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/conda/lib/python3.12/site-packages/azure/core/rest/_helpers.py", line 194, in decode_to_text
    |     return content.decode(encoding)
    |            ^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/conda/lib/python3.12/encodings/utf_8_sig.py", line 23, in decode
    |     (output, consumed) = codecs.utf_8_decode(input, errors, True)
    |                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    | UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa4 in position 1: invalid start byte
    | decoding with 'utf-8-sig' codec failed
    +------------------------------------

packages:
agent-framework-ag-ui
agent-framework>=1.0.0b251120

# Azure OpenAI (required)
agent-framework-azure-ai
azure-identity>=1.19.0

[nourshaker-msft]

I've got a workaround going, doing some patching for some internal functions, committed in:

patch_agent_framework.py
patch_azure_core.py
patch_openai_responses.py

[moonbox3]

I just saw this issue from someone else yesterday while they were using the AzureAIClient. Once they redid their environment and reinstalled agent framework, it worked. Can you try removing any uv cache / other cache and reinstall and see if it works without these patches? Otherwise, these decompression issues are at the service-side of things.