server chat/completion api fails - coroutine object not callable in llama_proxy

[PurnaChandraPanda]

• The server is ran in docker as: python -m llama_cpp.server --model path.gguf.
• The llama_cpp server initializes fine.
• However, when v1/chat/completions post request is fired, it fails with coroutine object not callable in llama_proxy.
• With llama-cpp-python==0.3.4, following error is noticed in llama_cpp server side.
• Note that the same code works fine 0.3.2 package version.

Exception: 'coroutine' object is not callable
Traceback (most recent call last):
  File "/azureml-envs/minimal/lib/python3.11/site-packages/llama_cpp/server/errors.py", line 173, in custom_route_handler
    response = await original_route_handler(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/azureml-envs/minimal/lib/python3.11/site-packages/fastapi/routing.py", line 301, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/azureml-envs/minimal/lib/python3.11/site-packages/fastapi/routing.py", line 212, in run_endpoint_function
    return await dependant.call(**values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/azureml-envs/minimal/lib/python3.11/site-packages/llama_cpp/server/app.py", line 491, in create_chat_completion
    llama = llama_proxy(body.model)
            ^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'coroutine' object is not callable
INFO:     ::1:42174 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error

[hikerell]

I also encountered this error, which was introduced in version 0.3.3.
Now I rolled back to 0.3.2 and it works.

pip install llama-cpp-python==0.3.2

[sergey21000]

Need to add missing await, fixes here
#1858

[sergey21000]

Fixed in version 0.3.5

pip install 'llama-cpp-python>=0.3.5'