AsyncEngineDeadError with QUEN

[SafeyahShemali]

Hello,

I am trying to test Qwen/Qwen-7B-Chat with openai.api_server and this error shows up:
'''
vllm.engine.async_llm_engine.AsyncEngineDeadError: Task finished unexpectedly. This should never happen! Please open an issue on Github. See stack trace above for the actual cause
'''

The Traceback:

Traceback (most recent call last):
File "/opt/conda/envs/myenv/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 426, in run_asgi
result = await app( # type: ignore[func-returns-value]
File "/opt/conda/envs/myenv/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in call
return await self.app(scope, receive, send)
File "/opt/conda/envs/myenv/lib/python3.10/site-packages/fastapi/applications.py", line 1106, in call
await super().call(scope, receive, send)
File "/opt/conda/envs/myenv/lib/python3.10/site-packages/starlette/applications.py", line 122, in call
await self.middleware_stack(scope, receive, send)
File "/opt/conda/envs/myenv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in call
raise exc
File "/opt/conda/envs/myenv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in call
await self.app(scope, receive, _send)
File "/opt/conda/envs/myenv/lib/python3.10/site-packages/starlette/middleware/cors.py", line 83, in call
await self.app(scope, receive, send)
File "/opt/conda/envs/myenv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 79, in call
raise exc
File "/opt/conda/envs/myenv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 68, in call
await self.app(scope, receive, sender)
File "/opt/conda/envs/myenv/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in call
raise e
File "/opt/conda/envs/myenv/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in call
await self.app(scope, receive, send)
File "/opt/conda/envs/myenv/lib/python3.10/site-packages/starlette/routing.py", line 718, in call
await route.handle(scope, receive, send)
File "/opt/conda/envs/myenv/lib/python3.10/site-packages/starlette/routing.py", line 276, in handle
await self.app(scope, receive, send)
File "/opt/conda/envs/myenv/lib/python3.10/site-packages/starlette/routing.py", line 66, in app
response = await func(request)
File "/opt/conda/envs/myenv/lib/python3.10/site-packages/fastapi/routing.py", line 274, in app
raw_response = await run_endpoint_function(
File "/opt/conda/envs/myenv/lib/python3.10/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
return await dependant.call(**values)
File "/opt/conda/envs/myenv/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 300, in create_chat_completion
async for res in result_generator:
File "/opt/conda/envs/myenv/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 436, in generate
raise e
File "/opt/conda/envs/myenv/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 430, in generate
async for request_output in stream:
File "/opt/conda/envs/myenv/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 70, in anext
raise result
File "uvloop/cbhandles.pyx", line 63, in uvloop.loop.Handle._run
File "/opt/conda/envs/myenv/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 37, in _raise_exception_on_finish
raise exc
File "/opt/conda/envs/myenv/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 32, in _raise_exception_on_finish
raise AsyncEngineDeadError(
.

[zhang-xh95]

do you find the reason? i met the same problem

[Wen-Chen-DS]

met the same problem, any solutions?

[github-actions]

This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!