[Bug]: Error occurred in v1/rerank interface after upgrading from version 0.7.3 to 0.8.1 #15371
Description
Your current environment
The output of `python collect_env.py`
Your output of `python collect_env.py` here
🐛 Describe the bug
Start using docker compose, which is the content of the docker compose file
x-vllm-common:
&common
image: vllm/vllm-openai:latest
restart: unless-stopped
environment:
VLLM_USE_MODELSCOPE: True
HF_ENDPOINT: https://hf-mirror.com
TZ: "Asia/Shanghai"
volumes:
- /root/.cache/modelscope/hub:/models # Please modify this to the actual model directory.
networks:
- vllm
services:
nginx:
image: nginx:latest
restart: unless-stopped
ports:
- "6090:80"
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf # Mount the nginx configuration file.
networks:
- vllm
depends_on:
- reranker
reranker:
<<: *common
deploy:
resources:
reservations:
devices:
- driver: nvidia
capabilities: [ gpu ]
count: all
command: [ "--model","/models/AI-ModelScope/bge-reranker-v2-m3", "--host", "0.0.0.0", "--port", "5000", "--tensor-parallel-size", "2", "--task", "score", "--served-model-name", "bge-reranker-v2-m3", "--trust-remote-code"]
networks:
vllm:
Report an error when accessing interface /v1/rerank
INFO: 172.25.0.3:36030 - "POST /v1/rerank HTTP/1.0" 500 Internal Server Error
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/opt/venv/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi
result = await app( # type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
return await self.app(scope, receive, send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.12/site-packages/fastapi/applications.py", line 1054, in __call__
await super().__call__(scope, receive, send)
File "/opt/venv/lib/python3.12/site-packages/starlette/applications.py", line 112, in __call__
await self.middleware_stack(scope, receive, send)
File "/opt/venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 187, in __call__
raise exc
File "/opt/venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 165, in __call__
await self.app(scope, receive, _send)
File "/opt/venv/lib/python3.12/site-packages/starlette/middleware/cors.py", line 85, in __call__
await self.app(scope, receive, send)
File "/opt/venv/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/opt/venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/opt/venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/opt/venv/lib/python3.12/site-packages/starlette/routing.py", line 714, in __call__
await self.middleware_stack(scope, receive, send)
File "/opt/venv/lib/python3.12/site-packages/starlette/routing.py", line 734, in app
await route.handle(scope, receive, send)
File "/opt/venv/lib/python3.12/site-packages/starlette/routing.py", line 288, in handle
await self.app(scope, receive, send)
File "/opt/venv/lib/python3.12/site-packages/starlette/routing.py", line 76, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/opt/venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/opt/venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/opt/venv/lib/python3.12/site-packages/starlette/routing.py", line 73, in app
response = await f(request)
^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.12/site-packages/fastapi/routing.py", line 301, in app
raw_response = await run_endpoint_function(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.12/site-packages/fastapi/routing.py", line 212, in run_endpoint_function
return await dependant.call(**values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.12/site-packages/vllm/entrypoints/utils.py", line 58, in wrapper
return handler_task.result()
^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 638, in do_rerank_v1
return await do_rerank(request, raw_request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.12/site-packages/vllm/entrypoints/utils.py", line 49, in wrapper
handler_task = asyncio.create_task(handler_func(*args, **kwargs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: do_rerank() missing 1 required keyword-only argument: 'raw_request'
What to do？
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.