Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bugfix] Fix 307 Redirect for /metrics #4523

Merged
merged 2 commits into from
May 1, 2024
Merged

[Bugfix] Fix 307 Redirect for /metrics #4523

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a33b924
fixes 307 redirect
robertgshaw2-redhat May 1, 2024
f5aa089
Update api_server.py
robertgshaw2-redhat May 1, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion vllm/engine/metrics.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ def __init__(self, labelnames: List[str], max_model_len: int):
buckets=[1, 2, 5, 10, 20],
)
self.counter_request_success = Counter(
name="vllm:request_success",
name="vllm:request_success_total",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this to match the name scheme of the other counters?

Also, can you check if it's used in the example dashboard and, if yes, update the name there too?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated this to:

  • match scheme of other counters
  • the dashboard is using request_success_total already (so the dashboard was broken)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah ok, nice.

documentation="Count of successfully processed requests.",
labelnames=labelnames + [Metrics.labelname_finish_reason])

Expand Down
8 changes: 6 additions & 2 deletions vllm/entrypoints/openai/api_server.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
import importlib
import inspect
import os
import re
from contextlib import asynccontextmanager
from http import HTTPStatus

Expand All @@ -12,6 +13,7 @@
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import JSONResponse, Response, StreamingResponse
from prometheus_client import make_asgi_app
from starlette.routing import Mount

import vllm
from vllm.engine.arg_utils import AsyncEngineArgs
Expand Down Expand Up @@ -55,8 +57,10 @@ def parse_args():


# Add prometheus asgi middleware to route /metrics requests
metrics_app = make_asgi_app()
app.mount("/metrics", metrics_app)
route = Mount("/metrics", make_asgi_app())
# Avoid 307 Redirect
route.path_regex = re.compile('^/metrics(?P<path>.*)$')
app.routes.append(route)


@app.exception_handler(RequestValidationError)
Expand Down
Loading