Skip to content

Commit df51e19

Browse files
committed
[V0][Metrics] Deprecated duplicate queue time metric
vllm:time_in_queue_requests appears to be an exact duplicate of vllm:request_queue_time_seconds. Both record first_scheduled_time-arrival_time: ``` if seq_group.is_finished(): time_queue_requests.append( seq_group.metrics.first_scheduled_time - seq_group.metrics.arrival_time) ``` ``` def maybe_set_first_scheduled_time(self, time: float) -> None: if self.metrics.first_scheduled_time is None: self.metrics.first_scheduled_time = time self.metrics.time_in_queue = time - self.metrics.arrival_time ``` vllm:time_in_queue_requests was added by vllm-project#9659 and vllm:request_queue_time_seconds was later added by vllm-project#4464. However, neither existed when each PR was first created. The latter seems like the right one to keep since it is implemented in V1, used in the Grafana dashboard, and has test coverage. Signed-off-by: Mark McLoughlin <markmc@redhat.com>
1 parent e584b85 commit df51e19

File tree

1 file changed

+5
-2
lines changed

1 file changed

+5
-2
lines changed

vllm/engine/metrics.py

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -181,10 +181,13 @@ def __init__(self, labelnames: List[str], vllm_config: VllmConfig):
181181
"Histogram of time spent in DECODE phase for request.",
182182
labelnames=labelnames,
183183
buckets=request_latency_buckets)
184+
# Deprecated in 0.8 - duplicates vllm:request_queue_time_seconds:
185+
# TODO: in 0.9, only enable if show_hidden_metrics=True
184186
self.histogram_time_in_queue_request = self._histogram_cls(
185187
name="vllm:time_in_queue_requests",
186-
documentation=
187-
"Histogram of time the request spent in the queue in seconds.",
188+
documentation=(
189+
"Histogram of time the request spent in the queue in seconds. "
190+
"DEPRECATED: use vllm:request_queue_time_seconds instead."),
188191
labelnames=labelnames,
189192
buckets=request_latency_buckets)
190193
self.histogram_model_forward_time_request = self._histogram_cls(

0 commit comments

Comments
 (0)