Skip to content

Commit 9d353fa

Browse files
committed
[Bugfix] Disable the statslogger if the api_server_count is greater than 1
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
1 parent 65e0389 commit 9d353fa

File tree

1 file changed

+5
-1
lines changed

1 file changed

+5
-1
lines changed

vllm/v1/engine/async_llm.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -104,7 +104,11 @@ def __init__(
104104
logger.info(
105105
"AsyncLLM created with log_stats=False and non-empty custom "
106106
"logger list; enabling logging without default stat loggers")
107-
107+
if client_count > 1:
108+
logger.warning(
109+
"AsyncLLM created with api_server_count more than 1; "
110+
"disabling stats logging to avoid incomplete stats.")
111+
log_stats = False
108112
if self.model_config.skip_tokenizer_init:
109113
self.tokenizer = None
110114
else:

0 commit comments

Comments
 (0)