Skip to content

Commit 78f892c

Browse files
wdhongtwyewentao256
authored andcommitted
[Misc] Reduce initialization time of auto_tune (#23682)
Signed-off-by: Weida Hong <wdhongtw@google.com> Signed-off-by: yewentao256 <zhyanwentao@126.com>
1 parent 26da2c6 commit 78f892c

File tree

1 file changed

+6
-1
lines changed

1 file changed

+6
-1
lines changed

benchmarks/auto_tune/auto_tune.sh

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -103,10 +103,15 @@ start_server() {
103103
VLLM_USE_V1=1 VLLM_SERVER_DEV_MODE=1 \
104104
vllm serve "${common_args_array[@]}" > "$vllm_log" 2>&1 &
105105
fi
106+
local server_pid=$!
106107

107108
# wait for 10 minutes...
108109
server_started=0
109110
for i in {1..60}; do
111+
# This line checks whether the server is still alive or not,
112+
# since that we should always have permission to send signal to the server process.
113+
kill -0 $server_pid 2> /dev/null || break
114+
110115
RESPONSE=$(curl -s -X GET "http://0.0.0.0:8004/health" -w "%{http_code}" -o /dev/stdout)
111116
STATUS_CODE=$(echo "$RESPONSE" | tail -n 1)
112117
if [[ "$STATUS_CODE" -eq 200 ]]; then
@@ -118,7 +123,7 @@ start_server() {
118123
done
119124

120125
if (( ! server_started )); then
121-
echo "server did not start within 10 minutes. Please check server log at $vllm_log".
126+
echo "server did not start within 10 minutes or crashed. Please check server log at $vllm_log".
122127
return 1
123128
else
124129
return 0

0 commit comments

Comments
 (0)