Skip to content

Commit fdd3868

Browse files
committed
Cleanly handle EOFError during interactive CLI commands
When running `vllm chat` or `vllm complete`, handle EOFError (Ctrl+d) and exit cleanly instead of leaving a traceback behind. Signed-off-by: Russell Bryant <rbryant@redhat.com>
1 parent dca82ef commit fdd3868

File tree

1 file changed

+8
-2
lines changed

1 file changed

+8
-2
lines changed

vllm/cmd/openai.py

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,10 @@ def chat(system_prompt: Optional[str], model_name: str,
4949

5050
print("Please enter a message for the chat model:")
5151
while True:
52-
input_message = input("> ")
52+
try:
53+
input_message = input("> ")
54+
except EOFError:
55+
return
5356
conversation.append({"role": "user", "content": input_message})
5457

5558
chat_completion = client.chat.completions.create(model=model_name,
@@ -103,7 +106,10 @@ def cmd(args: argparse.Namespace) -> None:
103106

104107
print("Please enter a message for the chat model:")
105108
while True:
106-
input_message = input("> ")
109+
try:
110+
input_message = input("> ")
111+
except EOFError:
112+
return
107113
conversation.append({"role": "user", "content": input_message})
108114

109115
chat_completion = client.chat.completions.create(

0 commit comments

Comments
 (0)