Skip to content
This repository has been archived by the owner on Oct 11, 2024. It is now read-only.

Commit

Permalink
[Bugfix][Minor] Make ignore_eos effective (vllm-project#4468)
Browse files Browse the repository at this point in the history
  • Loading branch information
bigPYJ1151 authored and robertgshaw2-redhat committed May 6, 2024
1 parent 9ad9b65 commit 195439e
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion vllm/sampling_params.py
Original file line number Diff line number Diff line change
Expand Up @@ -275,7 +275,8 @@ def update_from_generation_config(
self, generation_config: Dict[str, Any]) -> None:
"""Update if there are non-default values from generation_config"""
# Update eos_token_id for generation
if eos_ids := generation_config.get("eos_token_id"):
if (not self.ignore_eos) and (eos_ids :=
generation_config.get("eos_token_id")):
# it can be either int or list of int
if isinstance(eos_ids, int):
eos_ids = [eos_ids]
Expand Down

0 comments on commit 195439e

Please sign in to comment.