Skip to content

Commit

Permalink
[Frontend][Misc] Enforce Pixel Values as Input Type for VLMs in API S…
Browse files Browse the repository at this point in the history
  • Loading branch information
ywang96 authored and jimpang committed Jul 24, 2024
1 parent 29a366b commit 960ffd8
Showing 1 changed file with 10 additions and 0 deletions.
10 changes: 10 additions & 0 deletions vllm/entrypoints/openai/api_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -183,6 +183,16 @@ async def authentication(request: Request, call_next):
served_model_names = [args.model]

engine_args = AsyncEngineArgs.from_cli_args(args)

# Enforce pixel values as image input type for vision language models
# when serving with API server
if engine_args.image_input_type is not None and \
engine_args.image_input_type.upper() != "PIXEL_VALUES":
raise ValueError(
f"Invalid image_input_type: {engine_args.image_input_type}. "
"Only --image-input-type 'pixel_values' is supported for serving "
"vision language models with the vLLM API server.")

engine = AsyncLLMEngine.from_engine_args(
engine_args, usage_context=UsageContext.OPENAI_API_SERVER)

Expand Down

0 comments on commit 960ffd8

Please sign in to comment.