Skip to content
This repository was archived by the owner on Oct 11, 2024. It is now read-only.

Commit 0844ba8

Browse files
DarkLight1337Robert Shaw
authored andcommitted
[Doc] Add note about context length in Phi-3-Vision example (vllm-project#5887)
1 parent 7823612 commit 0844ba8

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

examples/phi3v_example.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,14 +9,16 @@
99

1010
def run_phi3v():
1111
model_path = "microsoft/Phi-3-vision-128k-instruct"
12+
13+
# Note: The model has 128k context length by default which may cause OOM
14+
# If that's the case, override `max_model_len` with a smaller value via args
1215
llm = LLM(
1316
model=model_path,
1417
trust_remote_code=True,
1518
image_input_type="pixel_values",
1619
image_token_id=32044,
1720
image_input_shape="1,3,1008,1344",
1821
image_feature_size=1921,
19-
disable_image_processor=False,
2022
)
2123

2224
image = Image.open("images/cherry_blossom.jpg")

0 commit comments

Comments
 (0)