Skip to content

[Model][VLM] Support Keye-VL-8B-Preview #20126

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Jul 2, 2025
+1,801 −2
Merged

[Model][VLM] Support Keye-VL-8B-Preview #20126

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
3674489
Support KeyeForConditionalGeneration
Kwai-Keye Jun 26, 2025
f3d445b
Merge branch 'main' of https://github.com/vllm-project/vllm
Kwai-Keye Jun 26, 2025
4a5abe4
Merge branch 'main' of https://github.com/vllm-project/vllm into Keye…
Kwai-Keye Jun 26, 2025
2f3acf0
refactor(keye): import KeyeProcessor & reuse merge_multimodal_embeddi…
Kwai-Keye Jun 30, 2025
56a7cd4
refactor(keye): remove nouse class/func & rename SiglipVisionEmbeddin…
Kwai-Keye Jul 1, 2025
fb6ff90
refactor(keye): keep registry in alphabetical order & add Keye prefix…
Kwai-Keye Jul 1, 2025
872a76a
refactor(keye): Add attn_backend check and raise exception for non-Fl…
Kwai-Keye Jul 1, 2025
f2264b4
docs(keye): add keye in support models doc & offline example
Kwai-Keye Jul 1, 2025
f81f75c
docs(keye): add keye offline multi_image example & delete system prompt
Kwai-Keye Jul 1, 2025
434da49
fix(keye-offline-example): add trust_remote_code=True & set max_model…
Kwai-Keye Jul 1, 2025
475d004
feat(keye): support xformers backend
Kwai-Keye Jul 1, 2025
31bbe8c
Merge branch 'main' into Keye-VL-8B-Preview
Kwai-Keye Jul 1, 2025
bb91c5d
fix(keye): add trust_remote_code in test registry
Kwai-Keye Jul 2, 2025
c89658e
Merge branch 'Keye-VL-8B-Preview' of https://github.com/Kwai-Keye/vll…
Kwai-Keye Jul 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions tests/models/registry.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -409,6 +409,7 @@ def check_available_online(
trust_remote_code=True), # noqa: E501
"MllamaForConditionalGeneration": _HfExamplesInfo("meta-llama/Llama-3.2-11B-Vision-Instruct"), # noqa: E501
"WhisperForConditionalGeneration": _HfExamplesInfo("openai/whisper-large-v3"), # noqa: E501
"KeyeForConditionalGeneration": _HfExamplesInfo("Kwai-Keye/Keye-VL-8B-Preview"), # noqa: E501
}

_SPECULATIVE_DECODING_EXAMPLE_MODELS = {
Expand Down
4 changes: 2 additions & 2 deletions vllm/entrypoints/chat_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -534,7 +534,7 @@ def _placeholder_str(self, modality: ModalityStr,
return "<image>"
if model_type in ("mllama", "llama4"):
return "<|image|>"
if model_type in ("qwen2_vl", "qwen2_5_vl"):
if model_type in ("qwen2_vl", "qwen2_5_vl", "keye", "Keye"):
return "<|vision_start|><|image_pad|><|vision_end|>"
if model_type == "qwen2_5_omni":
return "<|vision_start|><|IMAGE|><|vision_end|>"
Expand Down Expand Up @@ -562,7 +562,7 @@ def _placeholder_str(self, modality: ModalityStr,
elif modality == "video":
if model_type == "internvl_chat":
return "<video>"
if model_type in ("qwen2_vl", "qwen2_5_vl"):
if model_type in ("qwen2_vl", "qwen2_5_vl", "keye", "Keye"):
return "<|vision_start|><|video_pad|><|vision_end|>"
if model_type == "qwen2_5_omni":
return "<|vision_start|><|VIDEO|><|vision_end|>"
Expand Down
Loading