Skip to content

Upstream merge 25 03 10 #471

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 167 commits into from
Mar 10, 2025
Merged

Upstream merge 25 03 10 #471

merged 167 commits into from
Mar 10, 2025

Conversation

@gshtras
Copy link
Collaborator

@gshtras gshtras commented Mar 10, 2025 •
edited
Loading

Including early access to vllm-project#14316

hmellor and others added 30 commits March 3, 2025 17:48
@hmellor
Fix head_dim not existing in all model configs (Transformers backen…
91373a0 
…d) (vllm-project#14141)

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
@markmc
[V0][Metrics] Remove unimplemented vllm:tokens_total (vllm-project#…
c41d271 
…14134)

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
@markmc
[V0][Metrics] Deprecate some KV/prefix cache metrics (vllm-project#14136
2dfdfed 
)

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
@njhill
[V1] Simplify stats logging (vllm-project#14082)
872db2b 
Signed-off-by: Nick Hill <nhill@redhat.com>
@markmc
[WIP][[V1][Metrics] Implement max_num_generation_tokens, request_para…
ae122b1 
…ms_n, and request_params_max_tokens metrics (vllm-project#14055)

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
@mgoin
[Bugfix] Allow shared_experts skip quantization for DeepSeekV2/V3 (vl…
2b04c20 
…lm-project#14100)

Signed-off-by: mgoin <mgoin64@gmail.com>
@mgoin
[Kernel] Optimize moe intermediate_cache usage (vllm-project#13625)
19d98e0 
Signed-off-by: mgoin <mgoin64@gmail.com>
@Qubitium @mgoin
[Docs] Add GPTQModel (vllm-project#14056)
cd1d3c3 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
@vanbasten23 @mgoin
[v1] Add comments to the new ragged paged attention Pallas kernel (vl…
79e4937 
…lm-project#14155)

Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
@tjohnson31415 @DarkLight1337
[Model] Add support for GraniteMoeShared models (vllm-project#13313)
c060b71 
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
@divakar-amd
[core] moe fp8 block quant tuning support (vllm-project#14068)
bb5b640 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
@comaniac
[Misc] Remove lru_cache in NvmlCudaPlatform (vllm-project#14156)
989f4f4 
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
@ruisearch42
[core] Pass all driver env vars to ray workers unless excluded (vllm-…
bf13d40 
…project#14099)

Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
@zhanwenchen
Use math.prod instead of np.prod for trivial ops (vllm-project#14142)
66233af
@mgoin
Fix benchmark_moe.py tuning for CUDA devices (vllm-project#14164)
f78c0be
@youkaichao
[platform] add debug logging during inferring the device type (vllm-p…
ac65bc9 
…roject#14195)

Signed-off-by: youkaichao <youkaichao@gmail.com>
@youkaichao
[sleep mode] error out with expandable_segments (vllm-project#14189)
71c4b40 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@youkaichao
[doc] add "Failed to infer device type" to faq (vllm-project#14200)
3610fb4 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@mgoin
[Bugfix] Restrict MacOS CPU detection (vllm-project#14210)
6247bae 
Signed-off-by: mgoin <mgoin64@gmail.com>
@njhill
[V1][BugFix] Fix remaining sync engine client shutdown errors/hangs (v…
5db6b2c 
…llm-project#13869)

Signed-off-by: Nick Hill <nhill@redhat.com>
@markmc
[V0][Metrics] Deprecate some questionable request time metrics (vllm-…
c8525f0 
…project#14135)

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
@lk-chen
[V1][Molmo] Fix get_multimodal_embeddings() in molmo.py (vllm-project…
b3cf368 
…#14161)
@kushanam
add cutlass support for blackwell fp8 gemm (vllm-project#13798)
f89978a
@lsy323 @mgoin
[TPU][Profiler] Support start_profile/stop_profile in TPU worker (vll…
beebf47 
…m-project#13988)

Signed-off-by: Siyuan Liu <lsiyuan@google.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
@hmellor
Fix performance when --generation-config is not None (vllm-projec…
9badee5 
…t#14223)

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
@hmellor
[Frontend] Do prompt_logprobs clamping for chat as well as completi…
e5b2f16 
…ons (vllm-project#14225)

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
@mgoin
[Docs] Update Dockerfile dependency image (vllm-project#14215)
550c7ba 
Signed-off-by: mgoin <mgoin64@gmail.com>
@markmc @hmellor @comaniac
[v1][Metrics] Add design doc (vllm-project#12745)
c2bd219 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
@KuntaiDu
[Security] Serialize using safetensors instead of pickle in Mooncake …
288ca11 
…Pipe (vllm-project#14228)

Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
@tlrmchlsmth
Clean up unused padding_idx variables across many model definitions (v…
4f5b059 
…llm-project#13240)

Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
russellb and others added 19 commits March 8, 2025 17:19
@russellb
Update CODEOWNERS for structured output (vllm-project#14496)
0d5e73d 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
@DarkLight1337
[Misc] Upgrade to Python 3.9 typing for additional directories (vllm-…
9513290 
…project#14492)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@22quinn @njhill
[V1] Support bad_words in sampler (vllm-project#13376)
eb8b5eb 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
@robertgshaw2-redhat @ywang96
Revert "[V1][Core] Fix memory issue with logits & sampling" (vllm-pro…
5f0b53c 
…ject#14504)

Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
@LucasWilkinson @tlrmchlsmth
[Attention] Default to FlashMLA backend for MLA (vllm-project#14451)
b0d5419 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
@vanbasten23
[V1][TPU] Remove unnecessary padding for running on TPU. (vllm-projec…
10f7552 
…t#14467)
@YaoJiayi
[Feat] Support chunked prefill for LMCache connector (vllm-project#14505
6d7f037 
)

Signed-off-by: YaoJiayi <120040070@link.cuhk.edu.cn>
@yanyc428
[Bugfix] Fix tqdm progress bar when SamplingParams.n > 1 (vllm-projec…
73ae0b4 
…t#12428)

Signed-off-by: Yuchen Yan <740987012@qq.com>
@Isotr0py
[Bugfix] Revert QKVCrossParallelLinear usage in Mllama to keep BNB qu…
fb16eea 
…antization work (vllm-project#14498)

Signed-off-by: Isotr0py <2037008807@qq.com>
@yaochengji
[Hardware][TPU] Fix the recompiling issue in logits processor after w…
212007b 
…armup (vllm-project#14510)

Signed-off-by: Chengji Yao <chengjiyao@google.com>
@liuyanyi
[Misc] Ensure out-of-tree quantization method recognize by cli args (v…
a21076e 
…llm-project#14328)

Signed-off-by: liuyanyi <wolfsonliu@163.com>
@martinhoyer
[Bugfix] Wrong requirements path - rocm (vllm-project#14527)
dc74613 
Signed-off-by: Martin Hoyer <mhoyer@redhat.com>
@JenZhao @ywang96
[Feature] Consolidate performance benchmark datasets (vllm-project#14036
1253b15 
)

Signed-off-by: Jennifer Zhao <7443418+JenZhao@users.noreply.github.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Jennifer Zhao <7443418+JenZhao@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
@chaunceyjiang
[Misc] Add log information for handle_process_request. (vllm-project#…
460f553 
…14130)

Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
@hmellor
[Docs] Mention model_impl arg when explaining Transformers fallback (…
60a98b2 
…vllm-project#14552)

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
@chaunceyjiang
[Frontend] support image embeds (vllm-project#13955)
b0746fa 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
@SzymonOzog
[Kernel] Add more dtype support for GGUF kernels (vllm-project#14043)
89cdaa8 
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>
@DarkLight1337
[Doc] Update PaliGemma note to a warning (vllm-project#14565)
001a9c7 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@gshtras
Merge remote-tracking branch 'upstream/main'
9ef3d37
@gshtras gshtras merged commit 1095cff into main Mar 10, 2025
3 of 5 checks passed
@gshtras gshtras deleted the upstream_merge_25_03_10 branch March 10, 2025 21:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@shajrawi shajrawi shajrawi approved these changes

@Alexei-V-Ivanov-AMD Alexei-V-Ivanov-AMD Awaiting requested review from Alexei-V-Ivanov-AMD

@maleksan85 maleksan85 Awaiting requested review from maleksan85

@sunway513 sunway513 Awaiting requested review from sunway513

@divakar-amd divakar-amd Awaiting requested review from divakar-amd

@charlifu charlifu Awaiting requested review from charlifu

@mawong-amd mawong-amd Awaiting requested review from mawong-amd

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.