Skip to content

Upstream sync from vllm-project/vllm #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 478 commits into from
May 19, 2025
Merged

Upstream sync from vllm-project/vllm #1

merged 478 commits into from
May 19, 2025

Conversation

@cyril23
Copy link
Owner

@cyril23 cyril23 commented May 19, 2025

merge all changes from https://github.com/vllm-project/vllm while keeping the current changes of https://github.com/kaln27/vllm

hmellor and others added 30 commits May 8, 2025 04:06
@hmellor
Don't call the venv vllm (vllm-project#17810)
66ab3b1 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
@njhill
[BugFix] Fix --disable-log-stats in V1 server mode (vllm-project#17600
3d13ca0 
)

Signed-off-by: Nick Hill <nhill@redhat.com>
@chanh
[Core] Support full cuda graph in v1 (vllm-project#16072)
7ea2adb 
Signed-off-by: Chanh Nguyen <cnguyen@linkedin.com>
Co-authored-by: Chanh Nguyen <cnguyen@linkedin.com>
@vmarkovtsev
Improve exception reporting in MP engine (vllm-project#17800)
b2da14a 
Signed-off-by: Vadim Markovtsev <vadim@poolside.ai>
@Xarbirus
[Installation] OpenTelemetry version update (vllm-project#17771)
c747d84 
Signed-off-by: Mikhail Podvitskii <podvitskiymichael@gmail.com>
@hmellor
Only log non-default CLI args for online serving (vllm-project#17803)
998eea4 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
@russellb @njhill
[V1] Add VLLM_ALLOW_INSECURE_SERIALIZATION env var (vllm-project#17490)
6930a41 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
@amd-hhashemi @charlifu
[Kernel][Hardware][AMD] Bf16 mfma opt for ROCm skinny GEMMs (vllm-pro…
5a499e7 
…ject#17071)

Signed-off-by: Hashem Hashemi <hashem.hashemi@amd.com>
Signed-off-by: charlifu <charlifu@amd.com>
Co-authored-by: charlifu <charlifu@amd.com>
@Akashcodes732 @mgoin
[Hardware][Power] Enable compressed tensor W8A8 INT8 quantization for…
e515668 
… POWER (vllm-project#17153)

Signed-off-by: Akash Kaothalkar <akash.kaothalkar@ibm.com>
Co-authored-by: Akash Kaothalkar <akash.kaothalkar@ibm.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
@adobrzyn
[Hardware][Intel-Gaudi] Support Automatic Prefix Caching on HPU (vllm…
843b222 
…-project#17648)

Signed-off-by: Agata Dobrzyniewicz <adobrzyniewicz@habana.ai>
@DarkLight1337
[Frontend] Chat template fallbacks for multimodal models (vllm-projec…
96722aa 
…t#17805)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@Ximingwang-09
[Qwen3]add qwen3-235b-bf16 fused moe config on A100 (vllm-project#17715)
597051e
@qionghuang6
[Bugfix] Fix bad words for Mistral models (vllm-project#17753)
39956ef 
Signed-off-by: Qiong Zhou Huang <qiong@phonic.co>
@xsank
[Misc] support model prefix & add deepseek vl2 tiny fused moe config (v…
0a9bbaa 
…llm-project#17763)

Signed-off-by: 唯勤 <xsank.mz@alibaba-inc.com>
Co-authored-by: 唯勤 <xsank.mz@alibaba-inc.com>
@RIckYuan999 @aarnphm
[Bugfix] Fix tool call template validation for Mistral models (vllm-p…
ca04b97 
…roject#17644)

Signed-off-by: Rick Yuan <yuan821120@gmail.com>
Signed-off-by: RIck Yuan <yuan821120@gmail.com>
Co-authored-by: Aaron Pham <Aaronpham0103@gmail.com>
@bythew3i
[TPU] Fix the test_sampler (vllm-project#17820)
a463555
@fxmarty-amd
[Bugfix] Fix quark fp8 format loading on AMD GPUs (vllm-project#12612)
bb239a7 
Signed-off-by: Felix Marty <felmarty@amd.com>
Signed-off-by: kewang2 <kewang2@amd.com>
Co-authored-by: kewang2 <kewang2@amd.com>
@DarkLight1337
[Doc] Fix a typo in the file name (vllm-project#17836)
a1e19b6 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@houseroad
[Easy] Eliminate c10::optional usage in vllm/csrc (vllm-project#17819)
f50dcb7
@reidliu41
[Misc] add chatbox integration (vllm-project#17828)
53d0cb7 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
@hmellor
Fix transient dependency error in docs build (vllm-project#17848)
e4ca6e3 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
@DarkLight1337
[Bugfix] use_fast failing to be propagated to Qwen2-VL image proces…
015815f 
…sor (vllm-project#17838)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@jeejeelee
[Misc] Delete LoRA-related redundancy code (vllm-project#17841)
a944f8e 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
@russellb
[CI] Fix test_collective_rpc (vllm-project#17858)
ec54d73 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
@russellb
[V1] Improve VLLM_ALLOW_INSECURE_SERIALIZATION logging (vllm-project#…
226a427 
…17860)

Signed-off-by: Russell Bryant <rbryant@redhat.com>
@yarongmu-google
[Test] Attempt all TPU V1 tests, even if some of them fail. (vllm-pro…
a83a0f9 
…ject#17334)

Signed-off-by: Yarong Mu <ymu@google.com>
@mgoin
[CI] Prune down lm-eval small tests (vllm-project#17012)
8342e3a 
Signed-off-by: mgoin <mgoin64@gmail.com>
@mgoin
Fix noisy warning for uncalibrated q_scale/p_scale (vllm-project#17414)
4f605a6 
Signed-off-by: mgoin <mgoin64@gmail.com>
@wenscarl
Add cutlass support for blackwell fp8 blockwise gemm (vllm-project#14383
376786f 
)

Signed-off-by: Shu Wang <shuw@nvidia.com>
@vllmellm @qli88 @hongxiayang
[FEAT][ROCm]: Support AITER MLA on V1 Engine (vllm-project#17523)
3c9396a 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: qli88 <qiang.li2@amd.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
princepride and others added 24 commits May 17, 2025 04:20
@princepride @tracelogfb
[BugFix] Correct max_model_len derivation from config.json for Mistra…
4ee4826 
…l format (vllm-project#17937)

Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: tracelogfb <48808670+tracelogfb@users.noreply.github.com>
Co-authored-by: Stephen Chen <tracelog@meta.com>
@sdavidbd
[P/D][V1] Support dynamic loading of external KV connector implementa…
3e0d435 
…tions (vllm-project#18142)

Signed-off-by: David Ben-David <davidb@pliops.com>
Co-authored-by: David Ben-David <davidb@pliops.com>
@lsy323 @CAROLZXYZXY
@jianzs @hfan
[Hardware][TPU] Optionally import for TPU backend (vllm-project#18269)
48ac2be 
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com>
Co-authored-by: Carol Zheng <cazheng@google.com>
Co-authored-by: Jade Zheng <zheng.shoujian@outlook.com>
Co-authored-by: Hongmin Fan <fanhongmin@google.com>
@mgoin
Update Dockerfile to build for Blackwell (vllm-project#18095)
dcfe952
@npanpaliya
Fixed build on ppc64le due to openssl conflicts (vllm-project#18262)
f880d42 
Signed-off-by: Nishidha Panpaliya <nishidha.panpaliya@partner.ibm.com>
@lengrongfu
[Model] use AutoWeightsLoader for solar (vllm-project#18113)
9214e60
@andyxning
[MISC] fix typo (vllm-project#18305)
66e63e8 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
@cascade812
Support sequence parallelism combined with pipeline parallelism (vllm…
9ab2c02 
…-project#18243)

Signed-off-by: cascade812 <cascade812@outlook.com>
@reidliu41
[doc] update reasoning doc (vllm-project#18306)
1a8f68b 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
@22quinn
[Model] Use sigmoid for single-label classification (vllm-project#18313)
908733a 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
@lifuhuang
Fix copy-paste error in phi4mm image processing (vllm-project#18315)
4fb349f 
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
@reidliu41
[Misc] add litellm integration (vllm-project#18320)
b6a6e7a 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
@WangErXiao
[Doc] Add doc to explain the usage of Qwen3 thinking (vllm-project#18291
d1211f8 
)

Signed-off-by: WangErXiao <863579016@qq.com>
@wwl2755
[Spec Decode][V0] Fix spec decode correctness test in V0 eagle/medusa (…
9da1095 
…vllm-project#18175)

Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
@Nan2018 @Bryce1010
@qthequartermasterman @DarkLight1337
Feature/vllm/input embedding completion api (vllm-project#17590)
221cfc2 
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
Signed-off-by: Nan2018 <nan@protopia.ai>
Co-authored-by: 临景 <linjing.yx@alibaba-inc.com>
Co-authored-by: Bryce1010 <bryceyx@gmail.com>
Co-authored-by: Andrew Sansom <andrew@protopia.ai>
Co-authored-by: Andrew Sansom <qthequartermasterman@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
@reidliu41
[Misc] extract parser.parse_args() (vllm-project#18323)
27d0952 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
@simon-mo
[Build] Supports CUDA 12.6 and 11.8 after Blackwell Update (vllm-proj…
47fda6d 
…ect#18316)

Signed-off-by: simon-mo <simon.mo@hey.com>
@googs1025
fix: Add type specifications for CLI arguments in tensorizer options (v…
275c5da 
…llm-project#18314)
@shaoyuyoung @cascade812
[BugFix] [Vul] Add missing usedforsecurity=False in MD5 hashing to …
d637b96 
…enable FIPS (vllm-project#18319)

Signed-off-by: cascade812 <cascade812@outlook.com>
Signed-off-by: shaoyuyoung <shaoyuyoung@gmail.com>
Co-authored-by: cascade <cascade812@outlook.com>
@Potabk
[Doc] Fix prompt embedding examples (vllm-project#18350)
c5bb0eb 
Signed-off-by: wangli <wangli858794774@gmail.com>
@DarkLight1337
[Doc] Move input-related docs to Features (vllm-project#18353)
43b5f61 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@njhill @NickLucche
[BugFix] Fix handling of num_computed_tokens with connector (vllm-pro…
1b15df2 
…ject#18232)

Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>
@jeejeelee
[Quantization] Pool model support bitsandbytes (vllm-project#18087)
6781af5 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
@eladsegal
[Doc] Fix typo (vllm-project#18355)
84ab4fe
@github-actions
Copy link

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

@cyril23 cyril23 merged this pull request into main May 19, 2025
0 of 2 checks passed
@cyril23 cyril23 deleted the upstream-sync branch May 19, 2025 16:34
@cyril23 cyril23 mentioned this pull request May 19, 2025
Closed
1 task
@cyril23 cyril23 mentioned this pull request Jun 27, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.