Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upstream merge 25 01 13 #358

Merged
merged 116 commits into from
Jan 14, 2025
Merged

Upstream merge 25 01 13 #358

merged 116 commits into from
Jan 14, 2025

Conversation

gshtras
Copy link
Collaborator

@gshtras gshtras commented Jan 13, 2025 •
edited by github-actions bot
Loading

Upstream merge

  • reverting memory profiling changes due to it showing excessive memory usage

jeejeelee and others added 30 commits January 6, 2025 15:22
@jeejeelee
[Bugfix][V1] Fix molmo text-only inputs (vllm-project#11676)
32c9eff 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
@heheda12345
[Kernel] Move attn_type to Attention.__init__() (vllm-project#11690)
e20c92b 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
@ywang96 @DarkLight1337
[V1] Extend beyond image modality and support mixed-modality inferenc…
91b361a 
…e with Llava-OneVision (vllm-project#11685)

Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
@DarkLight1337
[Bugfix] Fix LLaVA-NeXT feature size precision error (for real) (vllm…
08fb75c 
…-project#11772)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@DarkLight1337
[Model] Future-proof Qwen2-Audio multi-modal processor (vllm-project#…
d0169e1 
…11776)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@ys950902
[XPU] Make pp group initilized for pipeline-parallelism (vllm-project…
d93d2d7 
…#11648)

Signed-off-by: yisheng <yi.sheng@intel.com>
@DarkLight1337
[Doc][3/N] Reorganize Serving section (vllm-project#11766)
8ceffbf 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@jeejeelee @Abatom
[Kernel][LoRA]Punica prefill kernels fusion (vllm-project#11234)
b278557 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Abatom <abzhonghua@gmail.com>
Co-authored-by: Zhonghua Deng <abatom@163.com>
@ywang96
[Bugfix] Update attention interface in Whisper (vllm-project#11784)
0f3f3c8 
Signed-off-by: Roger Wang <ywang@roblox.com>
@liangfu
[CI] Fix neuron CI and run offline tests (vllm-project#11779)
898cdf0 
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
@XiaobingSuper
fix init error for MessageQueue when n_local_reader is zero (vllm-pro…
e512f76 
…ject#11768)
@russellb
[Doc] Create a vulnerability management team (vllm-project#9925)
ce1917f 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
@zhouyuan
[CI][CPU] adding build number to docker image name (vllm-project#11788)
1e4ce29 
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>
@ywang96
[V1][Doc] Update V1 support for LLaVa-NeXT-Video (vllm-project#11798)
8082ad7 
Signed-off-by: Roger Wang <ywang@roblox.com>
@DarkLight1337
[Bugfix] Comprehensively test and fix LLaVA-NeXT feature size calcula…
8f37be3 
…tion (vllm-project#11800)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@youkaichao @DarkLight1337
[doc] add doc to explain how to use uv (vllm-project#11773)
869e829 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
@ywang96
[V1] Support audio language models on V1 (vllm-project#11733)
2de197b 
Signed-off-by: Roger Wang <ywang@roblox.com>
@youkaichao
[doc] update how pip can install nightly wheels (vllm-project#11806)
d9fa1c0 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@DarkLight1337
[Doc] Add note to gte-Qwen2 models (vllm-project#11808)
c0efe92 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@youkaichao
[optimization] remove python function call for custom op (vllm-projec…
869579a 
…t#11750)

Signed-off-by: youkaichao <youkaichao@gmail.com>
@jiangjiadi
[Bugfix] update the prefix for qwen2 (vllm-project#11795)
c994223 
Co-authored-by: jiadi.jjd <jiadi.jjd@antgroup.com>
@sroy745
[Doc]Add documentation for using EAGLE in vLLM (vllm-project#11417)
973f5dc 
Signed-off-by: Sourashis Roy <sroy@roblox.com>
@DamonFool
[Bugfix] Significant performance drop on CPUs with --num-scheduler-st…
a4e2b26 
…eps > 1 (vllm-project#11794)
@hmellor
[Doc] Group examples into categories (vllm-project#11782)
5950f55 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
@DarkLight1337
[Bugfix] Fix image input for Pixtral-HF (vllm-project#11741)
91445c7 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@divakar-amd
[Misc] sort torch profiler table by kernel timing (vllm-project#11813)
4d29e91
@WangErXiao
Remove the duplicate imports of MultiModalKwargs and PlaceholderRange… (
dc71af0 
vllm-project#11824)
@npanpaliya
Fixed docker build for ppc64le (vllm-project#11518)
b640b19 
Signed-off-by: Nishidha Panpaliya <nishidha.panpaliya@partner.ibm.com>
@ilya-lavrenov
[OpenVINO] Fixed Docker.openvino build (vllm-project#11732)
f4923cb 
Signed-off-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
@jeejeelee
[Bugfix] Add checks for LoRA and CPU offload (vllm-project#11810)
f645eb6 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
ywang96 and others added 27 commits January 12, 2025 06:36
@ywang96
[V1] Avoid sending text prompt to core engine (vllm-project#11963)
b25cfab 
Signed-off-by: Roger Wang <ywang@roblox.com>
@rafvasq
[CI/Build] Add markdown linter (vllm-project#11857)
43f3d9e 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
@Isotr0py @DarkLight1337
[Model] Initialize support for Deepseek-VL2 models (vllm-project#11578)
f967e51 
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
@Akshat-Tripathi @mosalov
@jeejeelee @Isotr0py
[Hardware][CPU] Multi-LoRA implementation for the CPU backend (vllm-p…
8bddb73 
…roject#11100)

Signed-off-by: Akshat Tripathi <akshat@krai.ai>
Signed-off-by: Oleg Mosalov <oleg@krai.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Oleg Mosalov <oleg@krai.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
@avshalomman
[Hardware][TPU] workaround fix for MoE on TPU (vllm-project#11764)
263a870
@robertgshaw2-redhat
[V1][Core][1/n] Logging and Metrics (vllm-project#11962)
9597a09 
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
@Isotr0py @DarkLight1337
[Model] Support GGUF models newly added in transformers 4.46.0 (vll…
d14e98d 
…m-project#9685)

Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
@robertgshaw2-redhat
[V1] [2/n] Logging and Metrics - OutputProcessor Abstraction (vllm-…
619ae26 
…project#11973)

Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
@yyccli
[MISC] fix typo in kv transfer send recv test (vllm-project#11983)
f7b3ba8
@liaoyanqing666
[Bug] Fix usage of .transpose() and .view() consecutively. (vllm-…
9dd02d8 
…project#11979)
@llsj14
[CI][Spec Decode] fix: broken test for EAGLE model (vllm-project#11972)
80ea3af 
Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
@Concurrensee
[Misc] Fix Deepseek V2 fp8 kv-scale remapping (vllm-project#11947)
cf6bbcb 
Signed-off-by: Yida Wu <yidawu@alumni.cmu.edu>
@noemotiovon
[Misc]Minor Changes about Worker (vllm-project#11555)
c3f05b0 
Signed-off-by: Chenguang Li <757486878@qq.com>
@youkaichao
[platform] add ray_device_key (vllm-project#11948)
89ce62a 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@alex-jw-brooks
Fix Max Token ID for Qwen-VL-Chat (vllm-project#11980)
5340a30 
Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>
@heheda12345
[Kernel] unified_attention for Attention.forward (vllm-project#11967)
0f8cafe 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
@ywang96 @DarkLight1337
[Doc][V1] Update model implementation guide for V1 support (vllm-proj…
cd82499 
…ect#11998)

Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
@hmellor
[Doc] Organise installation documentation into categories and tabs (v…
e8c23ff 
…llm-project#11935)

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
@youkaichao
[platform] add device_control env var (vllm-project#12009)
458e63a 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@shen-shanshan
[Platform] Move get_punica_wrapper() function to Platform (vllm-proje…
a7d5968 
…ct#11516)

Signed-off-by: Shanshan Shen <467638484@qq.com>
@e1ijah1
bugfix: Fix signature mismatch in benchmark's get_tokenizer function (
c6db213 
vllm-project#11982)

Signed-off-by: elijah <f1renze.142857@gmail.com>
@gshtras
Merge remote-tracking branch 'upstream/main'
ce53f46
@gshtras
Using list
5a51290
@gshtras
Revert "[misc] improve memory profiling (vllm-project#11809)"
079750e 
This reverts commit 889e662.
@gshtras
Trying to make scales work with compileable attention
043c93d
@gshtras
Docs lint
16f8680
@gshtras
Merge remote-tracking branch 'origin/main' into upstream_merge_25_01_13
eb4abfd
@gshtras gshtras merged commit 5976f48 into main Jan 14, 2025
21 of 26 checks passed
@gshtras gshtras deleted the upstream_merge_25_01_13 branch January 14, 2025 20:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shajrawi shajrawi shajrawi approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.