Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DO NOT MERGE] Ibm dev rebased #255

Open
wants to merge 43 commits into
base: main
Choose a base branch
from
Open

[DO NOT MERGE] Ibm dev rebased #255

wants to merge 43 commits into from

Conversation

prashantgupta24
Copy link

Description

How Has This Been Tested?

Merge criteria:

  • The commits are squashed in a cohesive manner and have meaningful messages.
  • Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
  • The developer has manually tested the changes and verified that the changes work

simon-mo and others added 30 commits December 3, 2024 10:38
@simon-mo @prashantgupta24
[Docs] Add Nebius as sponsors (vllm-project#10371)
42d3a81 
Signed-off-by: simon-mo <simon.mo@hey.com>
@russellb @prashantgupta24
[Frontend] Add --version flag to CLI (vllm-project#10369)
a18c4fb 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
@russellb @prashantgupta24
[Doc] Move PR template content to docs (vllm-project#10159)
067dd2b 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
@mikegre-google @prashantgupta24
[Docs] Misc updates to TPU installation instructions (vllm-project#10165
82be00f 
)
@DarkLight1337 @prashantgupta24
[Frontend] Automatic detection of chat content format from AST (vllm-…
2ca5912 
…project#9919)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@youkaichao @prashantgupta24
[doc] add doc for the plugin system (vllm-project#10372)
ea480a1 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@youkaichao @prashantgupta24
[misc][plugin] improve log messages (vllm-project#10386)
2e453bc 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@rasmith @prashantgupta24
[BugFix] [Kernel] Fix GPU SEGV occuring in fused_moe kernel (vllm-pro…
51813a4 
…ject#10385)

Signed-off-by: Randall Smith <Randall.Smith@amd.com>
@kakao-steve-ai @prashantgupta24
[Misc] Update benchmark to support image_url file or http (vllm-proje…
6c45f56 
…ct#10287)

Signed-off-by: rbbang <anjaehyun87@gmail.com>
@skylee-01 @prashantgupta24
[Misc] Medusa supports custom bias (vllm-project#10361)
d93bde9
@imkero @prashantgupta24
[Bugfix] Fix M-RoPE position calculation when chunked prefill is enab…
068451f 
…led (vllm-project#10388)

Signed-off-by: imkero <kerorek@outlook.com>
@WoosukKwon @prashantgupta24
[V1] Add code owners for V1 (vllm-project#10397)
d49dacb 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
@youkaichao @prashantgupta24
[2/N][torch.compile] make compilation cfg part of vllm cfg (vllm-proj…
11d2bbc 
…ect#10383)

Signed-off-by: youkaichao <youkaichao@gmail.com>
@ywang96 @prashantgupta24
[V1] Refactor model executable interface for all text-only language m…
2ea854d 
…odels (vllm-project#10374)

Signed-off-by: Roger Wang <ywang@roblox.com>
@xuechendi @prashantgupta24
[CI/Build] Fix IDC hpu [Device not found] issue (vllm-project#10384)
2652ea1 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
@Isotr0py @prashantgupta24
[Bugfix][CPU] Fix CPU embedding runner with tensor parallel (vllm-pro…
47325d5 
…ject#10394)

Signed-off-by: Isotr0py <2037008807@qq.com>
@youkaichao @prashantgupta24
[platforms] refactor cpu code (vllm-project#10402)
e288d72 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@jikunshang @prashantgupta24
[Hardware] [HPU]add mark_step for hpu (vllm-project#10239)
f6dc8be 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
@imkero @prashantgupta24
[Bugfix] Fix mrope_position_delta in non-last prefill chunk (vllm-pro…
dcccc62 
…ject#10403)

Signed-off-by: imkero <kerorek@outlook.com>
@wchen61 @prashantgupta24
[Misc] Enhance offline_inference to support user-configurable paramet… (
d97d269 
vllm-project#10392)

Signed-off-by: wchen61 <wchen61@foxmail.com>
@Isotr0py @prashantgupta24
[Misc] Add uninitialized params tracking for AutoWeightsLoader (vll…
d3fe99b 
…m-project#10327)

Signed-off-by: Isotr0py <2037008807@qq.com>
@HollowMan6 @prashantgupta24
[Bugfix] Ignore ray reinit error when current platform is ROCm or XPU (…
1f165b6 
…vllm-project#10375)

Signed-off-by: Hollow Man <hollowman@opensuse.org>
@youkaichao @prashantgupta24
[4/N][torch.compile] clean up set_torch_compile_backend (vllm-project…
56b9c49 
…#10401)

Signed-off-by: youkaichao <youkaichao@gmail.com>
@lk-chen @prashantgupta24
[VLM] Report multi_modal_placeholders in output (vllm-project#10407)
663fb57 
Signed-off-by: Linkun Chen <lkchen+anyscale@github.com>
@Maybewuss @prashantgupta24
[Model] Remove redundant softmax when using PoolingType.STEP (vllm-pr…
3c64fbb 
…oject#10415)
@B-201 @prashantgupta24
[Model][LoRA]LoRA support added for glm-4v (vllm-project#10418)
20ea0c3 
Signed-off-by: B-201 <Joy25810@foxmail.com>
@Isotr0py @prashantgupta24
[Model] Remove transformers attention porting in VITs (vllm-project#1…
b1d6a6a 
…0414)

Signed-off-by: Isotr0py <2037008807@qq.com>
@B-201 @prashantgupta24
[Doc] Update doc for LoRA support in GLM-4V (vllm-project#10425)
5d6df78 
Signed-off-by: B-201 <Joy25810@foxmail.com>
@youkaichao @prashantgupta24
[5/N][torch.compile] torch.jit.script --> torch.compile (vllm-project…
b4c641f 
…#10406)

Signed-off-by: youkaichao <youkaichao@gmail.com>
@ismael-dm @prashantgupta24
[Doc] Add documentation for Structured Outputs (vllm-project#9943)
c973945 
Signed-off-by: ismael-dm <ismaeldm99@gmail.com>
andrew and others added 11 commits December 3, 2024 10:39
@andrew @prashantgupta24
Fix open_collective value in FUNDING.yml (vllm-project#10426)
e19e15a 
Signed-off-by: Andrew Nesbitt <andrewnez@gmail.com>
@mgoin @prashantgupta24
[Model][Bugfix] Support TP for PixtralHF ViT (vllm-project#10405)
a00d8c1 
Signed-off-by: mgoin <michael@neuralmagic.com>
@yma11 @prashantgupta24
[Hardware][XPU] AWQ/GPTQ support for xpu backend (vllm-project#10107)
472b9cf 
Signed-off-by: yan ma <yan.ma@intel.com>
@angusYuhao @prashantgupta24
[Kernel] Explicitly specify other value in tl.load calls (vllm-projec…
9147b76 
…t#9014)

Signed-off-by: Angus Wang <wangjadehao@gmail.com>
@LucasWilkinson @prashantgupta24
[Kernel] Initial Machete W4A8 support + Refactors (vllm-project#9855)
ce83e18 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
@fialhocoelho @prashantgupta24
Squash 9522
bf5bf24 
Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
@fialhocoelho @prashantgupta24
Squash 6357
8b27172 
Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
@fialhocoelho @prashantgupta24
Squash 10235
e8fcdee 
Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
@fialhocoelho @prashantgupta24
Squash 10400
1e7f586 
Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
@fialhocoelho @prashantgupta24
Squash 10430
09f5b4a 
Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
@fialhocoelho @prashantgupta24
pin mistral and install adapter from branch 🚀
33c1553 
Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
@openshift-ci openshift-ci bot requested review from heyselbi and Xaenalt December 3, 2024 18:49
@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Dec 3, 2024

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: prashantgupta24
Once this PR has been reviewed and has the lgtm label, please assign rpancham for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

dtrifiro
dtrifiro reviewed Dec 4, 2024
View reviewed changes
Dockerfile.ubi
FROM vllm-openai as vllm-grpc-adapter

USER root

RUN --mount=type=cache,target=/root/.cache/pip \
--mount=type=cache,target=/root/.cache/uv \
--mount=type=bind,from=build,src=/workspace/dist,target=/workspace/dist \
HOME=/root uv pip install $(echo dist/*.whl)'[tensorizer]' vllm-tgis-adapter==0.5.3
uv pip install $(echo /workspace/dist/*.whl)'[tensorizer]' --verbose && \
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's an issue in this Dockerfile at line 188 (not part of this PR) which prevents the dockerfile from building. You can either rebase on main or cherry pick 039e209 to fix the issue.

@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Dec 4, 2024

@prashantgupta24: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/cuda-pr-image-mirror 87ebca7 link true /test cuda-pr-image-mirror
ci/prow/smoke-test 87ebca7 link true /test smoke-test
ci/prow/rocm-pr-image-mirror 87ebca7 link true /test rocm-pr-image-mirror
ci/prow/images 87ebca7 link true /test images

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-merge-robot
Copy link

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dtrifiro dtrifiro dtrifiro left review comments

@njhill njhill Awaiting requested review from njhill njhill is a code owner

@heyselbi heyselbi Awaiting requested review from heyselbi

@Xaenalt Xaenalt Awaiting requested review from Xaenalt

Assignees
No one assigned
Labels
needs-rebase
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.