Skip to content

Conversation

@DarkLight1337
Copy link
Member

@DarkLight1337 DarkLight1337 commented Aug 1, 2025

Essential Elements of an Effective PR Description Checklist

  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Purpose

This PR completes the basic features of #21995 with documentation and tests for it.

  • Support GPU processor outputs and cast them back to CPU for serialization
  • Update profiling code to run a full batch through the processor when GPU processing is enabled
  • Add optimization docs for multi-modal processing

Test Plan

Added entrypoints tests to test the functionality.

Test Result

(Optional) Documentation Update

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request enables Hugging Face processor execution on the GPU, a valuable performance enhancement. The changes include comprehensive documentation updates, new tests to validate the functionality, and significant refactoring to support the new feature.

The core logic for GPU processing, including moving data to and from the device and handling configuration, appears correct. The refactoring in the worker and test files improves code clarity and maintainability.

I've identified a couple of high-severity issues related to inefficient dummy data generation during profiling runs in both the GPU and TPU model runners. These inefficiencies could impact the accuracy of memory profiling and should be addressed. My review includes specific suggestions to resolve these issues.

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@mergify mergify bot removed the needs-rebase label Sep 22, 2025
@DarkLight1337 DarkLight1337 marked this pull request as ready for review September 22, 2025 09:42
@DarkLight1337 DarkLight1337 marked this pull request as draft September 22, 2025 10:03
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@mergify
Copy link

mergify bot commented Oct 8, 2025

Documentation preview: https://vllm--22070.org.readthedocs.build/en/22070/

@mergify
Copy link

mergify bot commented Oct 8, 2025

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @DarkLight1337.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify bot added the needs-rebase label Oct 8, 2025
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@mergify mergify bot removed the needs-rebase label Oct 10, 2025
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation frontend multi-modality Related to multi-modality (#4194) performance Performance-related issues ready ONLY add when PR is ready to merge/full CI is needed v1

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants