[Model] Add option to run Step3VisionEncoder in DP

[zzh142857]

Essential Elements of an Effective PR Description Checklist

Purpose

Add option to run step3 vision encoder in data parallel manner while the main model is in TP. Can be enabled by flag: vllm_config.parallel_config.enable_multimodal_encoder_data_parallel

Test Plan

Run step3 model and print GPU trace

Test Result

From local benchmark, vision encoder latency can be reduced by ~45%.
Trace before:

Trace after:

E2E benchmark:
Tested e2e with 320 max batch size, QPS has ~10% improvement (from 3.02 to 3.3), TTFT reduced from 3311ms to 3176ms.

(Optional) Documentation Update

[gemini-code-assist]

Code Review

This pull request introduces an option to run the step3 vision encoder with data parallelism, which is a great performance enhancement. The implementation is clean and follows existing patterns in the vLLM codebase. The logic for conditionally using ReplicatedLinear for data parallelism and the sharding/gathering of data in run_dp_sharded_vision_model seems correct. The changes are well-contained and properly gated by the new configuration flag. Overall, this is a solid contribution.

[DarkLight1337]

This LGTM but please fix pre-commit

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[DarkLight1337]

Thanks. For reference, can you also run E2E benchmarks (benchmark_serving.py) to compare the results?

[simon-mo]

@DarkLight1337 is this PR good to merge?

[DarkLight1337]

Yes, you can merge this. The benchmark results can be updated in the PR description later

[DarkLight1337]

Actually let me just merge this myself