[Bugfix][Model] Support (zero-padded) LoRA on Qwen3 output embedding

[klshuster]

Purpose

This PR achieves two goals:

1. Support LoRA on the embed/unembed layers for Qwen3 dense/MoE models
2. Support not expanding the vocabulary of the (un-)embed layer when applying LoRA to it

Test Plan

I've added a comprehensive test suite in tests/lora/test_qwen3_unembed.py, as well as tests in tests/lora/test_layers.py, that verify the changes work.

Test Result

$ python -m pytest tests/lora/test_qwen3_unembed.py tests/lora/test_layers.py::test_lm_head_logits_processor_zero_vocab_padding -v
========================================== test session starts ===========================================
platform linux -- Python 3.12.9, pytest-8.4.2, pluggy-1.6.0 -- /tmp/kurt/uv-venv/bin/python
cachedir: .pytest_cache
rootdir: /mnt/nfs/kurt/vllm
configfile: pyproject.toml
plugins: asyncio-1.2.0, anyio-4.11.0
asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 18 items

tests/lora/test_qwen3_unembed.py::test_qwen3_unembed_lora[8-0] PASSED                              [  5%]
tests/lora/test_qwen3_unembed.py::test_qwen3_unembed_lora[16-0] PASSED                             [ 11%]
tests/lora/test_qwen3_unembed.py::test_qwen3_unembed_lora_zero_vocab_padding PASSED                [ 16%]
tests/lora/test_qwen3_unembed.py::test_qwen3_unembed_lora_untied_weights[256] PASSED               [ 22%]
tests/lora/test_qwen3_unembed.py::test_qwen3_unembed_lora_untied_weights[512] PASSED               [ 27%]
tests/lora/test_qwen3_unembed.py::test_qwen3_moe_unembed_lora PASSED                               [ 33%]
tests/lora/test_layers.py::test_lm_head_logits_processor_zero_vocab_padding[True-cuda:0-1] PASSED  [ 38%]
tests/lora/test_layers.py::test_lm_head_logits_processor_zero_vocab_padding[True-cuda:0-2] PASSED  [ 44%]
tests/lora/test_layers.py::test_lm_head_logits_processor_zero_vocab_padding[True-cuda:0-4] PASSED  [ 50%]
tests/lora/test_layers.py::test_lm_head_logits_processor_zero_vocab_padding[True-cuda:1-1] PASSED  [ 55%]
tests/lora/test_layers.py::test_lm_head_logits_processor_zero_vocab_padding[True-cuda:1-2] PASSED  [ 61%]
tests/lora/test_layers.py::test_lm_head_logits_processor_zero_vocab_padding[True-cuda:1-4] PASSED  [ 66%]
tests/lora/test_layers.py::test_lm_head_logits_processor_zero_vocab_padding[False-cuda:0-1] SKIPPED [ 72%]
tests/lora/test_layers.py::test_lm_head_logits_processor_zero_vocab_padding[False-cuda:0-2] SKIPPED [ 77%]
tests/lora/test_layers.py::test_lm_head_logits_processor_zero_vocab_padding[False-cuda:0-4] SKIPPED [ 83%]
tests/lora/test_layers.py::test_lm_head_logits_processor_zero_vocab_padding[False-cuda:1-1] SKIPPED [ 88%]
tests/lora/test_layers.py::test_lm_head_logits_processor_zero_vocab_padding[False-cuda:1-2] SKIPPED [ 94%]
tests/lora/test_layers.py::test_lm_head_logits_processor_zero_vocab_padding[False-cuda:1-4] SKIPPED [100%]

========================== 12 passed, 6 skipped, 8 warnings in 73.01s (0:01:13) ==========================
sys:1: DeprecationWarning: builtin type swigvarlink has no __module__ attribute

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors.

You ask your reviewers to trigger select CI tests on top of fastcheck CI.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

🚀

[gemini-code-assist]

Code Review

This pull request introduces two key features: support for LoRA on the embedding and unembedding layers for Qwen3 models, and the ability to use LoRA without expanding the vocabulary by setting lora_extra_vocab_size=0. The changes are well-implemented across the configuration, LoRA layers, and model definitions. The logic correctly handles the zero vocabulary padding case by conditionally executing code related to extra vocabulary embeddings. The pull request also includes a comprehensive suite of new tests that validate these changes, ensuring correctness and preventing regressions. The code quality is high, and I did not find any issues.

[jeejeelee]

Thank you for your contribution, we are currently removing LoRA with extra vocal size, see: #23540

[klshuster]

Thanks @jeejeelee, in that case I can wait for that PR to merge and rebase on top (so i'll just have the qwen3 unembed changes). Alternatively, I may just remove the vocab changes from this PR proactively. Will give it a few days (or please let me know what approach you'd prefer)

[hmellor]

These conflicts are caused by our migration to ruff. Please see https://vllm-dev.slack.com/archives/C07R5Q1Q2BB/p1759663228844749 which contains detailed instructions to make updating your branch as painless as possible.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @klshuster.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork