[Bugfix] Fix encoder-only model support for transformers backend

[Isotr0py]

Purpose

• Fix broken encoder-only model support on transformers backend after [Misc] Refactor get_kv_cache_spec into AttentionLayerBase #26587

(EngineCore_DP0 pid=5260) INFO 11-04 03:34:52 [base.py:120] Using Transformers backend.
(EngineCore_DP0 pid=5260) INFO 11-04 03:34:52 [cuda.py:414] Using FlexAttention backend.
(EngineCore_DP0 pid=5260) [2025-11-04 03:34:54] INFO _client.py:1025: HTTP Request: GET https://huggingface.co/api/models/papluca/xlm-roberta-base-language-detection "HTTP/1.1 200 OK"
(EngineCore_DP0 pid=5260) [2025-11-04 03:34:54] INFO _client.py:1025: HTTP Request: GET https://huggingface.co/api/models/papluca/xlm-roberta-base-language-detection/tree/main?recursive=false&expand=false "HTTP/1.1 200 OK"
(EngineCore_DP0 pid=5260) [2025-11-04 03:34:54] INFO _client.py:1025: HTTP Request: GET https://huggingface.co/api/models/papluca/xlm-roberta-base-language-detection/revision/main "HTTP/1.1 200 OK"
(EngineCore_DP0 pid=5260) [2025-11-04 03:34:55] INFO _client.py:1025: HTTP Request: HEAD https://huggingface.co/papluca/xlm-roberta-base-language-detection/resolve/main/model.safetensors.index.json "HTTP/1.1 404 Not Found"
(EngineCore_DP0 pid=5260) INFO 11-04 03:34:55 [weight_utils.py:480] No model.safetensors.index.json found in remote.
Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  1.94it/s]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  1.93it/s]
(EngineCore_DP0 pid=5260) 
(EngineCore_DP0 pid=5260) INFO 11-04 03:34:55 [default_loader.py:314] Loading weights took 0.65 seconds
(EngineCore_DP0 pid=5260) INFO 11-04 03:34:56 [gpu_model_runner.py:2997] Model loading took 0.5192 GiB and 3.491527 seconds
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843] EngineCore failed to start.
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843] Traceback (most recent call last):
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843]   File "/kaggle/working/vllm/vllm/v1/engine/core.py", line 834, in run_engine_core
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843]     engine_core = EngineCoreProc(*args, **kwargs)
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843]                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843]   File "/kaggle/working/vllm/vllm/v1/engine/core.py", line 602, in __init__
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843]     super().__init__(
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843]   File "/kaggle/working/vllm/vllm/v1/engine/core.py", line 109, in __init__
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843]     num_gpu_blocks, num_cpu_blocks, kv_cache_config = self._initialize_kv_caches(
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843]                                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843]   File "/kaggle/working/vllm/vllm/v1/engine/core.py", line 223, in _initialize_kv_caches
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843]     kv_cache_specs = self.model_executor.get_kv_cache_specs()
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843]                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843]   File "/kaggle/working/vllm/vllm/v1/executor/abstract.py", line 129, in get_kv_cache_specs
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843]     return self.collective_rpc("get_kv_cache_spec")
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843]   File "/kaggle/working/vllm/vllm/v1/executor/uniproc_executor.py", line 73, in collective_rpc
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843]     return [run_method(self.driver_worker, method, args, kwargs)]
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843]             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843]   File "/kaggle/working/vllm/vllm/v1/serial_utils.py", line 459, in run_method
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843]     return func(*args, **kwargs)
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843]            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843]   File "/kaggle/working/vllm/vllm/v1/worker/gpu_worker.py", line 373, in get_kv_cache_spec
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843]     return self.model_runner.get_kv_cache_spec()
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843]   File "/kaggle/working/vllm/vllm/v1/worker/gpu_model_runner.py", line 4746, in get_kv_cache_spec
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843]     if spec := attn_module.get_kv_cache_spec(self.vllm_config):
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843]                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843]   File "/kaggle/working/vllm/vllm/attention/layer.py", line 462, in get_kv_cache_spec
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843]     assert self.attn_type == AttentionType.DECODER
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=5260) ERROR 11-04 03:34:56 [core.py:843] AssertionError

Test Plan

pytest -s -v tests/models/test_initialization.py -k TransformersForSequenceClassification

Test Result

Test should pass with nightly Transformers now

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[gemini-code-assist]

Code Review

This pull request addresses a bug that prevented encoder-only models from working with the transformers backend. The issue, as indicated by the traceback, was an assertion error during the KV cache initialization, where the attention type was expected to be DECODER. The fix involves conditionally selecting the EncoderOnlyAttention class for encoder layers, which correctly signals that no KV cache is needed. The change is precise, correct, and effectively resolves the reported bug.

[DarkLight1337]

Why does passing attn_type not work? Are the two not equivalent?

[hmellor]

#26587 (comment)

[NickLucche]

@heheda12345 do you want to handle it inside the Attention class init to signal deprecation with a warning?

[heheda12345]

We need to handle it like this now. For the deprecation warning, just add one line of warning in Attention class? (not necessary in this PR)