[Speculators][Speculative Decoding] Fix gpt-oss eagle3 accuracy issue

[jiahanc]

Purpose

Inherit from #23596

1. Current gpt-oss eagle3 does not work on https://huggingface.co/nvidia/gpt-oss-120b-Eagle3, mainly config issues like vocab_size and lm_head
2. The current implementation in spec-dec suppose only 1 attention groups :https://github.com/vllm-project/vllm/blob/main/vllm/v1/worker/gpu_model_runner.py#L1178-L1180.
   And https://github.com/vllm-project/vllm/blob/main/vllm/v1/spec_decode/eagle.py#L194-L199 only use the 1st attention builder.
   This causes accuracy issues because on models that have multiple attentions like gpt-oss, it will select the wrong attention group so draft model will use wrong attention as well as overwrite the KV cache of the target models

Test Plan

lm_eval --model local-completions --tasks gsm8k --model_args model=openai/gpt-oss-120b,base_url=http://0.0.0.0:30000/v1/completions,max_retries=3,tokenized_requests=False,timeout=1200,max_gen_toks=2048,max_length=8192 --batch_size 2048 --trust_remote_code --limit 0.8

Test Result

Before fix
Low AR:

(APIServer pid=21745) INFO 09-22 10:49:59 [metrics.py:96] SpecDecoding metrics: Mean acceptance length: 1.04, Accepted throughput: 77.80 tokens/s, Drafted throughput: 6284.37 tokens/s, Accepted: 778 tokens, Drafted: 62847 tokens, Per-position acceptance rate: 0.035, 0.002, 0.000, Avg Draft acceptance rate: 1.2%

Wrong Accuracy

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.0038|±  |0.0019|
|     |       |strict-match    |     5|exact_match|↑  |0.0000|±  |0.0000|

After fix
Correct AR

(APIServer pid=22270) INFO 09-22 10:55:25 [metrics.py:96] SpecDecoding metrics: Mean acceptance length: 2.81, Accepted throughput: 4559.99 tokens/s, Drafted throughput: 7571.32 tokens/s, Accepted: 45604 tokens, Drafted: 75720 tokens, Per-position acceptance rate: 0.755, 0.578, 0.474, Avg Draft acceptance rate: 60.2%

Correct Accuracy

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.8362|±  |0.0114|
|     |       |strict-match    |     5|exact_match|↑  |0.5909|±  |0.0151|

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[benchislett]

LGTM

[mgoin]

LGTM, thank you!

[mgoin]

@jiahanc The failures in CI seem related

[jiahanc]

@jiahanc The failures in CI seem related
Fixed and eagle test passes locally