feat: Enable engine-level arguments with speculators models

[rahul-tuli]

This PR enables users to combine engine-level arguments (like --tensor-parallel-size, --seed, --max-model-len) with speculators models using simplified command syntax.

Problem

Previously, users had to use verbose commands with explicit speculative configuration:

  VLLM_USE_V1=1 vllm serve "RedHatAI/Meta-Llama-3.1-8B-Instruct-FP8-dynamic" \
      --seed 42 \
      --tensor-parallel-size 4 \
      --speculative-config '{"model": "nm-testing/SpeculatorLlama3-1-8B-Eagle3-converted-0717-quantized", "num_speculative_tokens": 3, 
  "method":"eagle3"}'

Solution

Now users can use the simplified syntax:

vllm serve --seed 42 --tensor-parallel-size 4 "nm-testing/SpeculatorLlama3-1-8B-Eagle3-converted-0717-quantized"

• Detects speculators models by checking for embedded speculators_config
• Extracts and converts embedded speculative configuration to vLLM format
• CLI Precedence: Engine-level CLI arguments take precedence over embedded settings
• Maintains compatibility with regular models and existing workflows

Testing

Serve Command:

  export CUDA_VISIBLE_DEVICES=0,1
  export VLLM_USE_V1=1
  vllm serve \
      --host 127.0.0.1 \
      --port 8000 \
      --tensor-parallel-size 2 \
      --seed 42 \
      --max-model-len 4096 \
      "nm-testing/SpeculatorLlama3-1-8B-Eagle3-converted-0717-quantized"

Test Request:

  curl -s \
      -H "Content-Type: application/json" \
      -d '{
          "prompt": "The capital of France is",
          "max_tokens": 10,
          "temperature": 0.7
      }' \
      "http://127.0.0.1:8000/v1/completions"

[dsikka]

We have several other functions that are very similar to this. Instead of adding this method, I'd look into reusing something like get_config for example

[dsikka]

Wont this be incorrect?
You're missing the update step specific to each algorithm https://github.com/neuralmagic/vllm/blob/0faf3cc3e84a83bac56492d297f2c9909fec7de9/vllm/transformers_utils/configs/speculators/base.py#L39?

[rahul-tuli]

Closed in favor of #25250