[Frontend] Expose custom args in OpenAI APIs

[afeldman-nm]

Add a vllm_xargs: Optional[dict[str, Union[str,int,float]]] field to CompletionRequest, ChatCompletionRequest and TranscriptionRequest (these are the only OpenAIBaseModel subclasses which had a logits_processors field in v0.) This field is injected into SamplingParams.extra_args via SamplingParams.from_optional(); each dict key/value pair in extra_args becomes an assignment to an attribute of sampling_params.

Purpose

Enable extensible features such as logits processors and plugins to receive arbitrary custom arguments via the REST API. Mirror SamplingParams.extra_args in the REST API.

Test plan

Does not require additional unit tests (when logitsprocs extensibility is introduced later, this will implicitly test custom args). Pre-existing unit tests must pass so we know pre-existing features are not being broken.

Test results

N/A

Documentation changes

• SamplingParams docstring clarifies that extra_args may plumb custom args to logitsprocs, plugins, etc. (previous it just said logitsprocs)
• In vllm/entrypoints/openai/protocol.py: for CompletionRequest, ChatCompletionRequest, and TranscriptionRequest, move vllm_xargs definition inside the # --8<-- [start:completion-extra-params] section and add more detail to the description string.

Final note

The pre-existing behavior of protocol.py ChatCompletionRequest and CompletionRequest is that kv_transfer_params is passed into the engine via SamplingParams.extra_args; this PR simply merges vllm_xargs into SamplingParams.extra_args alongside kv_transfer_params. In the future it may be worth considering whether SamplingParams.extra_args is the best pathway for plumbing kv_transfer_params into the engine; it would seem to break the convention that SamplingParams.extra_args is not intended for "in-tree" functionality.

RFC: #17191

Fixes #16802

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[njhill]

Thanks @afeldman-nm! It would be good to include a test that shows how these can be passed via the OpenAI client sdk using its extra_body option: https://github.com/openai/openai-python?tab=readme-ov-file#undocumented-request-params

I'm unsure whether we want these new custom args to be in a nested json object (as you've done here) or just extra top-level args.

[afeldman-nm]

Thanks @njhill . Agree regarding the unit test. I need to think a bit about the right way to do it

[afeldman-nm]

@njhill @comaniac @WoosukKwon

CC: @robertgshaw2-redhat

[afeldman-nm]

Thanks for your review @comaniac . After chatting with Cody, I think this interface change is sufficiently impactful to merit an RFC which I will write and share shortly.

[afeldman-nm]

@afeldman-nm Glad to see it is still in progress. My use case is passing in truncate_prompt_tokens sampling parameters. Can we just unit test what we can test now and add more comprehensive unit tests when the logits process work is done?

Hi @helloworld1 - working on getting this PR landed as-is

[njhill]

Thanks @afeldman-nm LGTM, just a couple of minor comments.

[njhill]

Thanks @afeldman-nm