add option "--expand-tools-even-if-tool-choice-none" #2

okdshin · 2025-04-25T07:27:42Z

Add option to include tool definitions even when tool_choice is 'none'

Summary

This PR adds a new command-line option --expand-tools-even-if-tool-choice-none which allows including tool definitions in prompts even when tool_choice='none'.

Motivation

In the current implementation, when tool_choice is set to 'none', all tool definitions are removed from the request, preventing the model from seeing the tool schemas. This change enables a workflow where:

The model can be aware of available tools (via their definitions in the prompt)
But is not expected to use them automatically (since tool_choice='none')

This is useful for:

Models that need to "plan" about available tools before using them in subsequent requests
Cases where tool descriptions provide useful context even if the tools aren't used
Situations where you want the model to reference tool capabilities in its response without actually calling them

Implementation

Added a new CLI flag --expand-tools-even-if-tool-choice-none (default: False)
Modified the request validation in protocol.py to no longer remove tools when tool_choice='none'
Added the new parameter to OpenAIServingChat and passed it through from the API server

github-actions · 2025-04-25T07:27:51Z

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

…ength from tokenizer config (vllm-project#19660) Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>

Signed-off-by: Chengji Yao <chengjiyao@google.com> Co-authored-by: mgoin <mgoin64@gmail.com>

Signed-off-by: Andy Xie <andy.xning@gmail.com>

Signed-off-by: Russell Bryant <rbryant@redhat.com>

) Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>

…llm-project#19626) Signed-off-by: Nick Hill <nhill@redhat.com>

…9557)

…rts. (vllm-project#19652) Signed-off-by: Shawn Tan <shawntan@ibm.com>

) Signed-off-by: Isotr0py <2037008807@qq.com>

Signed-off-by: Andy Xie <andy.xning@gmail.com>

…oject#19662) Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>

Signed-off-by: Bill Nell <bnell@redhat.com>

…h MiniMaxText01ForCausalLM) (vllm-project#19677) Signed-off-by: QscQ <qscqesze@gmail.com>

…19446) Signed-off-by: Russell Bryant <rbryant@redhat.com>

… 100 (vllm-project#19563)

…t#19641) Signed-off-by: conroy-cheers <conroy@corncheese.org>

Signed-off-by: David Xia <david@davidxia.com>

Signed-off-by: nguyenhoangthuan99 <thuanhppro12@gmail.com>

Signed-off-by: Di Liu <liu-di@sjtu.edu.cn>

Signed-off-by: drisspg <drisspguessous@gmail.com>

…er (vllm-project#19686) Signed-off-by: 刘全 <quan.liu2@dbappsecurity.com.cn> Co-authored-by: 刘全 <quan.liu2@dbappsecurity.com.cn>

Signed-off-by: Qiang Li <qiang.li2@amd.com>

…oject#19583)

Signed-off-by: reidliu41 <reid201711@gmail.com> Co-authored-by: reidliu41 <reid201711@gmail.com>

…9851) Signed-off-by: reidliu41 <reid201711@gmail.com> Co-authored-by: reidliu41 <reid201711@gmail.com>

Signed-off-by: Andy Xie <andy.xning@gmail.com>

Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>

Signed-off-by: nie3e <adrcwiek@gmail.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

…llm-project#19164) Signed-off-by: Chendi.Xue <chendi.xue@intel.com>

…-project#18777) Signed-off-by: Vlad Mihailescu <vtmihailescu@gmail.com>

…ts (vllm-project#19901) Signed-off-by: jiang1.li <jiang1.li@intel.com>

…project#19749)

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

…t#19910) Signed-off-by: Rabin Adhikari <rabin.adk1@gmail.com>

Signed-off-by: Isotr0py <2037008807@qq.com>

Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>

Signed-off-by: okada shintarou <okada@preferred.jp>

This reverts commit 377f4ac3d47d67ca05ed2df6054aab6faec02b4d. Signed-off-by: okada shintarou <okada@preferred.jp>

Signed-off-by: okada shintarou <okada@preferred.jp>

Co-authored-by: Aaron Pham <Aaronpham0103@gmail.com> Signed-off-by: okada shintarou <okada@preferred.jp>

Signed-off-by: okada shintarou <okada@preferred.jp>

okdshin force-pushed the specifiable_tools_expansion branch 5 times, most recently from ef2bf5f to 25aec8d Compare June 14, 2025 13:29

yeqcharlotte and others added 23 commits June 16, 2025 06:30

[Misc] Fix skipped max-model-len validation when deriving max model l…

b692e9c

…ength from tokenizer config (vllm-project#19660) Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>

[TPU] support attention head dim smaller than 128 (vllm-project#19620)

a77aea5

Signed-off-by: Chengji Yao <chengjiyao@google.com> Co-authored-by: mgoin <mgoin64@gmail.com>

[MISC] typo fix (vllm-project#19672)

26bc46e

Signed-off-by: Andy Xie <andy.xning@gmail.com>

[CI] Add mteb testing for rerank models (vllm-project#19344)

f40f763

[Docs] Move multiproc doc to v1 dir (vllm-project#19651)

8d12070

Signed-off-by: Russell Bryant <rbryant@redhat.com>

[Kernel] GGUF MMVQ kernel for multiple input vectors (vllm-project#18754

dec66d2

) Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>

[BugFix] Don't catch BaseException when dumping execute_model errors (v…

ee35e96

…llm-project#19626) Signed-off-by: Nick Hill <nhill@redhat.com>

[DOC] Add reasoning capability to vLLM streamlit code (vllm-project#1…

3e75069

…9557)

[Feature]:Allow for Granite MoE Hybrid models with _only_ shared expe…

4d54240

…rts. (vllm-project#19652) Signed-off-by: Shawn Tan <shawntan@ibm.com>

[Bugfix] Fix TP inference for Flex attention backend (vllm-project#19657

1173804

) Signed-off-by: Isotr0py <2037008807@qq.com>

[MISC] bump huggingface_hub pkg to 0.33.0 (vllm-project#19547)

c3fec47

Signed-off-by: Andy Xie <andy.xning@gmail.com>

[Bugfix] fix missing 'finish_reason': null in streaming chat (vllm-pr…

836d4ce

…oject#19662) Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>

[Kernels] Use empty for modular MoE workspaces (vllm-project#19667)

5e5baa9

Signed-off-by: Bill Nell <bnell@redhat.com>

[Model] Add support for MiniMaxM1ForCausalLM (shares architecture wit…

387bdf0

…h MiniMaxText01ForCausalLM) (vllm-project#19677) Signed-off-by: QscQ <qscqesze@gmail.com>

[V1] Change return type on get_multimodal_embeddings() (vllm-project#…

90f9c2e

…19446) Signed-off-by: Russell Bryant <rbryant@redhat.com>

[Quantization] Remove FP4 emulation; Fall-back to marlin for device <…

6bc7b57

… 100 (vllm-project#19563)

[Fix] Fall back to Gloo when NCCL backend is unavailable (vllm-projec…

0860087

…t#19641) Signed-off-by: conroy-cheers <conroy@corncheese.org>

[doc] add project flag to gcloud TPU command (vllm-project#19664)

119f683

Signed-off-by: David Xia <david@davidxia.com>

[Wheel Size] Only build FA2 8.0+PTX (vllm-project#19336)

0733495

[Frontend] add chunking audio for > 30s audio (vllm-project#19597)

ede5c4e

Signed-off-by: nguyenhoangthuan99 <thuanhppro12@gmail.com>

[DOC] fix doc typos (vllm-project#19600)

5b3ad5e

Signed-off-by: Di Liu <liu-di@sjtu.edu.cn>

Fixes IMA for TP w/ flex-attention (vllm-project#19712)

ddfed31

Signed-off-by: drisspg <drisspguessous@gmail.com>

[Core] add remove_seq_from_computed_blocks_tracker to BlockSpaceManag…

5c76b9c

…er (vllm-project#19686) Signed-off-by: 刘全 <quan.liu2@dbappsecurity.com.cn> Co-authored-by: 刘全 <quan.liu2@dbappsecurity.com.cn>

qli88 and others added 15 commits June 20, 2025 12:43

[Bugfix] Enable PP with AITER+V1 (vllm-project#19822)

e3a3e4d

Signed-off-by: Qiang Li <qiang.li2@amd.com>

[Bugfix][Ray] Set the cuda context eagerly in the ray worker (vllm-pr…

5e666f7

…oject#19583)

[Misc] update cuda version (vllm-project#19526)

089a306

Signed-off-by: reidliu41 <reid201711@gmail.com> Co-authored-by: reidliu41 <reid201711@gmail.com>

[Misc] refactor example - openai_transcription_client (vllm-project#1…

e384f2f

…9851) Signed-off-by: reidliu41 <reid201711@gmail.com> Co-authored-by: reidliu41 <reid201711@gmail.com>

[Kernel] correct cpu worker function parameter type (vllm-project#19745)

71d1219

Signed-off-by: Andy Xie <andy.xning@gmail.com>

[Fix] import regex instead of re (vllm-project#19875)

7771d1d

Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>

[Model] GPT2ForSequenceClassification model (vllm-project#19663)

f1e840e

Signed-off-by: nie3e <adrcwiek@gmail.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

[custom_op][vllm-plugin] update custom_op class to use op_registry (v…

7e8977f

…llm-project#19164) Signed-off-by: Chendi.Xue <chendi.xue@intel.com>

Export NaNs in logits to scheduler_stats if output is corrupted (vllm…

2e3e3c8

…-project#18777) Signed-off-by: Vlad Mihailescu <vtmihailescu@gmail.com>

[CPU][CI] Fallback sliding window to v0 and fix CPU pooling model tes…

79f2f1c

…ts (vllm-project#19901) Signed-off-by: jiang1.li <jiang1.li@intel.com>

[Kernel] mark TorchSDPABackend swap_blocks NotImplementedError (vllm-…

71baf85

…project#19749)

[Misc] Clean up useless code (vllm-project#19889)

e773a9e

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

Fix: Check the type of params to be a Sequence not list. (vllm-projec…

8ca81bb

…t#19910) Signed-off-by: Rabin Adhikari <rabin.adk1@gmail.com>

[Bugfix] Fix bnb 8bit model weights loading (vllm-project#19917)

6f170f1

Signed-off-by: Isotr0py <2037008807@qq.com>

[New model support]Support Tarsier2 (vllm-project#19887)

c3bf9ba

Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>

okdshin force-pushed the specifiable_tools_expansion branch from 35649de to f2bc427 Compare June 21, 2025 14:36

okdshin added 8 commits June 21, 2025 23:59

add option "--expand-tools-even-if-tool-choice-none"

ba347cd

Signed-off-by: okada shintarou <okada@preferred.jp>

fix format

5b1b5e7

Signed-off-by: okada shintarou <okada@preferred.jp>

fix format

38eea42

Signed-off-by: okada shintarou <okada@preferred.jp>

remove the new option and change the default behaviour

bc031d9

Signed-off-by: okada shintarou <okada@preferred.jp>

Revert "remove the new option and change the default behaviour"

b5e4164

This reverts commit 377f4ac3d47d67ca05ed2df6054aab6faec02b4d. Signed-off-by: okada shintarou <okada@preferred.jp>

update help

e3733a9

Signed-off-by: okada shintarou <okada@preferred.jp>

add warn

1ffd7f0

Signed-off-by: okada shintarou <okada@preferred.jp>

fix docs

e282ebb

Signed-off-by: okada shintarou <okada@preferred.jp>

okdshin force-pushed the specifiable_tools_expansion branch from 9559897 to e282ebb Compare June 21, 2025 15:00

okdshin and others added 3 commits June 22, 2025 00:14

fix format

b51c7b9

Signed-off-by: okada shintarou <okada@preferred.jp>

fix format

0a0920a

Signed-off-by: okada shintarou <okada@preferred.jp>

Apply suggestions from code review

90eabb2

Co-authored-by: Aaron Pham <Aaronpham0103@gmail.com> Signed-off-by: okada shintarou <okada@preferred.jp>

okdshin force-pushed the specifiable_tools_expansion branch from 8750191 to 90eabb2 Compare June 22, 2025 07:56

remove assert

8a45fa8

Signed-off-by: okada shintarou <okada@preferred.jp>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

add option "--expand-tools-even-if-tool-choice-none" #2

add option "--expand-tools-even-if-tool-choice-none" #2

Uh oh!

okdshin commented Apr 25, 2025 •

edited by github-actions bot

Loading

Uh oh!

github-actions bot commented Apr 25, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

68 participants

add option "--expand-tools-even-if-tool-choice-none" #2

Are you sure you want to change the base?

add option "--expand-tools-even-if-tool-choice-none" #2

Uh oh!

Conversation

okdshin commented Apr 25, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Add option to include tool definitions even when tool_choice is 'none'

Summary

Motivation

Implementation

Uh oh!

github-actions bot commented Apr 25, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

68 participants

okdshin commented Apr 25, 2025 •

edited by github-actions bot

Loading