[Question]: qwen2-7B 是否支持 tool_choice为必选项required

[mawenju203]

Has this been raised before?

• I have checked the GitHub README.
• I have checked the Qwen documentation and cannot find an answer there.
• I have searched the issues and there is not a similar one.
• I confirm that this is not a bug report, a feature request, or a badcase.

Description

qwen2-7B 是否支持 tool_choice如果为：required，

必须调用tools中的一个
"tools": [
{
"function": {}
}]

是否支持这种呢？这是让模型判断还是自己修改代码呢？

[jklj077]

Hi, please refer to Qwen-Agent if you need the model to use a specific tool. Follow the instructions there, and you should be able to pass function_choice as a field of the extra_generate_cfg parameter to the chat method.

[mawenju203]

CUDA_VISIBLE_DEVICES=1 python -m vllm.entrypoints.openai.api_server --model ./Qwen2.5-7B-Instruct-AWQ --port 8009 --enable-auto-tool-choice --tool-call-parser hermes --max-seq-len-to-capture 2048 --gpu-memory-utilization 0.7

这个是我的启动方式，

我参考vllm中的vllm/entrypoints/openai/api_server.py

增加了这个参数；
"tool_choice":{
"function":{"name": "SearchTool"}
},
SearchTool：是自定义的tools；

但是报错了

numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
No implementation of function Function(<built-in function getitem>) found for signature:
 
 >>> getitem(class(int64), slice<a:b>)
 
There are 22 candidate implementations:
  - Of which 22 did not match due to:
  Overload of function 'getitem': File: <numerous>: Line N/A.
    With argument(s): '(class(int64), slice<a:b>)':
   No match.

During: typing of intrinsic-call at /home/mawenju/anaconda3/lib/python3.10/site-packages/outlines/fsm/regex.py (721)
During: typing of static-get-item at /home/mawenju/anaconda3/lib/python3.10/site-packages/outlines/fsm/regex.py (721)

File "../../anaconda3/lib/python3.10/site-packages/outlines/fsm/regex.py", line 721:
def get_vocabulary_transition_keys(
    <source elided>
    """
    vocab_transition_keys = numba.typed.List.empty_list(numba.int64[:])

[thiner]

你已经在启动参数中开启了--enable-auto-tool-choice，所以你的调用参数就应该改为"tool_choice": "auto".
我用vllm 0.6.2运行qwen2.5-72b-instruct，对工具调用的支持非常好！

[endNone]

tool-call-parser

How did you confirm the tool-call-parser?

[crystal-ltf]

你已经在启动参数中开启了--enable-auto-tool-choice，所以你的调用参数就应该改为"tool_choice": "auto". 我用vllm 0.6.2运行qwen2.5-72b-instruct，对工具调用的支持非常好！

我也是使用vllm 0.6.2，但运行的是qwen2.5-72b-instruct-AWQ，无法进行工具调用，后来又试了qwen2.5-72b-instruct是可以的，所以AWQ版本是不支持工具调用吗？

[jklj077]

vLLM does not support Qwen2 function calling. Please open a separate issue for Qwen2.5.

[thiner]

vllm supports tools calling for Qwen2.5 models perfectly.