Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pixtral-12b-2409 使用 openai 渠道,Stream 工具调用结果为空 #540

Open
5 tasks done
geneives233 opened this issue Oct 23, 2024 · 10 comments
Open
5 tasks done
Labels
bug Something isn't working

Comments

@geneives233
Copy link

例行检查

  • 我已确认目前没有类似 issue
  • 我已确认我已升级到最新版本
  • 我已完整查看过项目 README,尤其是常见问题部分
  • 我理解并愿意跟进此 issue,协助测试和提供反馈
  • 我理解并认可上述内容,并理解项目维护者精力有限,不遵循规则的 issue 可能会被无视或直接关闭

问题描述
在 v0.3.0.1-alpha.2 版本上使用 OpenAI 渠道设置好 pixtral-12b-2409, 发送 Stream 工具调用结果为空,直接调用官方是正常的。
复现步骤
使用OpenAI 渠道设置pixtral-12b-2409, 发送 Stream 工具调用请求。
预期结果
返回正常结果。
相关截图
官方截图:
image
new-api 截图:
image

@geneives233 geneives233 added the bug Something isn't working label Oct 23, 2024
@123hi123
Copy link

可以更新看看再回來測試
順便確認 這個模型能不能流式

@geneives233
Copy link
Author

这是官方调用结果:
image
这是new-api调用结果:
image
两次请求参数是相同的:

{
  "model": "pixtral-12b-2409",
  "stream": true,
  "messages": [
    {
      "role": "user",
      "content": "What is the weather in Toronto?"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_current_weather",
        "description": "Get the current weather for a city",
        "parameters": {
          "type": "object",
          "properties": {
            "city": {
              "type": "string",
              "description": "The name of the city"
            }
          },
          "required": [
            "city"
          ]
        }
      }
    }
  ]
}

@geneives233
Copy link
Author

这个也是我更新到最新版本的结果

@123hi123
Copy link

在playgroud裡面試試看呢

@geneives233
Copy link
Author

image
聊天是ok的,只是stream工具调用有问题。

@123hi123
Copy link

有一個可能就是mistral沒那麼兼容openai格式

@geneives233
Copy link
Author

可能是格式的问题,下面这个是官方的返回结果:

data: {
  "id": "f7831ba1240a4a26a0f9292a65668945",
  "object": "chat.completion.chunk",
  "created": 1731295728,
  "model": "pixtral-12b-2409",
  "choices": [
    {
      "index": 0,
      "delta": {
        "role": "assistant",
        "content": ""
      },
      "finish_reason": null
    }
  ]
}

data: {
  "id": "f7831ba1240a4a26a0f9292a65668945",
  "object": "chat.completion.chunk",
  "created": 1731295728,
  "model": "pixtral-12b-2409",
  "choices": [
    {
      "index": 0,
      "delta": {
        "content": null,
        "tool_calls": [
          {
            "id": "Mu9OidbCQ",
            "function": {
              "name": "get_current_weather",
              "arguments": "{\"city\": \"Toronto\"}"
            }
          }
        ]
      },
      "finish_reason": "tool_calls"
    }
  ],
  "usage": {
    "prompt_tokens": 83,
    "total_tokens": 104,
    "completion_tokens": 21
  }
}

@123hi123
Copy link

可能是格式的问题,下面这个是官方的返回结果:

data: {
  "id": "f7831ba1240a4a26a0f9292a65668945",
  "object": "chat.completion.chunk",
  "created": 1731295728,
  "model": "pixtral-12b-2409",
  "choices": [
    {
      "index": 0,
      "delta": {
        "role": "assistant",
        "content": ""
      },
      "finish_reason": null
    }
  ]
}

data: {
  "id": "f7831ba1240a4a26a0f9292a65668945",
  "object": "chat.completion.chunk",
  "created": 1731295728,
  "model": "pixtral-12b-2409",
  "choices": [
    {
      "index": 0,
      "delta": {
        "content": null,
        "tool_calls": [
          {
            "id": "Mu9OidbCQ",
            "function": {
              "name": "get_current_weather",
              "arguments": "{\"city\": \"Toronto\"}"
            }
          }
        ]
      },
      "finish_reason": "tool_calls"
    }
  ],
  "usage": {
    "prompt_tokens": 83,
    "total_tokens": 104,
    "completion_tokens": 21
  }
}

應該是 我今天嘗試了一下
雖然號稱支援openai
但有些地方命名可能不一樣
你被拒絕的理由應該是參數不正確

@geneives233
Copy link
Author

也许可以在新渠道Mistral下优化代码,就不使用OpenAI渠道了。

@123hi123
Copy link

確實是 目前沒啥問題 比較神奇的是pixtral沒有像他說的可以讀很多圖片
沒幾張就不行了

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants