✨ feat: Support Doubao Models #3055

sxjeru · 2024-06-27T04:41:36Z

💻 变更类型 | Change Type

🔀 变更说明 | Description of Change

参考 #2804 实现，已初步实现聊天。

豆包大模型采用独立模型终结点，即每个用户的 model id 不同，这就要求用户先手动配置 src/config/modelProviders/doubao.ts 才可进行聊天。
连通性测试使用经环境变量添加的模型（~~不知该如何实现，其他模型也有类似问题~~ 参考 Azure 已实现）
不支持客户端请求，存在 CORS 跨域问题。（其他 provider 也有类似情况，已禁用客户端请求）

关于第一点，暂定解决方案是使用环境变量预配，如：
DOUBAO_MODEL_LIST = Doubao-pro-32k->ep-2024xxxx-xxxx=displayName
其中 Doubao-pro-32k 是 deploymentName，请参考 src/config/modelProviders/doubao.ts

参考：https://lobehub.com/zh/docs/self-hosting/environment-variables/model-provider#azure-model-list

📝 补充信息 | Additional Information

~~模型支持 image_url，暂未适配。~~

根据文档，v3 api 接口暂不支持 image_url。
https://www.volcengine.com/docs/82379/1298454

~~不确定本 PR 是否还有必要完善，考虑第一个比较严重的问题，很可能会被砍掉。~~

close #5095

vercel · 2024-06-27T04:41:40Z

@sxjeru is attempting to deploy a commit to the LobeHub Pro Team on Vercel.

A member of the Team first needs to authorize it.

lobehubbot · 2024-06-27T04:41:47Z

👍 @sxjeru

Thank you for raising your pull request and contributing to our Community
Please make sure you have followed our contributing guidelines. We will review it as soon as possible.
If you encounter any problems, please feel free to connect with us.
非常感谢您提出拉取请求并为我们的社区做出贡献，请确保您已经遵循了我们的贡献指南，我们会尽快审查它。
如果您遇到任何问题，请随时与我们联系。

codecov · 2024-06-27T04:43:56Z

Codecov Report

Attention: Patch coverage is 92.50000% with 9 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/server/modules/AgentRuntime/index.ts	14.28%	6 Missing ⚠️
src/libs/agent-runtime/AgentRuntime.ts	57.14%	3 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff            @@
##             main    #3055    +/-   ##
========================================
  Coverage   92.30%   92.30%            
========================================
  Files         498      500     +2     
  Lines       36243    36363   +120     
  Branches     3293     3446   +153     
========================================
+ Hits        33454    33565   +111     
- Misses       2789     2798     +9

Flag	Coverage Δ
app	`92.30% <92.50%> (+<0.01%)`	⬆️
server	`97.33% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

arvinxx · 2024-07-01T09:36:11Z

独立终结点的模式比较像 Azure 的吧？是不是可以参考 azure 的方案来？

lobehubbot · 2024-07-01T09:36:22Z

The independent endpoint model is more like Azure's, right? Is it possible to refer to the azure solution?

Sam5440 · 2024-07-25T14:00:43Z

所以，这个模型还有希望支持吗

lobehubbot · 2024-07-25T14:00:54Z

So, is there any hope of supporting this model?

sxjeru · 2024-07-25T15:31:06Z

@Sam5440
其实这个分支的文字聊天应该已经能用了，只是需要你配置一下 src/config/modelProviders/doubao.ts 。
这边还在研究独立终结点的简便实现方法，短期不会有进展。

lobehubbot · 2024-07-25T15:31:21Z

@Sam5440
In fact, text chat in this branch should already be available, you just need to configure src/config/modelProviders/doubao.ts.
We are still studying a simple implementation method for independent endpoints, and there will be no progress in the short term.

davaded · 2024-08-15T01:46:55Z

感觉可以参考ChatGPT-Next-Web他们的处理方式
使用字节跳动提供的豆包模型的时候定义deploy_id
+Doubao-lite-4k@bytedance=ep-2024xxxx-xxx
以上配置会新增一个Doubao-lite-4k(ByteDance)的模型，当选择这个模型的时候，会将请求发送到ep-2024xxxx-xxx这个deploy-id服务上面

lobehubbot · 2024-08-15T01:47:07Z

I feel like you can refer to ChatGPT-Next-Web for their handling methods.
Define deploy_id when using the beanbao model provided by ByteDance
+Doubao-lite-4k@bytedance=ep-2024xxxx-xxx
The above configuration will add a Doubao-lite-4k (ByteDance) model. When this model is selected, the request will be sent to the deploy-id service ep-2024xxxx-xxx.

sxjeru · 2024-08-15T06:16:50Z

@davaded 感谢提醒，差点忘了 lobechat 也支持环境变量端添加 model 来着。

lobehubbot · 2024-08-15T06:17:01Z

@davaded Thanks for the reminder, I almost forgot that lobechat also supports adding model on the environment variable side.

davaded · 2024-08-15T07:33:15Z

豆包毕竟是特例，这种在环境变量上操作是最简单的我觉得，期待你的成功，成功后记得call me

lobehubbot · 2024-08-15T07:33:28Z

Doubao is a special case after all. This kind of operation on environment variables is the simplest. I look forward to your success. Remember to call me after success.

Sam5440 · 2024-08-18T05:49:10Z

怎么样了

lobehubbot · 2024-08-18T05:49:20Z

how's it going

sxjeru · 2024-08-18T05:59:45Z

算是能正常用了，通过在环境变量中预配模型。还有问题需要解决，这边也没啥思路。

lobehubbot · 2024-08-18T05:59:55Z

It works, but there are still problems that need to be solved, and there is no idea here.

arvinxx · 2024-10-22T12:33:41Z

@sxjeru 豆包的实现应该参考 azure 这样的来，azure 也有 deploymentId 的

lobehubbot · 2024-10-22T12:33:53Z

@sxjeru The implementation of beanbag should refer to azure. Azure also has deploymentId.

sxjeru · 2024-10-22T12:53:05Z

@arvinxx 其实没有那么复杂。

豆包不需要独立终结点，只是独立 model id，这点相当于在前端手动添加模型，这是 Lobechat 早已实现了的。
而环境变量预配的支持，应该算是补上最后一块拼图了。

lobehubbot · 2024-10-22T12:53:20Z

@arvinxx It’s not that complicated.

Doubao does not require an independent endpoint, only an independent model id. This is equivalent to manually adding a model on the front end, which Lobechat has already implemented.
The support for environment variable provisioning should be the last piece of the puzzle.

arvinxx · 2024-10-28T02:11:19Z

@sxjeru azure 每个都是独立 model id 的模式。Azure 的语法我也设计了:

AZURE_MODEL_LIST=gpt-35-turbo->my-deploy=GPT 3.5 Turbo,gpt-4-turbo->my-gpt4=GPT 4 Turbo<128000:vision:fc>

详见: https://lobehub.com/zh/docs/self-hosting/environment-variables/model-provider#azure-model-list

在Doubao的场景下我理解就等价于：

doubao_1->user_model_id=豆包模型1

lobehubbot · 2024-10-28T02:11:32Z

@sxjeru azure Each is a schema with an independent model id. I also designed the Azure syntax:

AZURE_MODEL_LIST=gpt-35-turbo->my-deploy=GPT 3.5 Turbo,gpt-4-turbo->my-gpt4=GPT 4 Turbo<128000:vision:fc>

For details, see: https://lobehub.com/zh/docs/self-hosting/environment-variables/model-provider#azure-model-list

In Doubao's scenario, I understand it to be equivalent to:

doubao_1->user_model_id=doubao model 1

src/config/llm.ts

arvinxx · 2024-10-28T06:00:40Z

可以配置禁用客户端请求，参考下 huggingface 的配置即可

lobehubbot · 2024-10-28T06:00:53Z

CORS can be configured to disable, please refer to the configuration of huggingface.

arvinxx · 2024-10-28T06:03:47Z

这个也参考 Azure 的就好，直接默认指定一个基础模型，然后用需要在环境变量中添加这个模型以及它的 deployName。走 Azure 模式时会自动替换成请求 deployName 的

lobehubbot · 2024-10-28T06:03:58Z

Just refer to Azure for this. Directly specify a basic model by default, and then add this model and its deployName to the environment variables as needed. If you use Azure mode, it will be automatically replaced by requesting deployName.

sxjeru · 2024-10-28T08:20:25Z

@arvinxx DOUBAO_MODEL_LIST 设置似乎有矛盾，不清楚 Azure 那边的逻辑是怎样的。

若设置 ep-2024xxxx-xxxx->Doubao-pro-32k=1234，聊天时 model 正确传参 ep-2024xxxx-xxxx，但连通性检查时会传递 Doubao-pro-32k，导致检查失败。

如果交换顺序设成 Doubao-pro-32k->ep-2024xxxx-xxxx=1234，连通性检查正常，无法聊天。

lobehubbot · 2024-10-28T08:20:38Z

@arvinxx There seems to be a contradiction in the DOUBAO_MODEL_LIST settings. I don’t know what the logic is on Azure’s side.

If ep-2024xxxx-xxxx->Doubao-pro-32k=1234 is set, the model parameter ep-2024xxxx-xxxx is correctly passed during chat, but Doubao-pro-32k is passed during connectivity check, causing the check to fail.

If the exchange sequence is set to Doubao-pro-32k->ep-2024xxxx-xxxx=1234, the connectivity check is normal and chat cannot be done.

arvinxx · 2024-10-29T07:36:19Z

之前是因为只有 Azure ，所以这里写死了判断，感觉既然doubao和azure一样，那么这部分的判断完全可以收敛成一个 ProviderCard 的配置，比如叫 useDeploymentName:true

lobe-chat/src/services/chat.ts

Lines 255 to 267 in 3e467ae

    
           if (provider === ModelProvider.Azure) { 
        
             const chatModelCards = modelProviderSelectors.getModelCardsById(provider)( 
        
               useUserStore.getState(), 
        
             ); 
        
             const deploymentName = chatModelCards.find((i) => i.id === model)?.deploymentName; 
        
             if (deploymentName) model = deploymentName; 
        
           } 
        
           const payload = merge( 
        
             { model: DEFAULT_AGENT_CONFIG.model, stream: true, ...DEFAULT_AGENT_CONFIG.params }, 
        
             { ...res, model }, 
        
           );

lobehubbot · 2024-10-29T07:36:32Z

Previously, because there was only Azure, the judgment was hard-coded here. I feel that since doubao is the same as azure, this part of the judgment can be completely condensed into a ProviderCard configuration, such as useDeploymentName:true

lobe-chat/src/services/chat.ts

Lines 255 to 267 in 3e467ae

    
           if (provider === ModelProvider.Azure) { 
        
             const chatModelCards = modelProviderSelectors.getModelCardsById(provider)( 
        
               useUserStore.getState(), 
        
             ); 
        
             const deploymentName = chatModelCards.find((i) => i.id === model)?.deploymentName; 
        
             if (deploymentName) model = deploymentName; 
        
           } 
        
           const payload = merge( 
        
             { model: DEFAULT_AGENT_CONFIG.model, stream: true, ...DEFAULT_AGENT_CONFIG.params }, 
        
             { ...res, model }, 
        
           );

arvinxx · 2024-10-29T07:38:22Z

标准写法应该是 Doubao-pro-32k->ep-2024xxxx-xxxx=豆包 Pro。

lobehubbot · 2024-10-29T07:38:35Z

The standard writing method should be Doubao-pro-32k->ep-2024xxxx-xxxx=1234.

arvinxx · 2025-01-15T04:59:34Z

@sxjeru 来 rebase 下？还是重新提个 PR ？

lobehubbot · 2025-01-15T04:59:45Z

@sxjeru Come to rebase? Or submit a PR again?

sxjeru added 2 commits June 27, 2024 11:46

feat: Add Doubao as a model provider

57dbdf1

fix

061a1d3

sxjeru added 2 commits July 1, 2024 23:42

Merge branch 'main' into doubao

686e41a

Update index.ts

57e4d27

sxjeru marked this pull request as draft July 4, 2024 10:23

sxjeru added 2 commits August 15, 2024 13:35

Merge branch 'main' into doubao

ee8aa17

ARK_MODEL_LIST

040b54f

spark

a1bf198

sxjeru marked this pull request as ready for review August 15, 2024 08:02

Merge branch 'main' into doubao

e0e6543

Update doubao.ts

1c4c9fe

Merge branch 'main' into doubao

bfd5ae9

arvinxx reviewed Oct 28, 2024

View reviewed changes

src/config/llm.ts Outdated Show resolved Hide resolved

sxjeru added 2 commits October 28, 2024 10:28

improve

b6885a4

Merge branch 'main' into doubao

186b8a6

Add Doubao preset models

1981bc2

dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Oct 28, 2024

sxjeru added 3 commits October 28, 2024 14:34

Merge branch 'main' into doubao

5775c68

尝试修复连通性测试

f05c972

Merge branch 'doubao' of https://github.com/sxjeru/lobe-chat into doubao

4551821

sxjeru closed this Jan 16, 2025

sxjeru mentioned this pull request Jan 16, 2025

✨ feat: Support Doubao Models #5481

Merged

11 tasks

✨ feat: Support Doubao Models #3055

✨ feat: Support Doubao Models #3055

Conversation

sxjeru commented Jun 27, 2024 • edited Loading

💻 变更类型 | Change Type

🔀 变更说明 | Description of Change

📝 补充信息 | Additional Information

vercel bot commented Jun 27, 2024

lobehubbot commented Jun 27, 2024

codecov bot commented Jun 27, 2024 • edited Loading

Codecov Report

arvinxx commented Jul 1, 2024

lobehubbot commented Jul 1, 2024

Sam5440 commented Jul 25, 2024

lobehubbot commented Jul 25, 2024

sxjeru commented Jul 25, 2024

lobehubbot commented Jul 25, 2024

davaded commented Aug 15, 2024

lobehubbot commented Aug 15, 2024

sxjeru commented Aug 15, 2024

lobehubbot commented Aug 15, 2024

davaded commented Aug 15, 2024

lobehubbot commented Aug 15, 2024

Sam5440 commented Aug 18, 2024

lobehubbot commented Aug 18, 2024

sxjeru commented Aug 18, 2024 • edited Loading

lobehubbot commented Aug 18, 2024

arvinxx commented Oct 22, 2024

lobehubbot commented Oct 22, 2024

sxjeru commented Oct 22, 2024

lobehubbot commented Oct 22, 2024

arvinxx commented Oct 28, 2024

lobehubbot commented Oct 28, 2024

arvinxx commented Oct 28, 2024 • edited Loading

lobehubbot commented Oct 28, 2024

arvinxx commented Oct 28, 2024 • edited Loading

lobehubbot commented Oct 28, 2024

sxjeru commented Oct 28, 2024

lobehubbot commented Oct 28, 2024

arvinxx commented Oct 29, 2024

lobehubbot commented Oct 29, 2024

arvinxx commented Oct 29, 2024 • edited Loading

lobehubbot commented Oct 29, 2024

arvinxx commented Jan 15, 2025

lobehubbot commented Jan 15, 2025

sxjeru commented Jun 27, 2024 •

edited

Loading

codecov bot commented Jun 27, 2024 •

edited

Loading

sxjeru commented Aug 18, 2024 •

edited

Loading

arvinxx commented Oct 28, 2024 •

edited

Loading

arvinxx commented Oct 28, 2024 •

edited

Loading

arvinxx commented Oct 29, 2024 •

edited

Loading