Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

希望 AI Proxy 插件能对未对接的接口进行反代 #1690

Open
daixijun opened this issue Jan 18, 2025 · 4 comments
Open

希望 AI Proxy 插件能对未对接的接口进行反代 #1690

daixijun opened this issue Jan 18, 2025 · 4 comments
Assignees
Labels

Comments

@daixijun
Copy link
Contributor

Why do you need it?

Is your feature request related to a problem? Please describe in details

How could it be?

AI Proxy 插件目前仅支持 completions / embeddings 相关接口的调用 , 但实际场景中可能还会使用到 audio / images / files / batches 等接口,希望能支持对接,或对不支持的接口请求透传(反代)。

Other related information

Add any other context or screenshots about the feature request here.

@johnlanni johnlanni added the help wanted Extra attention is needed label Jan 20, 2025
@github-project-automation github-project-automation bot moved this to Todo in Higress Jan 20, 2025
@johnlanni
Copy link
Collaborator

短期计划,支持 openai 的所有 API 能力透明反代;

长期计划,先以 openai 的 API 为锚,对目前已经支持的所有 provider 的能力做一轮梳理,看都能对齐哪些能力进行协议转换,后续完善到这个 HEP 里持续跟踪实现:#940

@pepesi
Copy link
Contributor

pepesi commented Jan 22, 2025

我也在看这个feature,现在的ai-proxy插件只限定了某些能力,如果不是插件内写明的收到支持的能力类型,会直接抛出errUnsupportedApiName的错误。但实际上,有些服务提供商可能已经直接支持了,如 zhipu 的embeddings 能力。

这可能需要在ai-proxy的配置中,为指定的provider配置受支持的能力,如果是收到支持的,可以直接透传。

@johnlanni
Copy link
Collaborator

@pepesi 你有兴趣实现一下吗

@pepesi
Copy link
Contributor

pepesi commented Jan 22, 2025

@pepesi 你有兴趣实现一下吗

正在...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Todo
Development

No branches or pull requests

4 participants