Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add feature: Support for multiple models on Azure and Azure model mapper #3344

Closed
wants to merge 8 commits into from

Conversation

shendongming
Copy link

Added support for multiple models on Azure; in practice, we will deploy several models on Azure, such as gpt-4, gpt-3.5, etc.

Two changes:

  1. Added a new configuration item: AZURE_OPENAI_MODEL_MAPPER
  2. Extended the configuration item: AZURE_URL

The changes are as follows:

AZURE_URL (optional)

Example: https://{azure-resource-url}/openai/deployments/{deploy-name}

Example: https://xxx.openai.azure.com/openai/deployments/{deploy-name}

Azure deploy url.

If {deploy-name} is using the template mode, then it will automatically replace the path based on the model selected by the client.
If your model name is different from the deployment name, then you need to set the AZURE_OPENAI_MODEL_MAPPER parameter.

AZURE_OPENAI_MODEL_MAPPER (optional)

Default: Empty
Example: gpt-3.5-turbo=gpt-35-turbo means map gpt-3.5-turbo to gpt-35-turbo

If you are deploying ChatGPT using Azure OpenAI, it is recommended to set the AZURE_OPENAI_MODEL_MAPPER.
The session summarization feature relies on the gpt-3.5-turbo model, unless the name of your Azure deployment is the same as it.

Copy link
Contributor

@H0llyW00dzZ H0llyW00dzZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this default: "2023-04", must made separate for azure as example, because it will affected for openai and become bad hallucination. if force default: "2023-04", for all service provider

app/constant.ts Outdated
@@ -91,7 +91,7 @@ Latex block: $$e=mc^2$$
export const SUMMARIZE_MODEL = "gpt-3.5-turbo";

export const KnowledgeCutOffDate: Record<string, string> = {
default: "2021-09",
default: "2023-04",
Copy link
Contributor

@H0llyW00dzZ H0llyW00dzZ Nov 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this default: "2023-04", must made separate for azure as example, because it will affected for openai and become bad hallucination. if force default: "2023-04", for all service provider

@H0llyW00dzZ
Copy link
Contributor

Warning

The KnowledgeCutOffDate is intended for system prompts. Exercise caution when changing it, as it must be correctly aligned with the models.

@shendongming
Copy link
Author

Thank you
I have set KnowledgeCutOffDate to 2021-09
@H0llyW00dzZ

Copy link

vercel bot commented Mar 29, 2024

Someone is attempting to deploy a commit to the NextChat Team on Vercel.

A member of the Team first needs to authorize it.

Copy link
Contributor

Your build has completed!

Preview deployment

@lloydzhou lloydzhou requested a review from Dogtiti July 2, 2024 16:54
用来控制模型列表,使用 `+` 增加一个模型,使用 `-` 来隐藏一个模型,使用 `模型名=展示名` 来自定义模型的展示名,用英文逗号隔开。

### `AZURE_OPENAI_MODEL_MAPPER` (optional)
> Default: Empty
> Example: `gpt-3.5-turbo=gpt-35-turbo` 意思是 `gpt-3.5-turbo` 映射到 `gpt-35-turbo`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

看起来似乎是一个容易在配置文件中实现的方案 @Dogtiti

@lloydzhou lloydzhou added good first issue Good for newcomers planned planned feature, will support in the future labels Jul 3, 2024
> Example: `gpt-3.5-turbo=gpt-35-turbo` 意思是 `gpt-3.5-turbo` 映射到 `gpt-35-turbo`

如果你使用azure openai 来部署的chatgpt,建议设定AZURE_OPENAI_MODEL_MAPPER .
会话摘要功能依赖gpt-3.5-turbo 这个model,除非你的azure deployment的名字和它相同
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

可以直接使用CUSTOM_MODELS中配置gpt-4-1106-preview=gpt-4-turbo的方式,等于后面既作为展示名称,也作为deploy_name使用

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

你好,这个最新 readme 文档有了吗?使用 azure 部署的遇到一些麻烦。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers planned planned feature, will support in the future
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants