Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

请问支持fastchat推理框架吗 #164

Open
fjchung opened this issue Jan 23, 2024 · 3 comments
Open

请问支持fastchat推理框架吗 #164

fjchung opened this issue Jan 23, 2024 · 3 comments

Comments

@fjchung
Copy link

fjchung commented Jan 23, 2024

使用fastchat框架推理,模型回答完问题之后,还会继续输出与问题无关的文字或者续写直到max length

@CberYellowstone
Copy link

CberYellowstone commented Jan 23, 2024

好像要做特殊适配,停止条件不一样
我自己写过一个很糟糕的适配尝试,勉强能用,你可以看看
https://github.com/CberYellowstone/Langchain-Chatchat

@i4never
Copy link
Contributor

i4never commented Jan 24, 2024

多数是停止条件的问题,框架没有识别到tokenizer.eos_token_id,需要针对具体框架调整。

@fjchung
Copy link
Author

fjchung commented Feb 1, 2024

好像要做特殊适配,停止条件不一样 我自己写过一个很糟糕的适配尝试,勉强能用,你可以看看 https://github.com/CberYellowstone/Langchain-Chatchat

大佬,代码有点多,能说下具体哪个层级目录下吗

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants