New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

eos #265

Open

irasin opened this issue Jan 30, 2023 · 1 comment

irasin commented Jan 30, 2023 •

edited

Loading

你好，请问一下如何判断句子的生成已经达到结尾了呢？gpt2的英语模型中，tokenizer提供了eos_token，可以通过该token判断是否停止。但是这里是用Bert的tokenizer，没有eos_token，该如何判断停止条件呢？

lizhipengpeng commented Sep 23, 2023

同问，在使用过程中也遇到同样的问题

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment