What should the high confidence score of tokens be set to ?

[greens007]

There are not any details about it in the paper. Look forward to your reply.

[buxiangzhiren]

For each token, the output of model is a set which cosists of probabilities of tokens in the vocabulary. We denote the max probabilities as the highest confidence score.

[greens007]

For each token, the output of model is a set which cosists of probabilities of tokens in the vocabulary. We denote the max probabilities as the highest confidence score.

Thanks for your immediate reply!