convert : ignore tokens if their IDs are within [0, vocab_size)#3831

Merged

KerfuffleV2 merged 1 commit intomasterfrom apply-3585

Oct 28, 2023

+12-9

Commits on Oct 28, 2023

convert : ignore tokens if their IDs are within [0, vocab_size)
ggerganov
committed