Does the tokenizer support Chinese?

[zhanxlin]

Hello, does the tokenizer support Chinese?

[jongwook]

Hello!

The tokenizer is based on byte-pair encoding and is able to tokenize any valid UTF-8 string, but it is not recommended to use this tokenizer for non-English sentences since its vocabulary is primarily based on English. Furthermore, CLIP models are trained with mostly English data, so we don't expect particularly good performance from CLIP with non-English text inputs. Multilingual CLIP would be an interesting future work.

[ruby0101]

Hi, @jongwook, "CLIP models are trained with mostly English data". So can it be re-trained with a new image text pair dataset in other languages?

[jongwook]

Yes; I presume it'll need to train from scratch (or at least the text encoder) in that case.

[datalee]

Multilingual CLIP 666

[yangapku]

Hi, maybe you can refer to this repo! https://github.com/OFA-Sys/Chinese-CLIP @zhanxlin