Fine tune Llama2 with Lora for foreign language

[konsalex]

Hey folks,

I watched a YouTube video, about how some LLMs tokenise languages other than English.

For example for the Greek language you will see that this is failing totally, as one character is one token always:

My question is, if I would fine-tune Llama2 with Lora based on Greek text, would the tokeniser change and work properly? Or the fine tune would not work as the tokeniser cannot be retrained/tuned?