Non-english languages?

[niansa]

Hey!

I am attempting to use llama to work with german, however the results are more than poor. Do you have any tips?

Thanks
Niansa / Tuxifan

[gjmulder]

Out of scope for llama.cpp, but fine tune with some German language data?

You might want to check out Dutch T5 models : UL2, T5, ByT5 and Long-T5

[niansa]

Alright, thanks!

[DominguesM]

I trained the Alpaca model for Brazilian Portuguese. I made available the scripts for translating the original dataset, maybe I can help you.

Alpaca LoRa 7B - PTBR

[x4080]

@DominguesM In your experience, just fine tune using data (instruction, input, response) that use another language is sufficient to make the model answer for that language ?

Do you need to add like "translate to Portugese" ... ?

[DominguesM]

@x4080 TL;DR: Yes, it was satisfactory for me.

After conducting several experiments, I developed two models: one trained with 52k instructions and the other with 300k instructions, all in Portuguese. Simply altering the language of the training data was sufficient for the model to exhibit improved performance in that specific language.

I also conducted additional experiments by introducing new information, such as "Who is the president of Brazil?" In the original version of the Llama model, as well as in my model trained with 52k instructions (translated from the Alpaca dataset), the model provided incorrect answers. However, after incorporating this specific knowledge, the model exhibited enhanced performance in generating accurate responses related to the "president of Brazil" query.

[x4080]

Thanks @DominguesM