Question: What's the recommended way to finetune newish models?

[0xDEADFED5]

I want to finetune Qwen 2.5 3B with Intel GPU.

Can't do it with the IPEX axolotl.

What's currently the best way to do this?

[qiyuangong]

I want to finetune Qwen 2.5 3B with Intel GPU.

Can't do it with the IPEX axolotl.

What's currently the best way to do this?

It seems axolotl begins to support Qwen 2 after V0.5.0. However, ipex-llm only supports axolotl v0.4.0 right now.

That means Qwen 2.5 3B is not supported right now. Can you try other models supported in 0.4.0 ?

If you find a candidate model in 0.4.0. You can use Linux OS + docker container and this guide to fine-tune models.

[0xDEADFED5]

it doesn't have to be axolotl, i want to know what will work right now.

will torchtune work? or transformers trainer with newish transformers version?

[qiyuangong]

it doesn't have to be axolotl, i want to know what will work right now.

will torchtune work? or transformers trainer with newish transformers version?

Torchtune is not supported yet. Also transformers trainer is not recommended. You can use Peft for finetuning.

You can check Running LLM Finetuning using IPEX-LLM on Intel GPU for all supported frameworks and examples.

[0xDEADFED5]

sorry for late response, closing this now. Thanks @qiyuangong for the advice, i didn't know that folder was in the repo