[Feature]: add DoRA support

[cmhungsteve]

🚀 The feature, motivation and pitch

The recent ICML'24 Oral paper, DoRA, has shown consistent improvement over LoRA on various tasks (LLM, MLLM, etc.) and backbones (LLaMA, LLaVA, etc.)
[Code] https://github.com/NVlabs/DoRA

DoRA has also been integrated into various open-source library/framework:

• Hugging Face: peft, diffusers
• Apple: MLX
• Meta: torchtune
• NVIDIA: NeMo
• Unsloth AI: unsloth
• Answer.AI: QDoRA + FSDP
• LLaMA-Factory
• LyCORIS
• ..., etc.

Alternatives

No response

Additional context

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[jeejeelee]

Thank you very much for bringing this feature. We will consider supporting it.

[TC10127]

I also need vllm to support dora !!!

[PacinoAn]

We need this feature too，this will help a lot!

[winglian]

@jeejeelee any updates on whether this is on the roadmap?

[cmhungsteve]

@jeejeelee
https://x.com/winglian/status/1888951180606202028
GRPO + DoRA converges faster than GRPO+ FFT or GRPO + LoRA (thanks @winglian for the great finding!)

[jeejeelee]

@jeejeelee https://x.com/winglian/status/1888951180606202028 GRPO + DoRA converges faster than GRPO+ FFT or GRPO + LoRA (thanks @winglian for the great finding!)

Thanks, will start trying to support DoRA soon