Support Ascend NPU adapter loading

[ji-huazhong]

What does this PR do?

• This PR introduces Lora and related adapter methods into npu since Accelerate has already provided support for npu.
• Btw thanks to abhilash1910 for his outstanding contribution, it is more convenient to support third-party accelerators.

[ji-huazhong]

@BenjaminBossan @younesbelkada could you please review this PR? thanks🤗

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[younesbelkada]

Thanks for adding this!

[BenjaminBossan]

LGTM, thx.

Just a more general question: Do we have any kind of test to show that NPU (or XPU for that matter) actually work?

[ji-huazhong]

LGTM, thx.

Just a more general question: Do we have any kind of test to show that NPU (or XPU for that matter) actually work?

@BenjaminBossan Thanks for your reply. I verified the effectiveness of this PR at LLaMA-Efficient-Tuning, which integrates peft.

If necessary, I could include additional test cases for third-party accelerators like NPU/XPU in a separate PR. This would be done under the assumption that these accelerators have cuda-like APIs within the PyTorch ecosystem.

[BenjaminBossan]

I verified the effectiveness of this PR at LLaMA-Efficient-Tuning, which integrates peft.

Very nice, thanks for the pointer.

If necessary, I could include additional test cases for third-party accelerators like NPU/XPU in a separate PR. This would be done under the assumption that these accelerators have cuda-like APIs within the PyTorch ecosystem.

For me it was just important to ensure that someone has actually tested it. I don't know if we need these additional tests. On the one hand, it would be nice to have those to catch regressions, on the other hand they're probably not easy to integrate. I'll leave it up to the other maintainers to decide.

[pacman100]

Thank you @statelesshz for adding NPU support, LGTM! 🚀