[NPU] Add HcclBackend for 1-bit adam, 1-bit lamb, 0/1 adam#4733
[NPU] Add HcclBackend for 1-bit adam, 1-bit lamb, 0/1 adam#4733conglongli merged 10 commits intodeepspeedai:masterfrom
Conversation
|
@conglongli @awan-10 Would you please take a look? Thank you :) |
|
@tjruwase, hi, could you review this pr? |
|
@conglongli @awan-10 @tjruwase Could you please review this PR, cause we have some following PRs based on it. If I have done something wrong, please tell me, I will be happy to solve it. Thank you :) |
|
@RUAN-ZX Since we don't have the hardware, we will only review/approve based on whether your PR pass all our unit tests (it already failed the formatting test). And we would have to redirect all related future bug reports to you. |
Of course, we'll fix all the bugs with NPU. On the other hand, is it possible to add NPU unit tests to the workflow if we provide an NPU CI server? |
@CurryRice233 please contact @tjruwase for this kind of requests. |
Head branch was pushed to by a user without write access
|
@conglongli, could you review and merge a latest commit? Sorry for this inconvenience and thank you :) |
@RUAN-ZX In the future please avoid adding last minute commit which will break our auto-merge mechanism and keep delaying the PR merge. |
…ai#4733) To support NPU devices fulfilling feature requirements like 1-bit Adam, 1-bit Lamb, 0/1 Adam, I add HcclBackend and its corresponding import logics. See what we have already done in deepspeedai#4567 . --------- Co-authored-by: ryan <ruanzhixiang1@huawei.com> Co-authored-by: Conglong Li <conglong.li@gmail.com>
3 participants
To support NPU devices fulfilling feature requirements like 1-bit Adam, 1-bit Lamb, 0/1 Adam, I add HcclBackend and its corresponding import logics.
See what we have already done in #4567 .