Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Add Adam-CPU operator to support offload calculation #39005

Closed
ForFishes opened this issue Jan 17, 2022 · 1 comment
Closed

[Feature] Add Adam-CPU operator to support offload calculation #39005

ForFishes opened this issue Jan 17, 2022 · 1 comment
Assignees
Labels
Milestone

Comments

@ForFishes
Copy link
Member

ForFishes commented Jan 17, 2022

When training a large model and using the adam optimizer, the optimizer parameters/status are often offloaded to the cpu memory, and then used to update the parameters on the cpu. At present, the Adam-CPU implemented by Paddle has poor performance, and I hope to provide higher performance. The Adam-CPU implementation improves the overall computing speed.

reference: Section 5.1 in the paper
https://arxiv.org/pdf/2101.06840.pdf

@ForFishes ForFishes added this to the 2022 Q1 milestone Jan 17, 2022
@paddle-bot-old
Copy link

您好,我们已经收到了您的问题,会安排技术人员尽快解答您的问题,请耐心等待。请您再次检查是否提供了清晰的问题描述、复现代码、环境&版本、报错信息等。同时,您也可以通过查看官网API文档常见问题历史IssueAI社区来寻求解答。祝您生活愉快~

Hi! We've received your issue and please be patient to get responded. We will arrange technicians to answer your questions as soon as possible. Please make sure that you have posted enough message to demo your request. You may also check out the APIFAQGithub Issue and AI community to get the answer.Have a nice day!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants