Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

set the lr dtype when create lr var in optimizer #44840

Merged
merged 8 commits into from
Aug 4, 2022
Merged

set the lr dtype when create lr var in optimizer #44840

merged 8 commits into from
Aug 4, 2022

Conversation

FeixLiu
Copy link
Contributor

@FeixLiu FeixLiu commented Aug 3, 2022 •
edited
Loading

PR types

Others

PR changes

Others

Describe

set the lr dtype to float32 if the default dtype is float16

sneaxiy
sneaxiy previously approved these changes Aug 3, 2022
View reviewed changes
Copy link
Collaborator

@sneaxiy sneaxiy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

ForFishes
ForFishes previously approved these changes Aug 3, 2022
View reviewed changes
sneaxiy
sneaxiy previously approved these changes Aug 3, 2022
View reviewed changes
Copy link
Collaborator

@sneaxiy sneaxiy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@FeixLiu FeixLiu changed the title cast variable LR to float 32 for adamw, test=allcases set the lr dtype when create lr var in optimizer Aug 3, 2022
@FeixLiu FeixLiu requested a review from zhiqiu August 3, 2022 09:14
zhiqiu
zhiqiu previously approved these changes Aug 3, 2022
View reviewed changes
Copy link
Contributor

@zhiqiu zhiqiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@FeixLiu FeixLiu dismissed stale reviews from zhiqiu and sneaxiy via ae6b7ca August 3, 2022 10:15
ForFishes
ForFishes approved these changes Aug 3, 2022
View reviewed changes
Copy link
Member

@ForFishes ForFishes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@FeixLiu FeixLiu merged commit 9e39d74 into PaddlePaddle:develop Aug 4, 2022
@FeixLiu FeixLiu deleted the adamw_fix_lr branch August 4, 2022 00:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zhiqiu zhiqiu zhiqiu approved these changes

@ForFishes ForFishes ForFishes approved these changes

@sneaxiy sneaxiy sneaxiy approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@FeixLiu @zhiqiu @ForFishes @sneaxiy