Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

set fp16 to false if bf16, update bf16: auto in example YAMLs #1122

Merged
merged 4 commits into from
Jan 22, 2024

Conversation

winglian
Copy link
Collaborator

#1116 enables bf16: auto support, which should simplify whether an end user needs to think about using bf16 for their particular GPU. This updates all the example yaml files to use this.

Copy link
Collaborator

@NanoCode012 NanoCode012 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool! I'm a bit conflicted on the naming choice of using bf16: auto instead of some autocast: true, but it works

README.md Outdated Show resolved Hide resolved
@winglian
Copy link
Collaborator Author

@NanoCode012 perhaps we can discuss having something like autocast: fp8,bf16,fp16 which would attempt to see if the gpu supports any of those in that order and fallback to fp32, whereas autocast: bf16 would only check for bf16 and skip fp16 for fp32?
For now, let's keep this and have a discussion about the above proposal?

Co-authored-by: NanoCode012 <kevinvong@rocketmail.com>
@winglian winglian merged commit 782b6a4 into main Jan 22, 2024
7 checks passed
@winglian winglian deleted the auto-bf16-templates branch January 22, 2024 23:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants