-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Interface with Hugging Face Accelerate for distributed training #11
Comments
Optional parameters:
|
Note: I think you will have to make heavy edits to get it to interface with the metrics properly (see this function). Also, true for model serialization. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Create a new
distributed_train
function intorchplate.experiment.Experiment
which interfaces with Hugging Face Accelerate for zero-overhead distributed training of PyTorch models. Avoid.to(device)
placements as the accelerate library will handle this for you. Can call this function even with one GPU.The text was updated successfully, but these errors were encountered: