New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[torch_xla] Performance of Llama 3.1 405B matches Huggingface fork on 1 pod #73

Open

tengyifei opened this issue Feb 1, 2025 · 1 comment

Assignees

Collaborator

tengyifei commented Feb 1, 2025

It's easier to bring up the model on 1 pod. We should do that and verify that the perf is as good as the recipe in https://github.com/AI-Hypercomputer/tpu-recipes/tree/main/training/trillium/Llama3-405B-PyTorch/GCE.

tengyifei self-assigned this

Collaborator Author

tengyifei commented Feb 1, 2025

I believe for this we just need #22.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment