Is posible that Full finetune Llama3-8B on 2xA100 (40GB Vram per GPU)? #1166
Answered
by
pbontrager
nguyenhoanganh2002
asked this question in
Q&A
-
|
Is posible that Full finetune Llama3-8B on 2xA100 (40GB Vram per GPU)? If so, how can i do it? |
Beta Was this translation helpful? Give feedback.
Answered by
pbontrager
Jul 12, 2024
Replies: 1 comment
-
|
I believe that you could even finetune it on only one A100 for the 8B model. But for two gpus, use the 8B_full config which is for distributed recipes. You can launch with |
Beta Was this translation helpful? Give feedback.
0 replies
Answer selected by
nguyenhoanganh2002
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I believe that you could even finetune it on only one A100 for the 8B model. But for two gpus, use the 8B_full config which is for distributed recipes. You can launch with
tune run --nnodes=1 --nproc-per-node=2 full_finetune_distributed --config llama3/8B_full. You can alsotune cp llama3/8B_full <my_config>to be able to modify the config for a batch size, learning rate, etc that it optimal for your compute setup.