Skip to content

Commit

Permalink
Update falcon-180b.md (huggingface#1458)
Browse files Browse the repository at this point in the history
  • Loading branch information
philschmid authored and kashif committed Sep 29, 2023
1 parent 6ec2422 commit 76e0a6b
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion falcon-180b.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,9 @@ You can easily try the Big Falcon Model (180 billion parameters!) in [this Space

### Hardware requirements

| | Type | Kind | Minimum Requirement | Example setup |
We ran several tests on the hardware needed to run the model for different use cases. Those are not the minimum numbers, but the minimum numbers for the configurations we had access to.

| | Type | Kind | Memory | Example |
| ----------- | --------- | ---------------- | ------------------- | --------------- |
| Falcon 180B | Training | Full fine-tuning | 5120GB | 8x 8x A100 80GB |
| Falcon 180B | Training | LoRA with ZeRO-3 | 1280GB | 2x 8x A100 80GB |
Expand Down

0 comments on commit 76e0a6b

Please sign in to comment.