Skip to content

Commit

Permalink
Update falcon-180b.md (#1458)
Browse files Browse the repository at this point in the history
  • Loading branch information
philschmid authored Sep 6, 2023
1 parent 108cf0a commit bf61c76
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion falcon-180b.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,9 @@ You can easily try the Big Falcon Model (180 billion parameters!) in [this Space

### Hardware requirements

| | Type | Kind | Minimum Requirement | Example setup |
We ran several tests on the hardware needed to run the model for different use cases. Those are not the minimum numbers, but the minimum numbers for the configurations we had access to.

| | Type | Kind | Memory | Example |
| ----------- | --------- | ---------------- | ------------------- | --------------- |
| Falcon 180B | Training | Full fine-tuning | 5120GB | 8x 8x A100 80GB |
| Falcon 180B | Training | LoRA with ZeRO-3 | 1280GB | 2x 8x A100 80GB |
Expand Down

0 comments on commit bf61c76

Please sign in to comment.