Skip to content
This repository has been archived by the owner on Nov 1, 2024. It is now read-only.

Commit

Permalink
Docs
Browse files Browse the repository at this point in the history
  • Loading branch information
stephenroller committed May 4, 2022
1 parent 0118e3c commit 48b9b6c
Showing 1 changed file with 10 additions and 1 deletion.
11 changes: 10 additions & 1 deletion projects/OPT/download_opt175b.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
After receiving an email with a presigned URL to access the model weights, follow the below set of instructions to get started with hosting the model.

## Download all shards
Since we trained OPT-175B on 124 hosts, we have 124*8 = 992 files corresponding to the model parameters (8x tensor parallelism). The pre-signed URL that you receive in your email will look something like the following:
Since we trained OPT-175B on 124 hosts, we have 124\*8 = 992 files corresponding to the model parameters (8x tensor parallelism). The pre-signed URL that you receive in your email will look something like the following:

```
https://<cloudfront_url>/175b/checkpoint_last_20220503/stubbed.pt?&<super_long_query_string>
Expand All @@ -20,6 +20,15 @@ To consolidate the 992 shards into 8 files model-parallel evaluation, run (assum
bash metaseq/scripts/reshard_sbatch.sh <directory_where_all_the_shards_are>/checkpoint_last <output_dir>/ 8 1
```

Note that most of our models expect to run with Model (Tensor) Parallelism. For smaller models, some
users may find it easier to eliminate model parallelism. The checkpoints can be converted
to eliminate use of MP with the `consolidate_fsdp_shards.py` script:

```bash
python metaseq.scripts.consolidate_fsdp_shards ${FOLDER_PATH}/checkpoint_last --new-arch-name transformer_lm_gpt --save-prefix ${FOLDER_PATH}/consolidated
```


## Run the API
Follow the instructions in the [API docs](../../docs/api.md) to spin up the API. You will need to update the constants in `metaseq/service/constants.py` to point to right directories.

Expand Down

0 comments on commit 48b9b6c

Please sign in to comment.