Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Delete old recipe #338

Merged
merged 4 commits into from
Feb 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 7 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,12 +126,12 @@ tune convert_checkpoint --checkpoint-path <CHECKPOINT_PATH>

On a single GPU
```
tune finetune_llm --config alpaca_llama2_finetune
tune --nnodes 1 --nproc_per_node 1 full_finetune --config alpaca_llama2_full_finetune
```

On multiple GPUs using FSDP
```
tune --nnodes 1 --nproc_per_node 4 finetune_llm --config alpaca_llama2_finetune --fsdp True
tune --nnodes 1 --nproc_per_node 4 full_finetune --config alpaca_llama2_full_finetune
```

&nbsp;
Expand All @@ -140,9 +140,9 @@ tune --nnodes 1 --nproc_per_node 4 finetune_llm --config alpaca_llama2_finetune

To copy a recipe to customize it yourself and then run
```
tune recipe cp finetune_llm my_recipe/finetune_llm.py
tune config cp alpaca_llama2_finetune my_recipe/alpaca_llama2_finetune.yaml
tune my_recipe/finetune_llm.py --config my_recipe/alpaca_llama2_finetune.yaml
tune recipe cp full_finetune my_recipe/full_finetune.py
tune config cp alpaca_llama2_full_finetune my_recipe/alpaca_llama2_full_finetune.yaml
tune my_recipe/full_finetune.py --config my_recipe/alpaca_llama2_full_finetune.yaml
```

&nbsp;
Expand All @@ -154,15 +154,11 @@ recipes. Aside from torchtune recipe utilties, it integrates with ``torch.distri
to support distributed job launching by default. ``tune`` offers everyting that ``torchrun``
does with the following additional functionalities:

1. ``tune <recipe> <recipe_args>`` with no optional ``torchrun`` options launches a single python process
1. ``tune <torchrun_options> <recipe> <recipe_args>`` will launch a torchrun job

2. ``<recipe>`` and recipe arg ``<config>`` can both be passed in as names instead of paths if they're included in torchtune

3. ``tune <path/to/recipe.py> <recipe_args>`` can be used to launch local recipes

4. ``tune <torchrun_options> <recipe> <recipe_args>`` will launch a torchrun job

5. ``tune recipe`` and ``tune config`` commands provide utilities for listing and copying packaged recipes and configs
3. ``tune recipe`` and ``tune config`` commands provide utilities for listing and copying packaged recipes and configs

&nbsp;

Expand Down
22 changes: 0 additions & 22 deletions docs/source/recipes/finetune_llm.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,25 +16,3 @@ This recipe supports:
* :ref:`Distributed Training with FSDP<dist_label>`

* :ref:`Activation Checkpointing<ac_label>`

To run the recipe directly, launch with

.. code-block:: bash

tune finetune_llm --config <finetune_config>

Recipe
------

Copy the recipe directly into your own script or notebook to modify and edit for yourself.

.. literalinclude:: ../../../recipes/finetune_llm.py

Configs
-------

.. tabs::

.. tab:: alpaca_llama2_finetune

.. literalinclude:: ../../../recipes/configs/alpaca_llama2_finetune.yaml
7 changes: 5 additions & 2 deletions recipes/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,11 @@
# LICENSE file in the root directory of this source tree.


_RECIPE_LIST = ["finetune_llm", "alpaca_generate"]
_CONFIG_LISTS = {"finetune_llm": ["alpaca_llama2_finetune"], "alpaca_generate": []}
_RECIPE_LIST = ["full_finetune", "alpaca_generate"]
_CONFIG_LISTS = {
"full_finetune": ["alpaca_llama2_full_finetune"],
"alpaca_generate": [],
}


def list_recipes():
Expand Down
32 changes: 0 additions & 32 deletions recipes/configs/alpaca_llama2_finetune.yaml

This file was deleted.

229 changes: 0 additions & 229 deletions recipes/finetune_llm.py

This file was deleted.

Loading
Loading