Skip to content

Commit 3cd65b5

Browse files
awaelchlikaushikb11pre-commit-ci[bot]justusschockSeanNaren
authored
Lightning Lite Examples (#9987)
Co-authored-by: Kaushik B <kaushikbokka@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com> Co-authored-by: SeanNaren <sean@grid.ai> Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com> Co-authored-by: thomas chaton <thomas@grid.ai> Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com> Co-authored-by: four4fish <88516121+four4fish@users.noreply.github.com> Co-authored-by: Nicki Skafte Detlefsen <skaftenicki@gmail.com> Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com> Co-authored-by: Pietro Lesci <61748653+pietrolesci@users.noreply.github.com>
1 parent e4ee6df commit 3cd65b5

24 files changed

+1016
-168
lines changed

‎.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -137,6 +137,9 @@ ENV/
137137
Datasets/
138138
mnist/
139139
legacy/checkpoints/
140+
*.gz
141+
*ubyte
142+
140143

141144
# pl tests
142145
ml-runs/

‎CHANGELOG.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -119,6 +119,8 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
119119
* Updated precision attributes in `DeepSpeedPlugin` ([#10164](https://github.com/PyTorchLightning/pytorch-lightning/pull/10164))
120120
* Added the ability to return a result from rank 0 in `DDPSpawnPlugin.spawn` ([#10162](https://github.com/PyTorchLightning/pytorch-lightning/pull/10162))
121121
* Added `pytorch_lightning.lite` package ([#10175](https://github.com/PyTorchLightning/pytorch-lightning/pull/10175))
122+
* Added `LightningLite` documentation ([#10043](https://github.com/PyTorchLightning/pytorch-lightning/pull/10043))
123+
* Added `LightningLite` examples ([#9987](https://github.com/PyTorchLightning/pytorch-lightning/pull/9987))
122124
* Make the `_LiteDataLoader` an iterator and add supports for custom dataloader ([#10279](https://github.com/PyTorchLightning/pytorch-lightning/pull/10279))
123125
- Added `use_omegaconf` argument to `save_hparams_to_yaml` plugin ([#9170](https://github.com/PyTorchLightning/pytorch-lightning/pull/9170))
124126
- Added `ckpt_path` argument for `Trainer.fit()` ([#10061](https://github.com/PyTorchLightning/pytorch-lightning/pull/10061))

‎docs/source/conf.py

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@
1616
import os
1717
import shutil
1818
import sys
19+
import warnings
1920
from importlib.util import module_from_spec, spec_from_file_location
2021

2122
import pt_lightning_sphinx_theme
@@ -26,10 +27,13 @@
2627
sys.path.insert(0, os.path.abspath(PATH_ROOT))
2728
sys.path.append(os.path.join(PATH_RAW_NB, ".actions"))
2829

30+
_SHOULD_COPY_NOTEBOOKS = True
31+
2932
try:
3033
from helpers import HelperCLI
3134
except Exception:
32-
raise ModuleNotFoundError("To build the code, please run: `git submodule update --init --recursive`")
35+
_SHOULD_COPY_NOTEBOOKS = False
36+
warnings.warn("To build the code, please run: `git submodule update --init --recursive`", stacklevel=2)
3337

3438
FOLDER_GENERATED = "generated"
3539
SPHINX_MOCK_REQUIREMENTS = int(os.environ.get("SPHINX_MOCK_REQUIREMENTS", True))
@@ -41,8 +45,8 @@
4145
spec.loader.exec_module(about)
4246

4347
# -- Project documents -------------------------------------------------------
44-
45-
HelperCLI.copy_notebooks(PATH_RAW_NB, PATH_HERE, "notebooks")
48+
if _SHOULD_COPY_NOTEBOOKS:
49+
HelperCLI.copy_notebooks(PATH_RAW_NB, PATH_HERE, "notebooks")
4650

4751

4852
def _transform_changelog(path_in: str, path_out: str) -> None:

‎pl_examples/README.md

Lines changed: 20 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -5,20 +5,31 @@ can be found in our sister library [Lightning Bolts](https://pytorch-lightning.r
55

66
______________________________________________________________________
77

8-
## Basic examples
8+
## MNIST Examples
99

10-
In this folder we add several starter examples:
10+
5 MNIST examples showing how to gradually convert from pure PyTorch to PyTorch Lightning.
1111

12-
- [MNIST Classifier](./basic_examples/simple_image_classifier.py): Shows how to define the model inside the `LightningModule`.
13-
- [Image Classifier](./basic_examples/backbone_image_classifier.py): Trains arbitrary datasets with arbitrary backbones.
14-
- [Autoencoder](./basic_examples/autoencoder.py): Shows how the `LightningModule` can be used as a system.
15-
- [Profiler](./basic_examples/profiler_example.py): Shows the basic usage of the PyTorch profilers and how to inspect traces in Google Chrome.
16-
- [Image Classifier with DALI](./basic_examples/dali_image_classifier.py): Shows how to use [NVIDIA DALI](https://developer.nvidia.com/DALI) with Lightning.
17-
- [Mnist Datamodule](.basic_examples/mnist_datamodule.py): Shows how to define a simple `LightningDataModule` using the MNIST dataset.
12+
The transition through [LightningLite](https://pytorch-lightning.readthedocs.io/en/latest/starter/lightning_lite.html) from pure PyTorch is optional but it might be helpful to learn about it.
13+
14+
- [MNIST with vanilla PyTorch](./basic_examples/mnist_examples/image_classifier_1_pytorch.py)
15+
- [MNIST with LightningLite](./basic_examples/mnist_examples/image_classifier_2_lite.py)
16+
- [MNIST LightningLite to LightningModule](./basic_examples/mnist_examples/image_classifier_3_lite_to_lightning_module.py)
17+
- [MNIST with LightningModule](./basic_examples/mnist_examples/image_classifier_4_lightning_module.py)
18+
- [MNIST with LightningModule + LightningDataModule](./basic_examples/mnist_examples/image_classifier_5_lightning_datamodule.py)
19+
20+
______________________________________________________________________
21+
22+
## Basic Examples
23+
24+
In this folder, we have 2 simple examples:
25+
26+
- [Image Classifier](./basic_examples/backbone_image_classifier.py) (trains arbitrary datasets with arbitrary backbones).
27+
- [Image Classifier + DALI](./basic_examples/mnist_examples/image_classifier_4_dali.py) (defines the model inside the `LightningModule`).
28+
- [Autoencoder](./basic_examples/autoencoder.py) (shows how the `LightningModule` can be used as a system)
1829

1930
______________________________________________________________________
2031

21-
## Domain examples
32+
## Domain Examples
2233

2334
This folder contains older examples. You should instead use the examples
2435
in [Lightning Bolts](https://pytorch-lightning.readthedocs.io/en/latest/ecosystem/bolts.html)

‎pl_examples/basic_examples/README.md

Lines changed: 37 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -2,62 +2,70 @@
22

33
Use these examples to test how Lightning works.
44

5-
#### MNIST
5+
## MNIST Examples
66

7-
Trains MNIST where the model is defined inside the `LightningModule`.
7+
Here are 5 MNIST examples showing you how to gradually convert from pure PyTorch to PyTorch Lightning.
88

9-
```bash
10-
# cpu
11-
python simple_image_classifier.py
9+
The transition through [LightningLite](https://pytorch-lightning.readthedocs.io/en/stable/starter/lightning_lite.rst) from pure PyTorch is optional but it might be helpful to learn about it.
10+
11+
#### 1. Image Classifier with Vanilla PyTorch
1212

13-
# gpus (any number)
14-
python simple_image_classifier.py --trainer.gpus 2
13+
Trains a simple CNN over MNIST using vanilla PyTorch.
1514

16-
# Distributed Data Parallel
17-
python simple_image_classifier.py --trainer.gpus 2 --trainer.accelerator ddp
15+
```bash
16+
# CPU
17+
python image_classifier_1_pytorch.py
1818
```
1919

2020
______________________________________________________________________
2121

22-
#### MNIST with DALI
22+
#### 2. Image Classifier with LightningLite
2323

24-
The MNIST example above using [NVIDIA DALI](https://developer.nvidia.com/DALI).
25-
Requires NVIDIA DALI to be installed based on your CUDA version, see [here](https://docs.nvidia.com/deeplearning/dali/user-guide/docs/installation.html).
24+
This script shows you how to scale the previous script to enable GPU and multi-GPU training using [LightningLite](https://pytorch-lightning.readthedocs.io/en/stable/starter/lightning_lite.html).
2625

2726
```bash
28-
python dali_image_classifier.py
27+
# CPU / multiple GPUs if available
28+
python image_classifier_2_lite.py
2929
```
3030

3131
______________________________________________________________________
3232

33-
#### Image classifier
33+
#### 3. Image Classifier - Conversion from Lite to Lightning
3434

35-
Generic image classifier with an arbitrary backbone (ie: a simple system)
35+
This script shows you how to prepare your conversion from [LightningLite](https://pytorch-lightning.readthedocs.io/en/stable/starter/lightning_lite.html) to `LightningModule`.
3636

3737
```bash
38-
# cpu
39-
python backbone_image_classifier.py
38+
# CPU / multiple GPUs if available
39+
python image_classifier_3_lite_to_lightning_module.py
40+
```
4041

41-
# gpus (any number)
42-
python backbone_image_classifier.py --trainer.gpus 2
42+
______________________________________________________________________
43+
44+
#### 4. Image Classifier with LightningModule
45+
46+
This script shows you the result of the conversion to the `LightningModule` and finally all the benefits you get from the Lightning ecosystem.
47+
48+
```bash
49+
# CPU
50+
python image_classifier_4_lightning_module.py
4351

44-
# Distributed Data Parallel
45-
python backbone_image_classifier.py --trainer.gpus 2 --trainer.accelerator ddp
52+
# GPUs (any number)
53+
python image_classifier_4_lightning_module.py --trainer.gpus 2
4654
```
4755

4856
______________________________________________________________________
4957

50-
#### Autoencoder
58+
#### 5. Image Classifier with LightningModule and LightningDataModule
5159

52-
Showing the power of a system... arbitrarily complex training loops
60+
This script shows you how to extract the data related components into a `LightningDataModule`.
5361

5462
```bash
55-
# cpu
56-
python autoencoder.py
63+
# CPU
64+
python image_classifier_5_lightning_datamodule.py
5765

58-
# gpus (any number)
59-
python autoencoder.py --trainer.gpus 2
66+
# GPUs (any number)
67+
python image_classifier_5_lightning_datamodule.py --trainer.gpus 2
6068

61-
# Distributed Data Parallel
62-
python autoencoder.py --trainer.gpus 2 --trainer.accelerator ddp
69+
# Distributed Data Parallel (DDP)
70+
python image_classifier_5_lightning_datamodule.py --trainer.gpus 2 --trainer.strategy 'ddp'
6371
```

‎pl_examples/basic_examples/mnist_examples/README.md

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
## MNIST Examples
2+
3+
Here are 5 MNIST examples showing you how to gradually convert from pure PyTorch to PyTorch Lightning.
4+
5+
The transition through [LightningLite](https://pytorch-lightning.readthedocs.io/en/latest/stable/lightning_lite.rst) from pure PyTorch is optional but it might be helpful to learn about it.
6+
7+
#### 1. Image Classifier with Vanilla PyTorch
8+
9+
Trains a simple CNN over MNIST using vanilla PyTorch.
10+
11+
```bash
12+
# CPU
13+
python image_classifier_1_pytorch.py
14+
```
15+
16+
______________________________________________________________________
17+
18+
#### 2. Image Classifier with LightningLite
19+
20+
This script shows you how to scale the previous script to enable GPU and multi-GPU training using [LightningLite](https://pytorch-lightning.readthedocs.io/en/stable/starter/lightning_lite.html).
21+
22+
```bash
23+
# CPU / multiple GPUs if available
24+
python image_classifier_2_lite.py
25+
```
26+
27+
______________________________________________________________________
28+
29+
#### 3. Image Classifier - Conversion from Lite to Lightning
30+
31+
This script shows you how to prepare your conversion from [LightningLite](https://pytorch-lightning.readthedocs.io/en/stable/starter/lightning_lite.html) to `LightningModule`.
32+
33+
```bash
34+
# CPU / multiple GPUs if available
35+
python image_classifier_3_lite_to_lightning_module.py
36+
```
37+
38+
______________________________________________________________________
39+
40+
#### 4. Image Classifier with LightningModule
41+
42+
This script shows you the result of the conversion to the `LightningModule` and finally all the benefits you get from Lightning.
43+
44+
```bash
45+
# CPU
46+
python image_classifier_4_lightning_module.py
47+
48+
# GPUs (any number)
49+
python image_classifier_4_lightning_module.py --trainer.gpus 2
50+
```
51+
52+
______________________________________________________________________
53+
54+
#### 5. Image Classifier with LightningModule and LightningDataModule
55+
56+
This script shows you how to extract the data related components into a `LightningDataModule`.
57+
58+
```bash
59+
# CPU
60+
python image_classifier_5_lightning_datamodule.py
61+
62+
# GPUs (any number)
63+
python image_classifier_5_lightning_datamodule.py --trainer.gpus 2
64+
65+
# Distributed Data parallel
66+
python image_classifier_5_lightning_datamodule.py --trainer.gpus 2 --trainer.strategy 'ddp'
67+
```

‎pl_examples/basic_examples/mnist_examples/__init__.py

Whitespace-only changes.

0 commit comments

Comments
 (0)