Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
102 changes: 51 additions & 51 deletions ignite/engine/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,17 +53,17 @@ def supervised_training_step(
"""Factory function for supervised training.

Args:
model (torch.nn.Module): the model to train.
optimizer (torch.optim.Optimizer): the optimizer to use.
loss_fn (torch.nn loss function): the loss function to use.
device (str, optional): device type specification (default: None).
model: the model to train.
optimizer: the optimizer to use.
loss_fn: the loss function to use.
device: device type specification (default: None).
Applies to batches after starting the engine. Model *will not* be moved.
Device can be CPU, GPU.
non_blocking (bool, optional): if True and this copy is between CPU and GPU, the copy may occur asynchronously
non_blocking: if True and this copy is between CPU and GPU, the copy may occur asynchronously
with respect to the host. For other cases, this argument has no effect.
prepare_batch (callable, optional): function that receives `batch`, `device`, `non_blocking` and outputs
prepare_batch: function that receives `batch`, `device`, `non_blocking` and outputs
tuple of tensors `(batch_x, batch_y)`.
output_transform (callable, optional): function that receives 'x', 'y', 'y_pred', 'loss' and returns value
output_transform: function that receives 'x', 'y', 'y_pred', 'loss' and returns value
to be assigned to engine's state.output after each iteration. Default is returning `loss.item()`.

Returns:
Expand Down Expand Up @@ -109,19 +109,19 @@ def supervised_training_step_amp(
"""Factory function for supervised training using ``torch.cuda.amp``.

Args:
model (torch.nn.Module): the model to train.
optimizer (torch.optim.Optimizer): the optimizer to use.
loss_fn (torch.nn loss function): the loss function to use.
device (str, optional): device type specification (default: None).
model: the model to train.
optimizer: the optimizer to use.
loss_fn: the loss function to use.
device: device type specification (default: None).
Applies to batches after starting the engine. Model *will not* be moved.
Device can be CPU, GPU.
non_blocking (bool, optional): if True and this copy is between CPU and GPU, the copy may occur asynchronously
non_blocking: if True and this copy is between CPU and GPU, the copy may occur asynchronously
with respect to the host. For other cases, this argument has no effect.
prepare_batch (callable, optional): function that receives `batch`, `device`, `non_blocking` and outputs
prepare_batch: function that receives `batch`, `device`, `non_blocking` and outputs
tuple of tensors `(batch_x, batch_y)`.
output_transform (callable, optional): function that receives 'x', 'y', 'y_pred', 'loss' and returns value
output_transform: function that receives 'x', 'y', 'y_pred', 'loss' and returns value
to be assigned to engine's state.output after each iteration. Default is returning `loss.item()`.
scaler (torch.cuda.amp.GradScaler, optional): GradScaler instance for gradient scaling. (default: None)
scaler: GradScaler instance for gradient scaling. (default: None)

Returns:
Callable: update function
Expand Down Expand Up @@ -177,17 +177,17 @@ def supervised_training_step_apex(
"""Factory function for supervised training using apex.

Args:
model (torch.nn.Module): the model to train.
optimizer (torch.optim.Optimizer): the optimizer to use.
loss_fn (torch.nn loss function): the loss function to use.
device (str, optional): device type specification (default: None).
model: the model to train.
optimizer: the optimizer to use.
loss_fn: the loss function to use.
device: device type specification (default: None).
Applies to batches after starting the engine. Model *will not* be moved.
Device can be CPU, GPU.
non_blocking (bool, optional): if True and this copy is between CPU and GPU, the copy may occur asynchronously
non_blocking: if True and this copy is between CPU and GPU, the copy may occur asynchronously
with respect to the host. For other cases, this argument has no effect.
prepare_batch (callable, optional): function that receives `batch`, `device`, `non_blocking` and outputs
prepare_batch: function that receives `batch`, `device`, `non_blocking` and outputs
tuple of tensors `(batch_x, batch_y)`.
output_transform (callable, optional): function that receives 'x', 'y', 'y_pred', 'loss' and returns value
output_transform: function that receives 'x', 'y', 'y_pred', 'loss' and returns value
to be assigned to engine's state.output after each iteration. Default is returning `loss.item()`.

Returns:
Expand Down Expand Up @@ -238,17 +238,17 @@ def supervised_training_step_tpu(
"""Factory function for supervised training using ``torch_xla``.

Args:
model (torch.nn.Module): the model to train.
optimizer (torch.optim.Optimizer): the optimizer to use.
loss_fn (torch.nn loss function): the loss function to use.
device (str, optional): device type specification (default: None).
model: the model to train.
optimizer: the optimizer to use.
loss_fn: the loss function to use.
device: device type specification (default: None).
Applies to batches after starting the engine. Model *will not* be moved.
Device can be CPU, TPU.
non_blocking (bool, optional): if True and this copy is between CPU and GPU, the copy may occur asynchronously
non_blocking: if True and this copy is between CPU and GPU, the copy may occur asynchronously
with respect to the host. For other cases, this argument has no effect.
prepare_batch (callable, optional): function that receives `batch`, `device`, `non_blocking` and outputs
prepare_batch: function that receives `batch`, `device`, `non_blocking` and outputs
tuple of tensors `(batch_x, batch_y)`.
output_transform (callable, optional): function that receives 'x', 'y', 'y_pred', 'loss' and returns value
output_transform: function that receives 'x', 'y', 'y_pred', 'loss' and returns value
to be assigned to engine's state.output after each iteration. Default is returning `loss.item()`.

Returns:
Expand Down Expand Up @@ -328,29 +328,32 @@ def create_supervised_trainer(
"""Factory function for creating a trainer for supervised models.

Args:
model (torch.nn.Module): the model to train.
optimizer (torch.optim.Optimizer): the optimizer to use.
loss_fn (torch.nn loss function): the loss function to use.
device (str, optional): device type specification (default: None).
model: the model to train.
optimizer: the optimizer to use.
loss_fn: the loss function to use.
device: device type specification (default: None).
Applies to batches after starting the engine. Model *will not* be moved.
Device can be CPU, GPU or TPU.
non_blocking (bool, optional): if True and this copy is between CPU and GPU, the copy may occur asynchronously
non_blocking: if True and this copy is between CPU and GPU, the copy may occur asynchronously
with respect to the host. For other cases, this argument has no effect.
prepare_batch (callable, optional): function that receives `batch`, `device`, `non_blocking` and outputs
prepare_batch: function that receives `batch`, `device`, `non_blocking` and outputs
tuple of tensors `(batch_x, batch_y)`.
output_transform (callable, optional): function that receives 'x', 'y', 'y_pred', 'loss' and returns value
output_transform: function that receives 'x', 'y', 'y_pred', 'loss' and returns value
to be assigned to engine's state.output after each iteration. Default is returning `loss.item()`.
deterministic (bool, optional): if True, returns deterministic engine of type
deterministic: if True, returns deterministic engine of type
:class:`~ignite.engine.deterministic.DeterministicEngine`, otherwise :class:`~ignite.engine.engine.Engine`
(default: False).
amp_mode (str, optional): can be ``amp`` or ``apex``, model and optimizer will be casted to float16 using
amp_mode: can be ``amp`` or ``apex``, model and optimizer will be casted to float16 using
`torch.cuda.amp <https://pytorch.org/docs/stable/amp.html>`_ for ``amp`` and
using `apex <https://nvidia.github.io/apex>`_ for ``apex``. (default: None)
scaler (torch.cuda.amp.GradScaler, bool, optional): GradScaler instance for gradient scaling if `torch>=1.6.0`
scaler: GradScaler instance for gradient scaling if `torch>=1.6.0`
and ``amp_mode`` is ``amp``. If ``amp_mode`` is ``apex``, this argument will be ignored.
If True, will create default GradScaler. If GradScaler instance is passed, it will be used instead.
(default: False)

Returns:
a trainer engine with supervised update function.

Note:
If ``scaler`` is True, GradScaler instance will be created internally and trainer state has attribute named
``scaler`` for that instance and can be used for saving and loading.
Expand All @@ -375,9 +378,6 @@ def create_supervised_trainer(

See more: https://nvidia.github.io/apex/amp.html#module-apex.amp

Returns:
Engine: a trainer engine with supervised update function.

.. versionchanged:: 0.5.0

- Added ``amp_mode`` argument for automatic mixed precision.
Expand Down Expand Up @@ -424,18 +424,21 @@ def create_supervised_evaluator(
Factory function for creating an evaluator for supervised models.

Args:
model (`torch.nn.Module`): the model to train.
metrics (dict of str - :class:`~ignite.metrics.Metric`): a map of metric names to Metrics.
device (str, optional): device type specification (default: None).
model: the model to train.
metrics: a map of metric names to Metrics.
device: device type specification (default: None).
Applies to batches after starting the engine. Model *will not* be moved.
non_blocking (bool, optional): if True and this copy is between CPU and GPU, the copy may occur asynchronously
non_blocking: if True and this copy is between CPU and GPU, the copy may occur asynchronously
with respect to the host. For other cases, this argument has no effect.
prepare_batch (callable, optional): function that receives `batch`, `device`, `non_blocking` and outputs
prepare_batch: function that receives `batch`, `device`, `non_blocking` and outputs
tuple of tensors `(batch_x, batch_y)`.
output_transform (callable, optional): function that receives 'x', 'y', 'y_pred' and returns value
output_transform: function that receives 'x', 'y', 'y_pred' and returns value
to be assigned to engine's state.output after each iteration. Default is returning `(y_pred, y,)` which fits
output expected by metrics. If you change it you should use `output_transform` in metrics.

Returns:
an evaluator engine with supervised inference function.

Note:
`engine.state.output` for this engine is defind by `output_transform` parameter and is
a tuple of `(batch_pred, batch_y)` by default.
Expand All @@ -451,9 +454,6 @@ def create_supervised_evaluator(
- `PyTorch Documentation <https://pytorch.org/docs/stable/optim.html#constructing-it>`_

- `PyTorch's Explanation <https://github.com/pytorch/pytorch/issues/7844#issuecomment-503713840>`_

Returns:
Engine: an evaluator engine with supervised inference function.
"""
metrics = metrics or {}

Expand Down
28 changes: 15 additions & 13 deletions ignite/engine/deterministic.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@ def update_dataloader(dataloader: DataLoader, new_batch_sampler: BatchSampler) -
dataloader with new batch sampler.

Args:
dataloader (torch.utils.data.DataLoader): input dataloader
new_batch_sampler (torch.utils.data.sampler.BatchSampler): new batch sampler to use
dataloader: input dataloader
new_batch_sampler: new batch sampler to use

Returns:
DataLoader
Expand All @@ -39,22 +39,21 @@ class ReproducibleBatchSampler(BatchSampler):
"""Reproducible batch sampler. This class internally iterates and stores indices of the input batch sampler.
This helps to start providing data batches from an iteration in a deterministic way.

Usage:
Example:

Setup dataloader with `ReproducibleBatchSampler` and start providing data batches from an iteration:
Setup dataloader with `ReproducibleBatchSampler` and start providing data batches from an iteration

.. code-block:: python
.. code-block:: python

from ignite.engine.deterministic import update_dataloader
from ignite.engine.deterministic import update_dataloader

dataloader = update_dataloader(dataloader, ReproducibleBatchSampler(dataloader.batch_sampler))
# rewind dataloader to a specific iteration:
dataloader.batch_sampler.start_iteration = start_iteration
dataloader = update_dataloader(dataloader, ReproducibleBatchSampler(dataloader.batch_sampler))
# rewind dataloader to a specific iteration:
dataloader.batch_sampler.start_iteration = start_iteration

Args:
batch_sampler (torch.utils.data.sampler.BatchSampler): batch sampler same as used with
`torch.utils.data.DataLoader`
start_iteration (int, optional): optional start iteration
batch_sampler: batch sampler same as used with `torch.utils.data.DataLoader`.
start_iteration: optional start iteration.
"""

def __init__(self, batch_sampler: BatchSampler, start_iteration: Optional[int] = None):
Expand Down Expand Up @@ -119,7 +118,7 @@ def keep_random_state(func: Callable) -> Callable:
while executing a function. For more details on usage, please see :ref:`Dataflow synchronization`.

Args:
func (callable): function to decorate
func: function to decorate
"""

@wraps(func)
Expand Down Expand Up @@ -167,6 +166,9 @@ class DeterministicEngine(Engine):
This class can produce exactly the same dataflow when resuming the run from an epoch (or more precisely from
dataflow restart) and using torch `DataLoader` with `num_workers > 1` as data provider.

Args:
process_function: A function receiving a handle to the engine and the current batch
in each iteration, and returns data to be stored in the engine's state.
"""

def __init__(self, process_function: Callable):
Expand Down
Loading