CI: Force docs warnings to be raised as errors (+ fix all)

.circleci/config.yml

@@ -56,7 +56,7 @@ references:
        pip install -r requirements.txt --user
        sudo pip install -r docs/requirements.txt
        # sphinx-apidoc -o ./docs/source ./pytorch_lightning **/test_* --force --follow-links
-       cd docs; make clean ; make html --debug --jobs 2
+       cd docs; make clean ; make html --debug --jobs 2 SPHINXOPTS="-W"
 
 jobs:
 

CHANGELOG.md

@@ -31,6 +31,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
 
 - Fixed bug related to type cheking of `ReduceLROnPlateau` lr schedulers([#1114](https://github.com/PyTorchLightning/pytorch-lightning/issues/1114))
 - Fixed a bug to ensure lightning checkpoints to be backward compatible ([#1132](https://github.com/PyTorchLightning/pytorch-lightning/pull/1132))
+- Fixed all warnings and errors in the docs build process ([#1191](https://github.com/PyTorchLightning/pytorch-lightning/pull/1191))
 
 ## [0.7.1] - 2020-03-07
 

docs/requirements.txt

@@ -8,4 +8,5 @@ sphinxcontrib-fulltoc
 sphinxcontrib-mockautodoc
 git+https://github.com/PytorchLightning/lightning_sphinx_theme.git
 # pip_shims
-sphinx-autodoc-typehints
+sphinx-autodoc-typehints
+sphinx-paramlinks

docs/source/callbacks.rst

@@ -1,6 +1,8 @@
 .. role:: hidden
     :class: hidden-section
 
+.. _callbacks:
+
 Callbacks
 =========
 

docs/source/conf.py

@@ -73,7 +73,7 @@
 # ones.
 extensions = [
     'sphinx.ext.autodoc',
-    'sphinxcontrib.mockautodoc',
+    # 'sphinxcontrib.mockautodoc',  # raises error: directive 'automodule' is already registered ...
     # 'sphinxcontrib.fulltoc',  # breaks pytorch-theme with unexpected kw argument 'titles_only'
     'sphinx.ext.doctest',
     'sphinx.ext.intersphinx',
@@ -87,6 +87,7 @@
     # 'm2r',
     'nbsphinx',
     'sphinx_autodoc_typehints',
+    'sphinx_paramlinks',
 ]
 
 # Add any paths that contain templates here, relative to this directory.
@@ -125,7 +126,20 @@
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.
 # This pattern also affects html_static_path and html_extra_path.
-exclude_patterns = ['*.test_*']
+exclude_patterns = [
+    'pytorch_lightning.rst',
+    'pl_examples.*',
+    'modules.rst',
+
+    # deprecated/renamed:
+    'pytorch_lightning.loggers.comet_logger.rst',           # TODO: remove in v0.8.0
+    'pytorch_lightning.loggers.mlflow_logger.rst',          # TODO: remove in v0.8.0
+    'pytorch_lightning.loggers.test_tube_logger.rst',       # TODO: remove in v0.8.0
+    'pytorch_lightning.callbacks.pt_callbacks.*',           # TODO: remove in v0.8.0
+    'pytorch_lightning.pt_overrides.*',                     # TODO: remove in v0.8.0
+    'pytorch_lightning.root_module.*',                      # TODO: remove in v0.8.0
+    'pytorch_lightning.logging.*',                          # TODO: remove in v0.8.0
+]
 
 # The name of the Pygments (syntax highlighting) style to use.
 pygments_style = None
@@ -297,8 +311,17 @@ def setup(app):
             MOCK_REQUIRE_PACKAGES.append(pkg.rstrip())
 
 # TODO: better parse from package since the import name and package name may differ
-MOCK_MANUAL_PACKAGES = ['torch', 'torchvision', 'test_tube',
-                        'mlflow', 'comet_ml', 'wandb', 'neptune', 'trains']
+MOCK_MANUAL_PACKAGES = [
+    'torch',
+    'torchvision',
+    'PIL',
+    'test_tube',
+    'mlflow',
+    'comet_ml',
+    'wandb',
+    'neptune',
+    'trains',
+]
 autodoc_mock_imports = MOCK_REQUIRE_PACKAGES + MOCK_MANUAL_PACKAGES
 # for mod_name in MOCK_REQUIRE_PACKAGES:
 #     sys.modules[mod_name] = mock.Mock()
@@ -369,7 +392,7 @@ def find_source():
 # This value determines the text for the permalink; it defaults to "¶". Set it to None or the empty
 #  string to disable permalinks.
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-html_add_permalinks
-html_add_permalinks = True
+html_add_permalinks = "¶"
 
 # True to prefix each section label with the name of the document it is in, followed by a colon.
 #  For example, index:Introduction for a section called Introduction that appears in document index.rst.

docs/source/debugging.rst

@@ -8,6 +8,9 @@ This flag runs a "unit test" by running 1 training batch and 1 validation batch.
 The point is to detect any bugs in the training/validation loop without having to wait for
 a full epoch to crash.
 
+(See: :paramref:`~pytorch_lightning.trainer.trainer.Trainer.fast_dev_run`
+argument of :class:`~pytorch_lightning.trainer.trainer.Trainer`)
+
 .. code-block:: python
 
     trainer = pl.Trainer(fast_dev_run=True)
@@ -16,6 +19,9 @@ Inspect gradient norms
 ----------------------
 Logs (to a logger), the norm of each weight matrix.
 
+(See: :paramref:`~pytorch_lightning.trainer.trainer.Trainer.track_grad_norm`
+argument of :class:`~pytorch_lightning.trainer.trainer.Trainer`)
+
 .. code-block:: python
 
     # the 2-norm
@@ -25,7 +31,8 @@ Log GPU usage
 -------------
 Logs (to a logger) the GPU usage for each GPU on the master machine.
 
-(See: :ref:`trainer`)
+(See: :paramref:`~pytorch_lightning.trainer.trainer.Trainer.log_gpu_memory`
+argument of :class:`~pytorch_lightning.trainer.trainer.Trainer`)
 
 .. code-block:: python
 
@@ -37,7 +44,8 @@ Make model overfit on subset of data
 A good debugging technique is to take a tiny portion of your data (say 2 samples per class),
 and try to get your model to overfit. If it can't, it's a sign it won't work with large datasets.
 
-(See: :ref:`trainer`)
+(See: :paramref:`~pytorch_lightning.trainer.trainer.Trainer.overfit_pct`
+argument of :class:`~pytorch_lightning.trainer.trainer.Trainer`)
 
 .. code-block:: python
 
@@ -48,28 +56,23 @@ Print the parameter count by layer
 Whenever the .fit() function gets called, the Trainer will print the weights summary for the lightningModule.
 To disable this behavior, turn off this flag:
 
-(See: :ref:`trainer.weights_summary`)
+(See: :paramref:`~pytorch_lightning.trainer.trainer.Trainer.weights_summary`
+argument of :class:`~pytorch_lightning.trainer.trainer.Trainer`)
 
 .. code-block:: python
 
     trainer = pl.Trainer(weights_summary=None)
 
-Print which gradients are nan
------------------------------
-Prints the tensors with nan gradients.
-
-(See: :meth:`trainer.print_nan_grads`)
-
-.. code-block:: python
-
-    trainer = pl.Trainer(print_nan_grads=False)
 
 Set the number of validation sanity steps
 -----------------------------------------
 Lightning runs a few steps of validation in the beginning of training.
 This avoids crashing in the validation loop sometime deep into a lengthy training loop.
 
+(See: :paramref:`~pytorch_lightning.trainer.trainer.Trainer.num_sanity_val_steps`
+argument of :class:`~pytorch_lightning.trainer.trainer.Trainer`)
+
 .. code-block:: python
 
     # DEFAULT
-    trainer = Trainer(nb_sanity_val_steps=5)
+    trainer = Trainer(num_sanity_val_steps=5)

docs/source/early_stopping.rst

@@ -11,7 +11,8 @@ Enable Early Stopping
 ---------------------
 There are two ways to enable early stopping.
 
-.. seealso:: :ref:`trainer`
+.. seealso::
+    :class:`~pytorch_lightning.trainer.trainer.Trainer`
 
 .. code-block:: python
 
@@ -35,4 +36,5 @@ To disable early stopping pass ``False`` to the `early_stop_callback`.
 Note that ``None`` will not disable early stopping but will lead to the
 default behaviour.
 
-.. seealso:: :ref:`trainer`
+.. seealso::
+    :class:`~pytorch_lightning.trainer.trainer.Trainer`

docs/source/experiment_logging.rst

@@ -7,7 +7,8 @@ Comet.ml
 `Comet.ml <https://www.comet.ml/site/>`_ is a third-party logger.
 To use CometLogger as your logger do the following.
 
-.. seealso:: :ref:`comet` docs.
+.. seealso::
+    :class:`~pytorch_lightning.loggers.CometLogger` docs.
 
 .. code-block:: python
 
@@ -38,7 +39,8 @@ Neptune.ai
 `Neptune.ai <https://neptune.ai/>`_ is a third-party logger.
 To use Neptune.ai as your logger do the following.
 
-.. seealso:: :ref:`neptune` docs.
+.. seealso::
+    :class:`~pytorch_lightning.loggers.NeptuneLogger` docs.
 
 .. code-block:: python
 
@@ -68,7 +70,8 @@ allegro.ai TRAINS
 `allegro.ai <https://github.com/allegroai/trains/>`_ is a third-party logger.
 To use TRAINS as your logger do the following.
 
-.. seealso:: :ref:`trains` docs.
+.. seealso::
+    :class:`~pytorch_lightning.loggers.TrainsLogger` docs.
 
 .. code-block:: python
 
@@ -95,7 +98,8 @@ Tensorboard
 
 To use `Tensorboard <https://pytorch.org/docs/stable/tensorboard.html>`_ as your logger do the following.
 
-.. seealso:: TensorBoardLogger :ref:`tf-logger`
+.. seealso::
+    :class:`~pytorch_lightning.loggers.TensorBoardLogger` docs.
 
 .. code-block:: python
 
@@ -121,7 +125,8 @@ Test Tube
 `Test Tube <https://github.com/williamFalcon/test-tube>`_ is a tensorboard logger but with nicer file structure.
 To use TestTube as your logger do the following.
 
-.. seealso:: TestTube :ref:`testTube`
+.. seealso::
+    :class:`~pytorch_lightning.loggers.TestTubeLogger` docs.
 
 .. code-block:: python
 
@@ -146,7 +151,8 @@ Wandb
 `Wandb <https://www.wandb.com/>`_ is a third-party logger.
 To use Wandb as your logger do the following.
 
-.. seealso:: :ref:`wandb` docs
+.. seealso::
+    :class:`~pytorch_lightning.loggers.WandbLogger` docs.
 
 .. code-block:: python
 
@@ -167,7 +173,7 @@ The Wandb logger is available anywhere except ``__init__`` in your LightningModu
 
 
 Multiple Loggers
-^^^^^^^^^^^^^^^^^
+^^^^^^^^^^^^^^^^
 
 PyTorch-Lightning supports use of multiple loggers, just pass a list to the `Trainer`.
 

docs/source/experiment_reporting.rst

@@ -22,7 +22,8 @@ Control log writing frequency
 Writing to a logger  can be expensive. In Lightning you can set the interval at which you
 want to log using this trainer flag.
 
-.. seealso:: :ref:`trainer`
+.. seealso::
+    :class:`~pytorch_lightning.trainer.trainer.Trainer`
 
 .. code-block:: python
 

docs/source/fast_training.rst

@@ -16,7 +16,8 @@ Force training for min or max epochs
 -------------------------------------
 It can be useful to force training for a minimum number of epochs or limit to a max number.
 
-.. seealso:: :ref:`trainer`
+.. seealso::
+    :class:`~pytorch_lightning.trainer.trainer.Trainer`
 
 .. code-block:: python
 

docs/source/hooks.rst

@@ -2,6 +2,7 @@ Hooks
 =====
 
 .. automodule:: pytorch_lightning.core.hooks
+    :noindex:
 
 Hooks lifecycle
 ---------------

docs/source/index.rst

@@ -11,6 +11,7 @@ PyTorch Lightning Documentation
    :name: start
    :caption: Start Here
 
+   new-project
    introduction_guide
 
 .. toctree::
@@ -24,13 +25,24 @@ PyTorch Lightning Documentation
    loggers
    trainer
 
-
 .. toctree::
    :maxdepth: 1
    :name: Community Examples
    :caption: Community Examples
 
-   examples
+   Contextual Emotion Detection (DoubleDistilBert) <https://github.com/PyTorchLightning/emotion_transformer>
+   Generative Adversarial Network <https://colab.research.google.com/drive/1F_RNcHzTfFuQf-LeKvSlud6x7jXYkG31#scrollTo=TyYOdg8g77P0>
+   Hyperparameter optimization with Optuna <https://github.com/optuna/optuna/blob/master/examples/pytorch_lightning_simple.py>
+   Image Inpainting using Partial Convolutions <https://github.com/ryanwongsa/Image-Inpainting>
+   MNIST on TPU <https://colab.research.google.com/drive/1-_LKx4HwAxl5M6xPJmqAAu444LTDQoa3#scrollTo=BHBz1_AnamN_>
+   NER (transformers, TPU) <https://colab.research.google.com/drive/1dBN-wwYUngLYVt985wGs_OKPlK_ANB9D>
+   NeuralTexture (CVPR) <https://github.com/PyTorchLightning/neuraltexture>
+   Recurrent Attentive Neural Process <https://github.com/PyTorchLightning/attentive-neural-processes>
+   Siamese Nets for One-shot Image Recognition <https://github.com/PyTorchLightning/Siamese-Neural-Networks>
+   Speech Transformers <https://github.com/PyTorchLightning/speech-transformer-pytorch_lightning>
+   Transformers transfer learning (Huggingface) <https://colab.research.google.com/drive/1F_RNcHzTfFuQf-LeKvSlud6x7jXYkG31#scrollTo=yr7eaxkF-djf>
+   Transformers text classification <https://github.com/ricardorei/lightning-text-classification>
+   VAE Library of over 18+ VAE flavors <https://github.com/AntixK/PyTorch-VAE>
 
 .. toctree::
    :maxdepth: 1
@@ -83,3 +95,17 @@ Indices and tables
 * :ref:`genindex`
 * :ref:`modindex`
 * :ref:`search`
+
+
+
+.. This is here to make sphinx aware of the modules but not throw an error/warning
+.. toctree::
+   :hidden:
+
+   pytorch_lightning.core
+   pytorch_lightning.callbacks
+   pytorch_lightning.loggers
+   pytorch_lightning.overrides
+   pytorch_lightning.profiler
+   pytorch_lightning.trainer
+   pytorch_lightning.utilities

docs/source/introduction_guide.rst

@@ -472,7 +472,7 @@ First, change the runtime to TPU (and reinstall lightning).
 
 Next, install the required xla library (adds support for PyTorch on TPUs)
 
-.. code-block::
+.. code-block:: python
 
     import collections
     from datetime import datetime, timedelta
@@ -504,6 +504,8 @@ Next, install the required xla library (adds support for PyTorch on TPUs)
     update = threading.Thread(target=update_server_xrt)
     update.start()
 
+.. code-block::
+
     # Install Colab TPU compat PyTorch/TPU wheels and dependencies
     !pip uninstall -y torch torchvision
     !gsutil cp "$DIST_BUCKET/$TORCH_WHEEL" .
@@ -981,7 +983,8 @@ And pass the callbacks into the trainer
 
     Trainer(callbacks=[MyPrintingCallback()])
 
-.. note:: See full list of 12+ hooks in the `Callback docs <callbacks.rst#callback-class>`_
+.. note::
+    See full list of 12+ hooks in the :ref:`callbacks`.
 
 ---------