diff --git a/spec-0001/index.md b/spec-0001/index.md index 168238f4..1d87ab4b 100644 --- a/spec-0001/index.md +++ b/spec-0001/index.md @@ -7,6 +7,7 @@ author: - "Dan Schult " discussion: https://discuss.scientific-python.org/t/spec-1-lazy-loading-for-submodules/25 endorsed-by: +shortcutDepth: 3 --- ## Description @@ -172,6 +173,57 @@ be immediately raised with: linalg = lazy.load('scipy.linalg', error_on_import=True) ``` +#### Type checkers + +The lazy loading shown above has one drawback: static type checkers +(such as [mypy](http://mypy-lang.org) and +[pyright](https://github.com/microsoft/pyright)) will not be able to +infer the types of lazy-loaded modules and functions. +Therefore, `mypy` won't be able to detect potential errors, and +integrated development environments such as [VS +Code](https://code.visualstudio.com) won't provide code completion. + +To work around this limitation, we provide an alternative way to define lazy +imports. +Instead of importing modules and functions in `__init__.py` +file with `lazy.attach`, you instead specify those imports in a `__init__.pyi` +file—called a "type stub". +Your `__init__.py` file then loads imports from the stub using `lazy.attach_stub`. + +Here's an example of how to convert this `__init__.py`: + +```python +# mypackage/__init__.py +import lazy_loader as lazy + +__getattr__, __dir__, __all__ = lazy.attach( + __name__, + submod_attrs={ + 'edges': ['sobel', 'sobel_h', 'sobel_v'] + } +) +``` + +Add a [type stub (`__init__.pyi`) file](https://mypy.readthedocs.io/en/stable/stubs.html) in the same directory as the `__init__.py`. + Type stubs are ignored at runtime, but used by static type checkers. + + ```python + # mypackage/__init__.pyi + from .edges import sobel, sobel_h, sobel_v + ``` + + Replace `lazy.attach` in `mypackage/__init__.py` with a call to `attach_stub`: + + ```python + import lazy_loader as lazy + + # this assumes there is a `.pyi` file adjacent to this module + __getattr__, __dir__, __all__ = lazy.attach_stub(__name__, __file__) + ``` + +_Note that if you use a type stub, you will need to take additional action to add the `.pyi` file to your sdist and wheel distributions. +See [PEP 561](https://peps.python.org/pep-0561/) and the [mypy documentation](https://mypy.readthedocs.io/en/stable/installed_packages.html#creating-pep-561-compatible-packages) for more information._ + ## Implementation Lazy loading is implemented at @@ -207,71 +259,6 @@ In the mean time, we now have the necessary mechanisms to implement it ourselves [^cannon]: Cannon B., personal communication, 7 January 2021. -### Caveats - -While this pattern has benefits at runtime, it does have one specific drawback: - -Static type checkers (such as [mypy](http://mypy-lang.org) and -[pyright](https://github.com/microsoft/pyright)) will not be able to infer the -types and locations dynamically-loaded modules and functions. This means that some -integrated development environments (e.g. -[VS Code](https://code.visualstudio.com)) will not be able to provide code -completion, parameter hints, documentation, or other features for any objects in your module. Nor will `mypy` be able to detect potential errors in your users' code. - -You can direct static type checkers to the actual location of dynamically loaded objects in one of two ways: - -1. Use an [`if typing.TYPE_CHECKING`](https://docs.python.org/3/library/typing.html#typing.TYPE_CHECKING) - clause . - - ```python - from typing import TYPE_CHECKING - - import lazy_loader as lazy - - if TYPE_CHECKING: - from .edges import sobel, sobel_h, sobel_v - - __getattr__, __dir__, __all__ = lazy.attach( - __name__, - submod_attrs={ - 'edges': ['sobel', 'sobel_h', 'sobel_v'] - } - ) - ``` - - `typing.TYPE_CHECKING` is a special variable that is set to `False` at runtime, and has negligible runtime impact. - -2. Use a [type stub (`.pyi`) file](https://mypy.readthedocs.io/en/stable/stubs.html). Type stubs are ignored at runtime, but used - by static type checkers. This method entails adding a new file with a `.pyi` extension in the same directory as your actual `.py`. For example: - - ```python - # mypackage/__init__.py - import lazy_loader as lazy - - __getattr__, __dir__, __all__ = lazy.attach( - __name__, - submod_attrs={ - 'edges': ['sobel', 'sobel_h', 'sobel_v'] - } - ) - ``` - - ```python - # mypackage/__init__.pyi - from .edges import sobel, sobel_h, sobel_v - ``` - - _Note that if you use a type stub, you will need to take additional action to add the `.pyi` file to your sdist and wheel distributions. See [PEP 561](https://peps.python.org/pep-0561/) and the [mypy documentation](https://mypy.readthedocs.io/en/stable/installed_packages.html#creating-pep-561-compatible-packages) for more information._ - -3. If you would like to avoid the unfortunate duplication of declaring exports in _both_ a type stub and in the arguments to `lazy.attach`, the aforementioned [lazy_loader](https://github.com/scientific-python/lazy_loader) package offers a [`lazy_loader.attach_stub` function](https://github.com/scientific-python/lazy_loader#lazily-load-subpackages-and-functions-from-type-stubs) that can be used to infer your module exports directly from your stub file – which now becomes the single source of truth for your module's exports. It is used as follows: - - ```python - from lazy_loader import attach_stub - - # this assumes there is a `.pyi` file adjacent to this module - __getattr__, __dir__, __all__ = attach_stub(__name__, __file__) - ``` - ### Core Project Endorsement