feat: add lazy_context (beware: magic)

[tlambert03]

I was intrigued by @maurosilber context manager concept from #11 ...

This PR achieves that syntax using some serious ridiculousness 😂

import lazy_loader

with lazy_loader.lazy_imports() as stop:
    raise stop

    from . import rank
    from ._fft_based import butterworth
    from ._gabor import gabor, gabor_kernel
    from ._gaussian import difference_of_gaussians, gaussian
    ...

believe it or not, it works! and it makes IDEs happy without any pyi files. It basically parses the source code from the with block using the same ast parser from #10 ... and then injects the output into the module globals

That said, I wouldn't be surprised or offended in the least if you closed this PR immediately and rushed off to take a shower 🛀 Opening this mostly as a proof of principle and record of a little magic exploration :)

[stefanv]

@tlambert03 Hah, fun and clever! But, indeed, the raise stop syntax is too counter-intuitive to include. I'll close this for now (it's open for discussion still, if others want to chime in!) and perhaps a way to avoid the raise clause jumps out (although, I cannot imagine, since you have to tell Python not to perform the imports).

[tlambert03]

(although, I cannot imagine, since you have to tell Python not to perform the imports).

there is a way by patching sys.settrace ... but it gets into even darker magic. yes. agreed. let us never speak of this again 😂

[kne42]

mad respect for the crazy magic

[kne42]

@stefanv you could achieve the same by loading an invalid package, e.g.:

import lazy_loader

with lazy_loader.lazy_imports():
    import LAZY_LOAD

    from . import rank
    from ._fft_based import butterworth
    from ._gabor import gabor, gabor_kernel
    from ._gaussian import difference_of_gaussians, gaussian
    ...

[kne42]

or really use any exception of your choice to stop the process. actually thinking about this i like that form of magic 😂

for example u could make a function lazy_loader.interrupt() which then raises an exception type caught by the with statement

[stefanv]

For extra confusion, you can use:

with lazy_loader.lazy_imports():
    ...
    0//4
    ...

    from . import rank
    from ._fft_based import butterworth
    from ._gabor import gabor, gabor_kernel
    from ._gaussian import difference_of_gaussians, gaussian

😄 Maybe more sensibly:

with lazy_loader.lazy_imports() as delay_imports:
    delay_imports()

    from . import rank
    from ._fft_based import butterworth
    from ._gabor import gabor, gabor_kernel
    from ._gaussian import difference_of_gaussians, gaussian

[vnmabus]

and perhaps a way to avoid the raise clause jumps out (although, I cannot imagine, since you have to tell Python not to perform the imports).

Apparently, the assignment to a variable also counts as part of the code inside the with statement, so you could make THAT fail instead (such as assigning to a read-only property of the module or something like it).

You would want to catch in the __exit__ the case where the user forgot to assign it (and thus the body didn't raise) and give the user a useful message.

[vnmabus]

I still cannot let this syntax go (it is better than stubs).
Why not just make the imports themselves fail:

from contextlib import contextmanager

def raise_exc(*args, **kwargs):
    raise ImportError

@contextmanager
def remove_exceptions():
    import builtins

    import_fun = builtins.__import__
    builtins.__import__ = raise_exc

    try:
        yield None
    finally:        
        builtins.__import__ = import_fun

with remove_exceptions:
    from numpy import mean

This approach even allows to remove the magic, letting Python machinery give us the import names themselves!!!:

import builtins
from types import SimpleNamespace

class SaveModules():

    def __init__(self):
        self.imports = []

    def __enter__(self):
        self.import_fun = builtins.__import__
        builtins.__import__ = self._my_import
        return None

    def __exit__(self, type, value, tb):
        builtins.__import__ = self.import_fun
        print(self.imports)

    def _my_import(self, name, globals=None, locals=None, fromlist=(), level=0):
        builtins.__import__ = self.import_fun
        self.imports.append({"name": name, "fromlist": fromlist, "level": level})
        builtins.__import__ = self._my_import

        if fromlist:
              return SimpleNamespace(**{k: None for k in fromlist})
        return None


with SaveModules():
    from numpy import mean
    import scipy
    from .. import kk

[stefanv]

The advantage of the stubs is that you get type inference, which is more important to a lot of people than I realized. But this is certainly an improvement in syntax over lazy_attach.

[vnmabus]

Why shouldn't you get type inference with this approach? As far as I know Mypy will assume that the imports work as normal (I haven't tested it) and use their associated types.

[stefanv]

Worth a try!

[tlambert03]

indeed. worth a try! :)