Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add typing for function_decorator #23

Merged
merged 12 commits into from
Feb 21, 2022
72 changes: 72 additions & 0 deletions src/decopatch/main.pyi
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
from typing import Any, Callable, Optional, Protocol, TypeVar, overload

from decopatch.utils_disambiguation import FirstArgDisambiguation
from decopatch.utils_modes import SignatureInfo

try:
from typing import ParamSpec
except ImportError:
from typing_extensions import ParamSpec

P = ParamSpec("P")
F = TypeVar("F", bound=Callable[..., Any])

class _Decorator(Protocol[P]):
"""
This is callable Protocol, to distinguish between cases where
created decorator is called as `@decorator` or `@decorator()`
"""

# decorator is called without parenthesis: @decorator
@overload
def __call__(self, func: F) -> F: ...
# decorator is called with options or parenthesis: @decorator(some_option=...)
@overload
def __call__(self, *args: P.args, **kwargs: P.kwargs) -> Callable[[F], F]: ...

# @function_decorator is called without options or parenthesis
smarie marked this conversation as resolved.
Show resolved Hide resolved
@overload
def function_decorator(
enable_stack_introspection: Callable[P, Any],
custom_disambiguator: Callable[[Any], FirstArgDisambiguation] = ...,
flat_mode_decorated_name: Optional[str] = ...,
) -> _Decorator[P]: ...

# @function_decorator() is called with options or parenthesis.
@overload
def function_decorator(
enable_stack_introspection: bool = ...,
custom_disambiguator: Callable[[Any], FirstArgDisambiguation] = ...,
flat_mode_decorated_name: Optional[str] = ...,
) -> Callable[[Callable[P, Any]], _Decorator[P]]: ...
def class_decorator(
enable_stack_introspection: bool = ...,
custom_disambiguator: Callable[[Any], FirstArgDisambiguation] = ...,
flat_mode_decorated_name: Optional[str] = ...,
): ...
Comment on lines +42 to +46
Copy link
Owner

@smarie smarie Feb 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally we should do the same for class_decorator and for decorator, don't you think ?

Note that _Decorator might need to change to support decorating Types too. Or, even more ideally, we would have a _FuncDecorator for function_decorator, a _ClassDecorator for class_decorator and a _Decorator = Union[_FuncDecorator, _ClassDecorator] for decorator.

Would it make sense ?

Note: this may be for another PR

Suggested change
def class_decorator(
enable_stack_introspection: bool = ...,
custom_disambiguator: Callable[[Any], FirstArgDisambiguation] = ...,
flat_mode_decorated_name: Optional[str] = ...,
): ...
# @class_decorator() is called without options or parenthesis
@overload
def class_decorator(
enable_stack_introspection: Callable[P, Any],
custom_disambiguator: Callable[[Any], FirstArgDisambiguation] = ...,
flat_mode_decorated_name: Optional[str] = ...,
) -> _Decorator[P]: ...
# @class_decorator() is called with options or parenthesis.
@overload
def class_decorator(
enable_stack_introspection: bool = ...,
custom_disambiguator: Callable[[Any], FirstArgDisambiguation] = ...,
flat_mode_decorated_name: Optional[str] = ...,
) -> Callable[[Callable[P, Any]], _Decorator[P]]: ...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that makes sense. I could try to do it in next PR.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perfect

def decorator(
is_function_decorator: bool = ...,
is_class_decorator: bool = ...,
enable_stack_introspection: bool = ...,
custom_disambiguator: Callable[[Any], FirstArgDisambiguation] = ...,
use_signature_trick: bool = ...,
flat_mode_decorated_name: str = ...,
): ...
def create_decorator(
impl_function,
is_function_decorator: bool = ...,
is_class_decorator: bool = ...,
enable_stack_introspection: bool = ...,
custom_disambiguator: Callable[[Any], FirstArgDisambiguation] = ...,
use_signature_trick: bool = ...,
flat_mode_decorated_name: Optional[str] = ...,
): ...
def create_no_args_decorator(
decorator_function, function_for_metadata: Any | None = ...
): ...
def create_kwonly_decorator(
sig_info: SignatureInfo, decorator_function, disambiguator, function_for_metadata
): ...
def create_general_case_decorator(
sig_info: SignatureInfo, impl_function, disambiguator, function_for_metadata
): ...
smarie marked this conversation as resolved.
Show resolved Hide resolved
57 changes: 57 additions & 0 deletions tests/test_typing.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# This is test file for typing,
# No automatic testing is used at the moment. Just use your type checker and see if it works.
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just being curious here: isn't there a way to assert typing-related things in pytest ? We can even consider using typing-inspect or another such package if it makes sense..

Alternately we could add a mypy step in the nox file, scanning precisely this test file ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember few days ago seeing article about testing typing in python packages, but couldn't find it at the moment.

We cannot just run mypy or pyright in CI, becouse it will exit with errors, and we want to be sure that it fails where we expect it to fail.

I think, our best shot is to run mypy and pyright using subprocess in pytest, and compare it's output with what we're expecting. I could try to do this in next PR or in this, what do you think?

But before that i want to think a little about other options, and maybe see what others are doing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As for typing-inspect, i don't think it could be used in this case. Sure it could check something, but i'd want to test exact cases.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, i found something.

django-types are running pyright in subprocess, and parsing it's messages

https://github.com/sbdchd/django-types/blob/main/tests/pyright/base.py

and, mypy could be called from pytest using it's api:

https://mypy.readthedocs.io/en/stable/extending_mypy.html

I tried using mypy, but strumbled upon a bug which crashes mypy. Now i'm trying to fix it or make minimal example to report bug in mypy.

I think, merging this could actually crash mypy for someone who uses it.

python/mypy#8645

They already have similar issues with other code.

Copy link
Contributor Author

@last-partizan last-partizan Feb 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, nevermind. It was crashing becouse it has cached some data from previous version in my environment. It's working fine. Just doesn't supports those ParamSpecs properly yet.

So, i think, if we want to add type testing, we could use pyright just like django-types. (But maybe that's for another PR)

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great ! Ok, let's keep this for another PR.

# Pytest here is used to make sure that runtime behavir matches with type checker expecter errors.
from typing import Any, Callable

import pytest

from decopatch import DECORATED, function_decorator


def test_invalid_parameter():
with pytest.raises(TypeError):
# Error, invalid argument
@function_decorator(invalid_param=True)
def decorator_wint_invalid_param(fn=DECORATED):
return fn


def test_normal_decorator():
@function_decorator
def decorator(scope="test", fn=DECORATED): # type: (str, Any) -> Callable[..., Any]
assert isinstance(scope, str)
return fn

# Ok
@decorator
def decorated_flat():
pass

assert decorated_flat

with pytest.raises(AssertionError):
# Error, Literal[2] is incompatible with str
@decorator(scope=2)
def decorated_with_invalid_options():
pass

# Ok, should reveal correct type for `scope`
@decorator(scope="success")
def decorated_with_valid_options():
pass

assert decorated_with_valid_options


def test_function_decorator_with_params():
# Ok, should reveal correct type for `enable_stack_introspection`
@function_decorator(enable_stack_introspection=True)
def decorator_with_params(scope = "test", fn=DECORATED): # type: (str, Any) -> Callable[..., Any]
return fn

# Ok, should reveal correct type for `scope`
@decorator_with_params(scope="success")
def decorated_with_valid_options():
pass

assert decorated_with_valid_options