Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wrapping class with icontract invariant decorator causes PyLance to incorrectly detect type. #1667

Closed
ciresnave opened this issue Aug 12, 2021 · 6 comments
Labels
waiting for upstream Waiting for upstream to release a fix

Comments

@ciresnave
Copy link

ciresnave commented Aug 12, 2021

Environment data

  • Language Server version: 2021.8.2-pre.1
  • OS and version: win32 x64
  • Python version (and distribution if applicable, e.g. Anaconda): Python 3.9.6
  • python.analysis.indexing: undefined
  • python.analysis.typeCheckingMode: strict

Expected behaviour

When using icontract to add an invariant to a class, both mypy and Python's own type() command show the type of a class wrapped by icontract to be the same as one that was never wrapped. PyLance shouldn't have an issue with using a wrapped class in place of the bare class for type hinting.

Actual behaviour

TestClass1 is a bare class.
TestClass2 is a wrapped class.
(see below for the actual code)

PyLance states that TestClass1 is of type "Type[TestClass1]" while it shows that TestClass2 is of type "type".
PyLance states that TestClass2 can't be used to type annotate a dataclass field because it expected a Class but got a type instead. Further, because of that issue, PyLance states that variables within TestClass2 are of unknown type and can't be referenced. The code works fine.

Mypy states:

src\stock_trader\invariant_type_change.py:9:13: note: Revealed type is "def () -> stock_trader.invariant_type_change.TestClass1"
src\stock_trader\invariant_type_change.py:18:13: note: Revealed type is "def () -> stock_trader.invariant_type_change.TestClass2"

..and printing the type of each from Python shows:

<class '__main__.TestClass1'>
<class '__main__.TestClass2'>

Logs

Python Language Server Log

Background analysis message: getDiagnosticsForRange
Background analysis message: getDiagnosticsForRange
Background analysis message: getDiagnosticsForRange
Background analysis message: getDiagnosticsForRange
Background analysis message: markFilesDirty
Background analysis message: markFilesDirty
Background analysis message: analyze
Background analysis message: markFilesDirty
Background analysis message: analyze
Background analysis message: getDiagnosticsForRange
Background analysis message: getDiagnosticsForRange
Background analysis message: markFilesDirty
Background analysis message: invalidateAndForceReanalysis
Background analysis message: invalidateAndForceReanalysis
Background analysis message: markFilesDirty
[Info  - 12:45:04 PM] Searching for source files
[Info  - 12:45:04 PM] Auto-excluding c:\Users\cires\Documents\GitHub\stock_trader\.venv
[Info  - 12:45:04 PM] Found 9 source files
Background analysis message: setTrackedFiles
Background analysis message: markAllFilesDirty
Background analysis message: analyze
[BG(1)] analyzing: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\invariant_type_change.py ...
[BG(1)]   parsing: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\invariant_type_change.py (142ms)
[BG(1)]   parsing: c:\Users\cires\.vscode\extensions\ms-python.vscode-pylance-2021.8.2-pre.1\dist\typeshed-fallback\stdlib\builtins.pyi [fs read 0ms] (23ms)
[BG(1)]   binding: c:\Users\cires\.vscode\extensions\ms-python.vscode-pylance-2021.8.2-pre.1\dist\typeshed-fallback\stdlib\builtins.pyi (19ms)
[BG(1)]   binding: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\invariant_type_change.py (0ms)
[BG(1)]   checking: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\invariant_type_change.py ...
[BG(1)]     parsing: c:\Users\cires\.vscode\extensions\ms-python.vscode-pylance-2021.8.2-pre.1\dist\typeshed-fallback\stdlib\dataclasses.pyi [fs read 0ms] (4ms)
[BG(1)]     binding: c:\Users\cires\.vscode\extensions\ms-python.vscode-pylance-2021.8.2-pre.1\dist\typeshed-fallback\stdlib\dataclasses.pyi (1ms)
[BG(1)]   checking: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\invariant_type_change.py (5ms)
[BG(1)] analyzing: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\invariant_type_change.py (189ms)
[BG(1)] parsing: c:\Users\cires\Documents\GitHub\stock_trader\noxfile.py [fs read 1ms] (7ms)
[BG(1)] binding: c:\Users\cires\Documents\GitHub\stock_trader\noxfile.py (2ms)
[BG(1)] indexing: c:\Users\cires\Documents\GitHub\stock_trader\noxfile.py (0ms)
Background analysis message: markFilesDirty
Background analysis message: markFilesDirty
Background analysis message: markFilesDirty
Background analysis message: analyze
[BG(1)] analyzing: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\invariant_type_change.py ...
[BG(1)]   checking: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\invariant_type_change.py ...
[BG(1)]     parsing: c:\Users\cires\.vscode\extensions\ms-python.vscode-pylance-2021.8.2-pre.1\dist\typeshed-fallback\stdlib\typing.pyi [fs read 0ms] (11ms)
[BG(1)]     binding: c:\Users\cires\.vscode\extensions\ms-python.vscode-pylance-2021.8.2-pre.1\dist\typeshed-fallback\stdlib\typing.pyi (5ms)
[BG(1)]     parsing: c:\Users\cires\.vscode\extensions\ms-python.vscode-pylance-2021.8.2-pre.1\dist\typeshed-fallback\stdlib\_typeshed\__init__.pyi [fs read 1ms] (5ms)
[BG(1)]     binding: c:\Users\cires\.vscode\extensions\ms-python.vscode-pylance-2021.8.2-pre.1\dist\typeshed-fallback\stdlib\_typeshed\__init__.pyi (1ms)
[BG(1)]     parsing: c:\Users\cires\.vscode\extensions\ms-python.vscode-pylance-2021.8.2-pre.1\dist\typeshed-fallback\stdlib\typing_extensions.pyi [fs read 0ms] (2ms)
[BG(1)]     binding: c:\Users\cires\.vscode\extensions\ms-python.vscode-pylance-2021.8.2-pre.1\dist\typeshed-fallback\stdlib\typing_extensions.pyi (1ms)
[BG(1)]     parsing: c:\Users\cires\Documents\GitHub\stock_trader\.venv\Lib\site-packages\icontract\__init__.py [fs read 1ms] (8ms)
[BG(1)]     binding: c:\Users\cires\Documents\GitHub\stock_trader\.venv\Lib\site-packages\icontract\__init__.py (0ms)
[BG(1)]     parsing: c:\Users\cires\Documents\GitHub\stock_trader\.venv\Lib\site-packages\icontract\_decorators.py [fs read 1ms] (8ms)
[BG(1)]     binding: c:\Users\cires\Documents\GitHub\stock_trader\.venv\Lib\site-packages\icontract\_decorators.py (1ms)
[BG(1)]     parsing: c:\Users\cires\Documents\GitHub\stock_trader\.venv\Lib\site-packages\icontract\_globals.py [fs read 0ms] (1ms)
[BG(1)]     binding: c:\Users\cires\Documents\GitHub\stock_trader\.venv\Lib\site-packages\icontract\_globals.py (0ms)
[BG(1)]     parsing: c:\Users\cires\Documents\GitHub\stock_trader\.venv\Lib\site-packages\icontract\_metaclass.py [fs read 0ms] (4ms)
[BG(1)]     binding: c:\Users\cires\Documents\GitHub\stock_trader\.venv\Lib\site-packages\icontract\_metaclass.py (2ms)
[BG(1)]     parsing: c:\Users\cires\Documents\GitHub\stock_trader\.venv\Lib\site-packages\icontract\_types.py [fs read 0ms] (1ms)
[BG(1)]     binding: c:\Users\cires\Documents\GitHub\stock_trader\.venv\Lib\site-packages\icontract\_types.py (1ms)
[BG(1)]     parsing: c:\Users\cires\Documents\GitHub\stock_trader\.venv\Lib\site-packages\icontract\errors.py [fs read 0ms] (0ms)
[BG(1)]     binding: c:\Users\cires\Documents\GitHub\stock_trader\.venv\Lib\site-packages\icontract\errors.py (0ms)
[BG(1)]     parsing: c:\Users\cires\.vscode\extensions\ms-python.vscode-pylance-2021.8.2-pre.1\dist\typeshed-fallback\stdlib\reprlib.pyi [fs read 0ms] (1ms)
[BG(1)]     binding: c:\Users\cires\.vscode\extensions\ms-python.vscode-pylance-2021.8.2-pre.1\dist\typeshed-fallback\stdlib\reprlib.pyi (0ms)
[BG(1)]     parsing: c:\Users\cires\Documents\GitHub\stock_trader\.venv\Lib\site-packages\icontract\_checkers.py [fs read 1ms] (10ms)
[BG(1)]     binding: c:\Users\cires\Documents\GitHub\stock_trader\.venv\Lib\site-packages\icontract\_checkers.py (6ms)
[BG(1)]     parsing: c:\Users\cires\.vscode\extensions\ms-python.vscode-pylance-2021.8.2-pre.1\dist\typeshed-fallback\stdlib\os\__init__.pyi [fs read 1ms] (11ms)
[BG(1)]     binding: c:\Users\cires\.vscode\extensions\ms-python.vscode-pylance-2021.8.2-pre.1\dist\typeshed-fallback\stdlib\os\__init__.pyi (3ms)
[BG(1)]     parsing: c:\Users\cires\.vscode\extensions\ms-python.vscode-pylance-2021.8.2-pre.1\dist\typeshed-fallback\stdlib\abc.pyi [fs read 0ms] (0ms)
[BG(1)]     binding: c:\Users\cires\.vscode\extensions\ms-python.vscode-pylance-2021.8.2-pre.1\dist\typeshed-fallback\stdlib\abc.pyi (1ms)
[BG(1)]   checking: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\invariant_type_change.py (106ms)
[BG(1)] analyzing: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\invariant_type_change.py (106ms)
Background analysis message: resumeAnalysis
[BG(1)] parsing: c:\Users\cires\Documents\GitHub\stock_trader\docs\conf.py [fs read 0ms] (1ms)
[BG(1)] binding: c:\Users\cires\Documents\GitHub\stock_trader\docs\conf.py (0ms)
[BG(1)] indexing: c:\Users\cires\Documents\GitHub\stock_trader\docs\conf.py [found 6] (0ms)
Indexing Done: c:\Users\cires\Documents\GitHub\stock_trader\docs\conf.py
[BG(1)] parsing: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\Quote.py [fs read 0ms] (1ms)
[BG(1)] binding: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\Quote.py (0ms)
[BG(1)] indexing: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\Quote.py [found 4] (0ms)
Indexing Done: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\Quote.py
[BG(1)] parsing: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\__init__.py [fs read 1ms] (1ms)
[BG(1)] binding: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\__init__.py (0ms)
[BG(1)] indexing: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\__init__.py [found 0] (0ms)
Indexing Done: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\__init__.py
[BG(1)] parsing: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\__main__.py [fs read 0ms] (1ms)
[BG(1)] binding: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\__main__.py (0ms)
[BG(1)] indexing: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\__main__.py [found 2] (0ms)
Indexing Done: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\__main__.py
[BG(1)] parsing: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\broker_protocols.py [fs read 1ms] (4ms)
[BG(1)] binding: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\broker_protocols.py (1ms)
[BG(1)] indexing: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\broker_protocols.py [found 2] (0ms)
Indexing Done: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\broker_protocols.py
[BG(1)] indexing: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\invariant_type_change.py [found 6] (0ms)
Indexing Done: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\invariant_type_change.py
[BG(1)] parsing: c:\Users\cires\Documents\GitHub\stock_trader\tests\__init__.py [fs read 1ms] (1ms)
[BG(1)] binding: c:\Users\cires\Documents\GitHub\stock_trader\tests\__init__.py (0ms)
[BG(1)] indexing: c:\Users\cires\Documents\GitHub\stock_trader\tests\__init__.py [found 0] (0ms)
Indexing Done: c:\Users\cires\Documents\GitHub\stock_trader\tests\__init__.py
[BG(1)] parsing: c:\Users\cires\Documents\GitHub\stock_trader\tests\test_main.py [fs read 0ms] (2ms)
[BG(1)] binding: c:\Users\cires\Documents\GitHub\stock_trader\tests\test_main.py (1ms)
[BG(1)] indexing: c:\Users\cires\Documents\GitHub\stock_trader\tests\test_main.py [found 2] (0ms)
Indexing Done: c:\Users\cires\Documents\GitHub\stock_trader\tests\test_main.py
Background analysis message: getDiagnosticsForRange
Background analysis message: getDiagnosticsForRange
[FG] parsing: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\invariant_type_change.py (139ms)
[FG] parsing: c:\Users\cires\.vscode\extensions\ms-python.vscode-pylance-2021.8.2-pre.1\dist\typeshed-fallback\stdlib\builtins.pyi [fs read 1ms] (77ms)
[FG] binding: c:\Users\cires\.vscode\extensions\ms-python.vscode-pylance-2021.8.2-pre.1\dist\typeshed-fallback\stdlib\builtins.pyi (23ms)
[FG] binding: c:\Users\cires\Documents\GitHub\stock_trader\src\stock_trader\invariant_type_change.py (0ms)

Code Snippet / Additional information

from dataclasses import dataclass
from icontract import invariant


class TestClass1:
    test_int: int = 1


reveal_type(TestClass1)
# According to PyLance: Type[TestClass1]
# According to mypy: src\stock_trader\invariant_type_change.py:9:13: note: Revealed type is "def () -> stock_trader.invariant_type_change.TestClass1"

@invariant(lambda self: self.test_int == 1)
class TestClass2:
    test_int: int = 1


reveal_type(TestClass2)
# According to PyLance: type
# According to mypy: src\stock_trader\invariant_type_change.py:18:13: note: Revealed type is "def () -> stock_trader.invariant_type_change.TestClass2"


@dataclass
class TestClass3:
    test_class4: TestClass1
    test_class5: TestClass2 # PyLance states: Expected "class" type but received type


instance_of_TestClass1 = TestClass1()
instance_of_TestClass2 = TestClass2()

print(type(instance_of_TestClass1))
# <class '__main__.TestClass1'>
print(type(instance_of_TestClass2))
# <class '__main__.TestClass2'>

print(instance_of_TestClass1.test_int)
print(instance_of_TestClass2.test_int) # PyLance states test_int is of type Any

class_holder = TestClass3(
    test_class4=instance_of_TestClass1, test_class5=instance_of_TestClass2
)
print(class_holder.test_class4.test_int)
print(class_holder.test_class5.test_int)
"""When trying to print class_holder.test_class5.test_int, PyLance has the following 3 complaints:
Type of "test_int" is unknown.
Argument type is unknown.
Cannot access member "test_int" for type "type"
"""
@erictraut
Copy link
Contributor

This looks like it's a bug in the type annotations within the icontract library.

The decorator invariant is defined in icontract as a class with a __call__ method. This method returns type, which obscures the type once the decorator is applied.

    def __call__(self, cls: type) -> type:

The correct type annotation for this method is:

        def __call__(self, cls: _T) -> _T:

where `_T is defined as:

_T = TypeVar("_T", bound=type)

This preserves the type of the decorated class. If I make this change, it type checks fine in pylance and in mypy.

I recommend filing a bug or submitting a PR in the icontract repo.

@jakebailey jakebailey added the waiting for upstream Waiting for upstream to release a fix label Aug 12, 2021
@github-actions github-actions bot removed the triage label Aug 12, 2021
@ciresnave
Copy link
Author

I worked with the developer behind icontract and he and I verified a fix on his side is possible (see his pull request #227). I still question why Python itself plus mypy had no issue with the looser typing but PyLance did. Either way, this may be solved by a change in PyLance.

@jakebailey
Copy link
Member

jakebailey commented Aug 12, 2021

Python doesn't care about annotations at runtime, so you could put anything there and it will run.

As for mypy, that's hard to say. It could be a bug they haven't fixed or had reported to them.

@erictraut
Copy link
Contributor

This appears to be a bug in mypy. They are not honoring the __call__ method when applying a class decorator. I'll file a bug in the mypy repo.

@erictraut
Copy link
Contributor

It looks like this is a known bug in mypy. It has been open for more than four years: python/mypy#3135.

Here's an sample that demonstrates how pyright (the type checker that underlies pylance) applies the class decorator whereas mypy does not.

from typing import Callable, TYPE_CHECKING, Type

class replace_with_int1:
    def __init__(self):
        ...

    def __call__(self, cls: type) -> Type[int]:
        return int

@replace_with_int1()
class NotInt1:
    ...

if TYPE_CHECKING:
    reveal_type(NotInt1)  # pyright: "Type[int]"", mypy: "def () -> NotInt1"
print(NotInt1) # <class 'int'>

def replace_with_int2() -> Callable[..., Type[int]]:
    return lambda x: int

@replace_with_int2()
class NotInt2:
    ...

if TYPE_CHECKING:
    reveal_type(NotInt2)  # pyright: "Type[int]"", mypy: "def () -> NotInt2"
print(NotInt2) # <class 'int'>

@jakebailey
Copy link
Member

I'm going to close this as needing a fix upstream; we're doing the right thing according to the type annotations here, so there's no work to do.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
waiting for upstream Waiting for upstream to release a fix
Projects
None yet
Development

No branches or pull requests

3 participants