TypeVar bound with Optional checking does not work any more

[darkclouder]

Bug Report
I use a pattern where I bind a TypeVar to an optional so I can use a function to return either an optional or non-optional depending on what is passed to the function:
MaybeParent = TypeVar("MaybeParent", bound=Optional[Parent]).

So with a function with this signature try_copy(parent: MaybeParent) -> MaybeParent, the function returns a Parent if I pass a Parent and returns a Optional[Parent] if a pass a Optional[Parent].
So I can have a function which can handle optionals but explicitly returns a non-optional if I pass a non-optional.

This used to pass mypy with version 0.812 but now with 0.942, it yields an error as it does not manage to extract the optional type (any more) if you check for it first.
And IMO the typing is correct and it should therefore pass.

To Reproduce

1. pip install mypy==0.942
2. Check this code with mypy:

from typing import Optional, TypeVar

class Parent:
    def copy(self):
        return self

MaybeParent = TypeVar("MaybeParent", bound=Optional[Parent])

def try_copy_works(parent: Optional[Parent]) -> Optional[Parent]:
    if parent is None:
        return None
    return parent.copy()

def try_copy_doesnt_work(parent: MaybeParent) -> MaybeParent:
    if parent is None:
        return
    # Item "object" of the upper bound "Optional[Parent]" of type variable "MaybeParent" has no attribute "copy"
    return parent.copy()

def try_copy_doesnt_work_either(parent: MaybeParent) -> MaybeParent:
    if parent is None:
        return
    # Incompatible types in assignment (expression has type "MaybeParent", variable has type "Parent")
    guarded_parent: Parent = parent
    return guarded_parent.copy()

Expected Behavior
Should pass type checking as MaybeParent is bound with None | Parent.
Since I check None in the beginning, the variable should be bound with Parent.

Actual Behavior

Does not pass type checking:
Item "object" of the upper bound "Optional[Parent]" of type variable "MaybeParent" has no attribute "copy"
Incompatible types in assignment (expression has type "MaybeParent", variable has type "Parent")

Your Environment

• Mypy version used: 0.942
• Mypy command-line flags: none
• Python version used: 3.8.13
• Operating system and version: macOS 11.6.5

[darkclouder]

I just realised that parent.copy() returns a Parent, however MaybeParent is not None | Parent but BOUND to None | Parent. That means if you have class Child(Parent) and you pass a Child object, you might get a Parent in return, not a Child.
However, then the error is still wrong, because the Child should still have a copy function.
It should complain about an incompatible return type instead.

Btw, if someone has a better idea how to tell mypy "this function can take an optional or a non-optional and it will return exactly that again what you passed into the function", I'd be happy to hear suggestions.

[wrobell]

@darkclouder It seems you need typing.overload.

[bbatliner]

@overload is a good idea. Something like this?

from typing import Optional, TypeVar, overload

class Parent:
    def copy(self):
        return self

class Child(Parent):
    pass

ParentT = TypeVar("ParentT", bound=Parent)

@overload
def try_copy_works(parent: None) -> None: ...
@overload
def try_copy_works(parent: ParentT) -> ParentT: ...
def try_copy_works(parent: Optional[ParentT]) -> Optional[ParentT]:
    if parent is None:
        return None
    return parent.copy()


reveal_type(try_copy_works(None))  # None
reveal_type(try_copy_works(Parent()))  # Parent
reveal_type(try_copy_works(Child()))  # Child

Looks like #9424 is a very similar issue as this one, and also suggests using overload as a workaround.

[darkclouder]

Yes, thanks. Overload seems to be the thing I need.

Coming back to the issue though: The error by mypy is still wrong, right? After type narrowing with the None check, MaybeParent should be bound to be Parent and therefore have copy().

[bbatliner]

I think you're right, and is what #9424 is trying to capture. Comparing Pylance type narrowing to mypy in your example:

def try_copy_doesnt_work(parent: MaybeParent) -> MaybeParent:
    if parent is None:
        reveal_type(parent)  # Pylance: Type of "parent" is "None"
        return None  # mypy: statement is unreachable
    reveal_type(parent)  # Pylance: Type of "parent" is "Parent"
                         # mypy: Type of "parent" is "MaybeParent`-1"
    return parent.copy()  # mypy: Item "object" of the upper bound "Optional[Parent]" of type variable "MaybeParent" has no attribute "copy"

Clearly mypy is failing to narrow the typevar with the None check. I don't see any root cause solutions discussed on that issue except for workarounds.

Your example is nearly identical to #9424 (comment):

U = TypeVar('U', bound=Optional[int])

def sleep(x: int) -> None: ...

def f(u: U) -> U:
    if u is None:
        return None
    sleep(u)
    return u

Shall we close as duplicate?

[darkclouder]

Yup, seems to be a duplicate of that. Thanks for finding that.