Add documentation for annotating descriptors

[0xTowel]

Documentation

Based on the discussions in #2266 and #244, it seems MyPy supports annotations for descriptors, but there doesn't seem to be anything in the documentation about this or how to properly annotate them.

Annotating these isn't obvious, and some best practice for common usages would be helpful.

For example, one common paradigm for generic data descriptors is to return self when called from a class (instance is None), and I'm struggling to find the best way to annotate that.

[gvanrossum]

Can you show some (unannotated) self-contained code for which you can't figure out how to annotate it, plus some things you've tried that didn't work? Maybe the docs really do need something specifically about this (even though in general I don't think we need to document for every Python construct how to annotate it -- usually if you annotate the types you're actually using everything should just work). Or maybe there's a reason you're struggling.

[0xTowel]

The code that was driving me to look up some sort of documentation was the following:

class Share():
    def __init__(self, initial_value=None):
        self.value = initial_value
        self._name = None

    def __get__(self, instance, owner):
        if instance is None:
            return self
        if self.value is None:
            raise AttributeError(f'{self._name} uninitialized')
        return self.value

    def __set__(self, instance, value):
        self.value = value

    def __set_name__(self, owner, name):
        self._name = name

After some considerable digging and question asking I came up with a solution that works but still requires an annotation during the creation:

T = TypeVar('T')
V = TypeVar('V')

class Share(Generic[T, V]):
    def __init__(self, initial_value: Optional[V] = None) -> None:
        self.value = initial_value
        self._name: Optional[str] = None

    @overload
    def __get__(self, instance: None, owner: Type[T]) -> Share: ...
    @overload
    def __get__(self, instance: T, owner: Type[T]) -> V: ...  # noqa: F811

    def __get__(self, instance: Optional[T], owner: Type[T]) -> Union[Share, V]:  # noqa: F811
        if instance is None:
            return self
        if self.value is None:
            raise AttributeError(f'{self._name} uninitialized')
        return self.value

    def __set__(self, instance: T, value: V) -> None:
        self.value = value

    def __set_name__(self, owner: Type[T], name: str) -> None:
        self._name = name

Even this isn't the most satisfying since it is still easy to silently mess up the type:

class ExampleUsage:
    descriptor: Share = Share(0)  # descriptor: Share[ExampleUsage, int] would be the fix

    def __init__(self) -> None:
        self.descriptor += 'a'  # Type error won't be caught

Any suggestions in this example would be much appreciated.

Maybe it's just my naivete, but given the complexity of the solution contrasted with the simplicity of the unannotated original I was surprised there was no mention of descriptors anywhere in the documentation. Classes, coroutines, decorators, metaclasses, even regular expressions all have a mention somewhere and I'd say are somewhat similar.

I also don't want to point the ship at documenting every Python construct and how to annotate it, but having at least a quick mention of descriptors or a best practice example would've saved some time and made this clearer.

[gvanrossum]

I'm guessing the problem is that your decorator is generic in T and V, but the constructor only specifies V, so a type annotation is required, and that's easily messed up (since Shape defaults to Shape[Any, Any]).

Could you make a type alias for specific classes, e.g.

ExampleShare = Share['ExampleUsage', V]

class ExampleUsage:
   descriptor = ExampleShare(0)

If that works we could recommend that in the docs (maybe in the cheat sheets).

[0xTowel]

Sadly no, as we won't catch type errors if we use multiple types:

V = TypeVar('V')
ExampleShare = Share['ExampleUsage', V]


class ExampleUsage:
    descriptor_a = ExampleShare(0)
    descriptor_b = ExampleShare('A')

    def __init__(self) -> None:
        self.descriptor_a += 'a'  # Should be invalid
        self.descriptor_b += 1    # Should also be invalid

(I also find it a little unwieldy to need to bring in TypeVar to annotate something as generic if it was already specified as generic in its initial definition, but if it works then I'll take it.)

[gvanrossum]

Too bad that doesn't work. All I can offer is this:

class ExampleShare(Share['ExampleUsage', V]): pass

Then it reports the bugs. But this is pretty unwieldy, I agree. :-(

[hauntsaninja]

I seem to recall some discussion of defaults for TypeVars recently, maybe that would help here. Additionally, --disallow-any-generics would help with the silently messing up the type, again at the cost of verbosity.