Plugins OR mypy integration OR ideas for custom classes: Pydantic support

[tiangolo]

First, I just re-discovered Pyright a couple of days ago, and it's been amazing! ✨ 🍰

GREAT JOB with it, thanks for building it (I've been tweeting about it all day 😅 ).

I had heard about it before, but I didn't pay much attention assuming it was just a "type checker", and I thought "Mypy does that for me". But now I'm seeing that it can do autoimports, dead code coloring, autocompletion, and even use Type stubs for completion (I'm using sqlalchemy-stubs), etc.

I'm really happy with it ✨ And I really like that it's made with TypeScript and not C#, so I think I could understand it and help at some point... at least the language itself is not a barrier.

---

Now to the feature request:

Is your feature request related to a problem? Please describe.

My final target use case is, I would like to have autocompletion for Pydantic models. I built FastAPI and it's tightly integrated with Pydantic, and I would love to be able to have completion for Pydatnic models in VS Code.

More details

This would be a case similar to support for Attrs or SQLAlchemy. The 3 libraries (and others, including most ORMs) create a dynamic __init__ based on class attributes.

All these libraries work in a somewhat similar style to dataclasses, for different purposes.

I read other issues and I see that there's already support with custom handling for dataclasses, I would love if there was a way to extend that for other libraries.

I know you would favor following standards, but from all the specs/PEPs I've read, I don't think there's yet a "standard" way to declare this type of behavior as with dataclasses. Creating a dynamic __init__ from the declared class attributes.

Describe the solution you'd like

A clear and concise description of what you want to happen.

Some options I could think of:

• Pydantic already has a Mypy plugin, if there was a way to request a signature to Mypy, then that could be an option. But I'm not sure if it can be done (I know Mypy internals and plugins are not very well documented).

• Maybe if there was a way to somehow annotate a class to mark it as behaving like dataclasses to generate __init__s from class attributes, maybe with a custom extra attribute in the class, or a comment, that could work.

• Or maybe Pyright could call some predefined classmethod or similar in the class to request __init__ signature data. In that case, I could probably add that to Pydantic.

• Or if there was a way to plug into Pyright and add custom logic, or extract the information it's already producing to extend it with custom logic, maybe that could work too. I have never built a language server nor a VS Code extension for it, but I guess it could potentially be done.

Additional context
Add any other context or screenshots about the feature request here.

I asked something similar in the official VS code Python language server: microsoft/python-language-server#1898

But there I can only be a needy user, I can't help much as I don't know C#. But here, at least I know TypeScript. I'm pretty sure the codebase must be complex, but at least the language is not a barrier, so I think I could help/hack around a bit.

[patrick91]

+1 one for this.

I think it might be interesting to allow to provide plugins from external libraries (similar to what MyPy has). It might also be interesting to auto-discover the plugins, so it would work out of the box when installing a library with a pyright plugin

I'm working on Strawberry which is based on data classes, but pyright does not work well with it[1]. I'd be happy to provide a plugin for it :)

[1] Same goes for mypy, we had to make a simple plugin for it.

[tiangolo]

I just found some extra info that I think might be useful for this at some point later.

It might be totally irrelevant as maybe it doesn't make sense to ask stuff to mypy from Pyright, but anyway, here's what I found.

---

Mypy is indeed adding a way to "query" for signatures.

It's part of the daemon version dmypy that is designed to provide suggestions while adding types.

https://mypy.readthedocs.io/en/stable/mypy_daemon.html?highlight=dmypy#static-inference-of-annotations

To be able to use it with Pydantic it needs the new --follow-imports=normal from python/mypy#8654

But as of now, it is only available in master (no release has it yet): python/mypy#5870 (comment)

---

To test what is currently available in dmypy:

I created a file sand.py with:

from pydantic import BaseModel


class Model(BaseModel):
    title: str
    description: str
    age: int
    date: str


def hello_world(*, name: str, age: int) -> None:
    print("Hello")

Model(title="John")

A file mypy.ini:

[mypy]
plugins = pydantic.mypy

Then:

$ python -m pip install -U git+git://github.com/python/mypy.git

$ dmypy start -- sand.py

$ dmypy check sand.py

sand.py:14: error: Missing named argument "description" for "Model"
sand.py:14: error: Missing named argument "age" for "Model"
sand.py:14: error: Missing named argument "date" for "Model"
Found 3 errors in 1 file (checked 76 source files)

Notice the right types for this function, str and int, but no keyword names 😞

$ dmypy suggest sand.hello

(str, int) -> None

Here we get the right number of args, self plus all the model fields, this comes from Pydantic's mypy plugin, without it, __init__ would be "unknown".

We still don't have the correct types yet, but at least some info. Maybe we can add types in Pydantic's mypy plugin side.

$ dmypy suggest sand.Model.__init__

(Any, Any, Any, Any, Any) -> None

This is still missing the keyword argument names, but maybe there's something around here that can be done.

[erictraut]

I've spent some time investigating the possibility of using existing mypy plugins. Unfortunately, that's not going to be possible. The extension interface for mypy is not well enough defined and is very tied to the internal implementation of mypy. It would be next to impossible to provide any level of compatibility with such extensions.

Creating a custom extension model for Pyright is something we might eventually do, but it's not going to happen any time soon. And I'm hopeful that it won't be necessary in the long run. We are focusing efforts on improving the standard type system capabilities in Python and working with library authors to build interfaces (and provide stubs) that conform to those standards. That will eliminate the need for custom plugins over time and is a much better and scalable solution than requiring each smart editor, linter, and static type checker to know about non-standard special-case behaviors.

I'm going to close this feature request for now.

[erictraut]

We have been exploring ways to provide better support for pydantic and other libraries that support semantics similar to dataclass. Pylance version 2021.4.2 (released earlier today) contains a proposed solution that is documented in this spec. If you want to try it, I've included instructions on how to modify your local copy of attrs or pydantic source code. We welcome early feedback.

I've started a discussion in the pyright repo. Please post questions, comments, or feedback there.

[tiangolo]

That's awesome! 🎉 I already commented there and made a first PR to pydantic. I'll continue any conversation there.

Thanks for this! 🙇☕