Generic classes cause massive slowdown or hang

[RobertCraigie]

Hi, this commit, causes mypy to go from ~7 minutes to ~30 minutes to typecheck the whole codebase.

It should be noted that the code in the mentioned commit is used to auto-generate 5 classes, each with 9 methods that use the generic typevar. These methods don't do anything special but the arguments are heavily nested TypedDicts. The classes look like this:

from pydantic import BaseModel

BaseModelT = TypeVar('BaseModelT', bound=BaseModel)

class UserActions(Generic[BaseModelT]):
    async def create(
        self,
        data: 'UserCreateInput',
    ) -> BaseModelT:
        ...
    ...

If it is still relevant, this #3796 (comment) might explain some of the reasons behind this performance loss.

I'm sorry I don't have a smaller example, I don't have enough time to dig any deeper into this issue.

[RobertCraigie]

I actually cannot run this on my machine, left it running for ~2 hours with no result.

[hauntsaninja]

Is there a standalone Python file that is slow to type check? I also see you have multiple mypy plugins. Does disabling them help?

[RobertCraigie]

@hauntsaninja I think the worst offender is the code generated from https://github.com/RobertCraigie/prisma-client-py/blob/main/src/prisma/generator/templates/types.py.jinja

On mobile right now but will share an example of the generated file when possible and check if disabling the plugins helps.

The auto-generated file is basically a lot of nested and pseudo-recursive TypedDicts, I think the last time I checked the size of the mypy cache for this file was around 400MB, that could be the issue?

[RobertCraigie]

generated types.py file: https://mystb.in/EvilPlanesPain.python

stand alone types.py file: https://mystb.in/ConsistencySlightAerial.python

[RobertCraigie]

Mypy was slightly faster without the plugins, but still taking nearly 28 minutes

https://github.com/RobertCraigie/prisma-client-py/runs/4273696907?check_suite_focus=true

[RobertCraigie]

The stand alone types.py file does not seem to be the root cause of this issue, type checking took around a minute:

https://github.com/RobertCraigie/prisma-client-py/runs/4280666648?check_suite_focus=true

[Fulguritude]

I'm in a very similar situation, and can confirm that generics and nested dicts/type aliases do not mesh well currently. I recently started to add a bunch of genericity to refactor some of our code. I've tried running mypy for more than 15 minutes, and mypy still hadn't returned its errors; so I was thinking this was possibly an issue on my end - until I came across this thread. Clearly it's taking an ungodly long time, compared to the usual minute or so that it takes to check our "non-generic" codebase.

[hauntsaninja]

Did you ever manage to make a standalone repro? We've made a series of recent performance fixes.

[RobertCraigie]

I did not manage to create a standalone repository but I have been able to add the generic types back in and running mypy with v1.4.1 is much faster. It now seems to be between 15-20s locally which is insane, thank you!