Fix performance in union subtyping #15104
Conversation
This is a performance optimisation for subtyping between two unions that are largely the same. Fixes python#14034
|
This is amazing! It's a huge (>99%) improvement. |
|
According to mypy_primer, this change has no effect on the checked open source code. 🤖🎉 |
|
Massive improvement, I compiled this locally and was able to type check the main branches of pydantic and pydantic-core (with our For comparison, mypy 1.2.0 with This is fast enough that it is viable once again for us to use mypy again for type-checking pydantic, but we'd definitely appreciate any further performance improvements that might be possible from |
|
Awesome, thank you so much. |
This is a performance optimisation for subtyping between two unions that are largely the same. Fixes #14034 This makes @adriangb's example in #14034 (comment) finish basically instantly. I could add it as a unit test? Type checking pydantic core is still not fast — takes like four or five minutes with uncompiled mypy — but at least it's now feasible. I think there's room for doing some optimisation in make_simplified_union that would improve this.
The following code optimises make_simplified_union in the common case that there are exact duplicates in the union. In this regard, this is similar to python#15104 To get this to work, I needed to use partial tuple fallbacks in a couple places (these maybe had the potential to be latent crashes anyway?) There were some interesting things going on with recursive type aliases and type state assumptions This is about a 25% speedup on the pydantic codebase and about a 2% speedup on self check (measured with uncompiled mypy)
Fixes #15192 The following code optimises make_simplified_union in the common case that there are exact duplicates in the union. In this regard, this is similar to #15104 There's a behaviour change in one unit test. I think it's good? We'll see what mypy_primer has to say. To get this to work, I needed to use partial tuple fallbacks in a couple places. These could cause crashes anyway. There were some interesting things going on with recursive type aliases and type state assumptions This is about a 25% speedup on the pydantic codebase and about a 2% speedup on self check (measured with uncompiled mypy)
This is a performance optimisation for subtyping between two unions that are largely the same.
Fixes #14034
This makes @adriangb's example in #14034 (comment) finish basically instantly. I could add it as a unit test?
Type checking pydantic core is still not fast — takes like four or five minutes with uncompiled mypy — but at least it's now feasible. I think there's room for doing some optimisation in make_simplified_union that would improve this.