Avoid type size explosion when expanding types #17842
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
If TypedDict A has multiple items that refer to TypedDict B, don't
duplicate the types representing B during type expansion (or generally
when translating types). If TypedDicts are deeply nested, this could
result in lot of redundant type objects.
Example where this could matter (assume B is a big TypedDict):
```
class A(TypedDict):
a: B
b: B
c: B
...
z : B
```
Also deduplicate large unions. It's common to have aliases
that are defined as large unions, and again we want to avoid
duplicating these unions.
|
According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅ |
jhance
approved these changes
Sep 27, 2024
|
I ran this commit against mypy_primer with at compile level 3 using an average of 5 runs This made https://github.com/vega/altair 6.4x faster, going from 110s to 17s! (all other meaningful projects are in the 0.95x to 1.05x range) |
|
Nice, thanks for looking into the perf impact! And I think we still have a bunch of improvements we can make to very large types. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
If TypedDict A has multiple items that refer to TypedDict B, don't duplicate the types representing B during type expansion (or generally when translating types). If TypedDicts are deeply nested, this could result in lot of redundant type objects.
Example where this could matter (assume B is a big TypedDict):
Also deduplicate large unions. It's common to have aliases that are defined as large unions, and again we want to avoid duplicating these unions.
This may help with #17231, but this fix may not be sufficient.