[ty] Validate writes to TypedDict keys
#19782
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sharkdp
commented
Aug 6, 2025
| NAME_FINAL: Final = "name" | ||
| AGE_FINAL: Final[Literal["age"]] = "age" | ||
|
|
||
| def _(person: Person): |
Contributor
Author
There was a problem hiding this comment.
I split these tests into separate _ functions because I didn't want assignment-based place narrowing to interfere somehow.
Contributor
Diagnostic diff on typing conformance testsNo changes detected when running ty on typing conformance tests ✅ |
sharkdp
commented
Aug 6, 2025
| Some(diagnostic) | ||
| } | ||
|
|
||
| pub(crate) fn report_invalid_key_on_typed_dict<'db>( |
Contributor
Author
There was a problem hiding this comment.
This code has just been moved to a separate function.
Contributor
|
sharkdp
force-pushed
the
david/typeddict-setitem
branch
from
cc8a70d to
8bb7069
Compare
Contributor
|
| Lint rule | Added | Removed | Changed |
|---|---|---|---|
unused-ignore-comment |
0 | 6 | 0 |
invalid-key |
4 | 0 | 0 |
invalid-assignment |
3 | 0 | 0 |
| Total | 7 | 6 | 0 |
sharkdp
force-pushed
the
david/typeddict-setitem
branch
from
8bb7069 to
4021823
Compare
sharkdp
requested review from
AlexWaygood,
MichaReiser,
carljm and
dcreager
as code owners
carljm
approved these changes
Aug 6, 2025
| def _(person: Person, union_of_keys: Literal["name", "surname"]): | ||
| person[union_of_keys] = "unknown" | ||
|
|
||
| # error: [invalid-assignment] "Cannot assign value of type `Literal[1]` to key of type `Literal["name", "surname"]` on TypedDict `Person`" |
Contributor
There was a problem hiding this comment.
I feel like ideally we would tell you which key(s) actually failed to assign, if only some of them do. But I see that that would add significant complexity to the implementation (we'd have to dig into the CallError details), so I'm fine not doing it for now and seeing if it's an issue in practice
Contributor
|
Going ahead and merging since @sharkdp is probably already done for the day and will be OOO for a bit. |
dcreager
added a commit
that referenced
this pull request
Aug 7, 2025
* origin/main: [ty] Implemented support for "rename" language server feature (#19551) [ty] Reduce size of member table (#19572) [ty] Move server capabilities creation (#19798) [ty] Repurpose `FunctionType.into_bound_method_type` to return `BoundMethodType` (#19793) [ty] Validate writes to `TypedDict` keys (#19782) [ty] Add support for using the test command emitted when a mdtest fails (#19794)
sharkdp
added a commit
that referenced
this pull request
Aug 25, 2025
## Summary Implement validation for `TypedDict` constructor calls and dictionary literal assignments, including support for `total=False` and proper field management. Also add support for `Required` and `NotRequired` type qualifiers in `TypedDict` classes, along with proper inheritance behavior and the `total=` parameter. Support both constructor calls and dict literal syntax part of astral-sh/ty#154 ### Basic Required Field Validation ```py class Person(TypedDict): name: str age: int | None # Error: Missing required field 'name' in TypedDict `Person` constructor incomplete = Person(age=25) # Error: Invalid argument to key "name" with declared type `str` on TypedDict `Person` wrong_type = Person(name=123, age=25) # Error: Invalid key access on TypedDict `Person`: Unknown key "extra" extra_field = Person(name="Bob", age=25, extra=True) ``` <img width="773" height="191" alt="Screenshot 2025-08-07 at 17 59 22" src="https://github.com/user-attachments/assets/79076d98-e85f-4495-93d6-a731aa72a5c9" /> ### Support for `total=False` ```py class OptionalPerson(TypedDict, total=False): name: str age: int | None # All valid - all fields are optional with total=False charlie = OptionalPerson() david = OptionalPerson(name="David") emily = OptionalPerson(age=30) frank = OptionalPerson(name="Frank", age=25) # But type validation and extra fields still apply invalid_type = OptionalPerson(name=123) # Error: Invalid argument type invalid_extra = OptionalPerson(extra=True) # Error: Invalid key access ``` ### Dictionary Literal Validation ```py # Type checking works for both constructors and dict literals person: Person = {"name": "Alice", "age": 30} reveal_type(person["name"]) # revealed: str reveal_type(person["age"]) # revealed: int | None # Error: Invalid key access on TypedDict `Person`: Unknown key "non_existing" reveal_type(person["non_existing"]) # revealed: Unknown ``` ### `Required`, `NotRequired`, `total` ```python from typing import TypedDict from typing_extensions import Required, NotRequired class PartialUser(TypedDict, total=False): name: Required[str] # Required despite total=False age: int # Optional due to total=False email: NotRequired[str] # Explicitly optional (redundant) class User(TypedDict): name: Required[str] # Explicitly required (redundant) age: int # Required due to total=True bio: NotRequired[str] # Optional despite total=True # Valid constructions partial = PartialUser(name="Alice") # name required, age optional full = User(name="Bob", age=25) # name and age required, bio optional # Inheritance maintains original field requirements class Employee(PartialUser): department: str # Required (new field) # name: still Required (inherited) # age: still optional (inherited) emp = Employee(name="Charlie", department="Engineering") # ✅ Employee(department="Engineering") # ❌ e: Employee = {"age": 1} # ❌ ``` <img width="898" height="683" alt="Screenshot 2025-08-11 at 22 02 57" src="https://github.com/user-attachments/assets/4c1b18cd-cb2e-493a-a948-51589d121738" /> ## Implementation The implementation reuses existing validation logic done in #19782 ### ℹ️ Why I did NOT synthesize an `__init__` for `TypedDict`: `TypedDict` inherits `dict.__init__(self, *args, **kwargs)` that accepts all arguments. The type resolution system finds this inherited signature **before** looking for synthesized members. So `own_synthesized_member()` is never called because a signature already exists. To force synthesis, you'd have to override Python’s inheritance mechanism, which would break compatibility with the existing ecosystem. This is why I went with ad-hoc validation. IMO it's the only viable approach that respects Python’s inheritance semantics while providing the required validation. ### Refacto of `Field` **Before:** ```rust struct Field<'db> { declared_ty: Type<'db>, default_ty: Option<Type<'db>>, // NamedTuple and dataclass only init_only: bool, // dataclass only init: bool, // dataclass only is_required: Option<bool>, // TypedDict only } ``` **After:** ```rust struct Field<'db> { declared_ty: Type<'db>, kind: FieldKind<'db>, } enum FieldKind<'db> { NamedTuple { default_ty: Option<Type<'db>> }, Dataclass { default_ty: Option<Type<'db>>, init_only: bool, init: bool }, TypedDict { is_required: bool }, } ``` ## Test Plan Updated Markdown tests --------- Co-authored-by: David Peter <mail@david-peter.de>
second-ed
pushed a commit
to second-ed/ruff
that referenced
this pull request
Sep 9, 2025
## Summary Implement validation for `TypedDict` constructor calls and dictionary literal assignments, including support for `total=False` and proper field management. Also add support for `Required` and `NotRequired` type qualifiers in `TypedDict` classes, along with proper inheritance behavior and the `total=` parameter. Support both constructor calls and dict literal syntax part of astral-sh/ty#154 ### Basic Required Field Validation ```py class Person(TypedDict): name: str age: int | None # Error: Missing required field 'name' in TypedDict `Person` constructor incomplete = Person(age=25) # Error: Invalid argument to key "name" with declared type `str` on TypedDict `Person` wrong_type = Person(name=123, age=25) # Error: Invalid key access on TypedDict `Person`: Unknown key "extra" extra_field = Person(name="Bob", age=25, extra=True) ``` <img width="773" height="191" alt="Screenshot 2025-08-07 at 17 59 22" src="https://github.com/user-attachments/assets/79076d98-e85f-4495-93d6-a731aa72a5c9" /> ### Support for `total=False` ```py class OptionalPerson(TypedDict, total=False): name: str age: int | None # All valid - all fields are optional with total=False charlie = OptionalPerson() david = OptionalPerson(name="David") emily = OptionalPerson(age=30) frank = OptionalPerson(name="Frank", age=25) # But type validation and extra fields still apply invalid_type = OptionalPerson(name=123) # Error: Invalid argument type invalid_extra = OptionalPerson(extra=True) # Error: Invalid key access ``` ### Dictionary Literal Validation ```py # Type checking works for both constructors and dict literals person: Person = {"name": "Alice", "age": 30} reveal_type(person["name"]) # revealed: str reveal_type(person["age"]) # revealed: int | None # Error: Invalid key access on TypedDict `Person`: Unknown key "non_existing" reveal_type(person["non_existing"]) # revealed: Unknown ``` ### `Required`, `NotRequired`, `total` ```python from typing import TypedDict from typing_extensions import Required, NotRequired class PartialUser(TypedDict, total=False): name: Required[str] # Required despite total=False age: int # Optional due to total=False email: NotRequired[str] # Explicitly optional (redundant) class User(TypedDict): name: Required[str] # Explicitly required (redundant) age: int # Required due to total=True bio: NotRequired[str] # Optional despite total=True # Valid constructions partial = PartialUser(name="Alice") # name required, age optional full = User(name="Bob", age=25) # name and age required, bio optional # Inheritance maintains original field requirements class Employee(PartialUser): department: str # Required (new field) # name: still Required (inherited) # age: still optional (inherited) emp = Employee(name="Charlie", department="Engineering") # ✅ Employee(department="Engineering") # ❌ e: Employee = {"age": 1} # ❌ ``` <img width="898" height="683" alt="Screenshot 2025-08-11 at 22 02 57" src="https://github.com/user-attachments/assets/4c1b18cd-cb2e-493a-a948-51589d121738" /> ## Implementation The implementation reuses existing validation logic done in astral-sh#19782 ### ℹ️ Why I did NOT synthesize an `__init__` for `TypedDict`: `TypedDict` inherits `dict.__init__(self, *args, **kwargs)` that accepts all arguments. The type resolution system finds this inherited signature **before** looking for synthesized members. So `own_synthesized_member()` is never called because a signature already exists. To force synthesis, you'd have to override Python’s inheritance mechanism, which would break compatibility with the existing ecosystem. This is why I went with ad-hoc validation. IMO it's the only viable approach that respects Python’s inheritance semantics while providing the required validation. ### Refacto of `Field` **Before:** ```rust struct Field<'db> { declared_ty: Type<'db>, default_ty: Option<Type<'db>>, // NamedTuple and dataclass only init_only: bool, // dataclass only init: bool, // dataclass only is_required: Option<bool>, // TypedDict only } ``` **After:** ```rust struct Field<'db> { declared_ty: Type<'db>, kind: FieldKind<'db>, } enum FieldKind<'db> { NamedTuple { default_ty: Option<Type<'db>> }, Dataclass { default_ty: Option<Type<'db>>, init_only: bool, init: bool }, TypedDict { is_required: bool }, } ``` ## Test Plan Updated Markdown tests --------- Co-authored-by: David Peter <mail@david-peter.de>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Validates writes to
TypedDictkeys, for example:The new specialized
invalid-assignmentdiagnostic looks like this:Ecosystem analysis
As far as I can tell, all true positives!
There are some extremely long diagnostic messages. We should truncate our display of overload sets somehow.
Test Plan
New Markdown tests