Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update typeshed #17409

Merged
merged 5 commits into from
Jun 20, 2024
Merged

Update typeshed #17409

merged 5 commits into from
Jun 20, 2024

Conversation

cdce8p
Copy link
Collaborator

@cdce8p cdce8p commented Jun 20, 2024

The automatic sync failed due to a merge conflict.

Source commit:
python/typeshed@6dda799

This comment has been minimized.

@cdce8p cdce8p requested a review from AlexWaygood June 20, 2024 11:35
@AlexWaygood
Copy link
Member

AlexWaygood commented Jun 20, 2024

Ah, looks like you'll need to re-add the type: ignores that were added to stdlib/builtins.pyi in the vendored copy of typeshed in 7cb733a. (Possibly we could add them to the script to be applied automatically, but we'll add them to typeshed as soon as there's a mypy release that starts emitting that error, so it may not be worth the bother)

Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should fix CI:

mypy/typeshed/stdlib/builtins.pyi Outdated Show resolved Hide resolved
mypy/typeshed/stdlib/builtins.pyi Outdated Show resolved Hide resolved
@cdce8p cdce8p force-pushed the update-typeshed branch from 44e5bee to c681fcb Compare June 20, 2024 11:47
@cdce8p
Copy link
Collaborator Author

cdce8p commented Jun 20, 2024

Ah, looks like you'll need to re-add the type: ignores that were added to stdlib/builtins.pyi in the vendored copy of typeshed in 7cb733a. (Possibly we could add them to the script to be applied automatically, but we'll add them to typeshed as soon as there's a mypy release that starts emitting that error, so it may not be worth the bother)

Added a new patch to apply it automatically. We can remove it, after the next mypy release.

--
The new primer hits seem to be from python/typeshed#11929.

Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't double-checked whether the typeshed change to use TypeIs for is_dataclass is correct, but LGTM otherwise

This comment has been minimized.

@JelleZijlstra
Copy link
Member

I'd prefer to add the necessary type ignores to typeshed now so we don't have to maintain a patch here that we'll have to remember to remove later.

@cdce8p
Copy link
Collaborator Author

cdce8p commented Jun 20, 2024

I'd prefer to add the necessary type ignores to typeshed now so we don't have to maintain a patch here that we'll have to remember to remove later.

Removed the patch. Unfortunately, we'll have to wait for the next mypy release before we can add the type ignores to typeshed proper (I believe). So we just have to remember to apply them to each typeshed sync until the release happens. Might only be one or two more anyway.

@AlexWaygood
Copy link
Member

AlexWaygood commented Jun 20, 2024

Removed the patch. Unfortunately, we'll have to wait for the next mypy release before we can add the type ignores to typeshed proper (I believe).

We're actually unable to switch on MyPy's --warn-unused-ignore at typeshed due to some annoying special-casing mypy does if it detects a module is part of a "typeshed directory" (which is obviously always the case in the context of typeshed's CI). So we should be able to add these ignores to the typeshed main branch now. Normally I'd object to adding ignores that are unused (for now), but this is something of a special case

This comment has been minimized.

@cdce8p
Copy link
Collaborator Author

cdce8p commented Jun 20, 2024

We're actually unable to switch on MyPy's --warn-unused-ignore at typeshed due to some annoying special-casing mypy does if it detects a module is part of a "typeshed directory" (which is obviously always the case in the context of typeshed's CI). So we should be able to add these ignores to the typeshed main branch now. Normally I'd object to adding ignores that are unused (for now), but this is something of a special case

Didn't know that. Opened python/typeshed#12177 for it.

--
Anyway, we could merge this one as is. I already applied the change from the PR earlier.

cdce8p and others added 4 commits June 20, 2024 16:51
This is allegedly causing large performance problems, see 13821

typeshed/8231 had zero hits on mypy_primer, so it's not the worst thing
to undo. Patching this in typeshed also feels weird, since there's a
more general soundness issue. If a typevar has a bound or constraint, we
might not want to solve it to a Literal.

If we can confirm the performance regression or fix the unsoundness
within mypy, I might pursue upstreaming this in typeshed.

(Reminder: add this to the sync_typeshed script once merged)
@cdce8p cdce8p force-pushed the update-typeshed branch from 4d35b7a to a0b4232 Compare June 20, 2024 14:52
@cdce8p
Copy link
Collaborator Author

cdce8p commented Jun 20, 2024

Just to have a clean commit history here, I redid the typeshed sync with python/typeshed@6dda799.

Copy link
Contributor

Diff from mypy_primer, showing the effect of this PR on open source code:

anyio (https://github.com/agronholm/anyio)
+ src/anyio/_core/_fileio.py:362: error: Unused "type: ignore" comment  [unused-ignore]
+ src/anyio/_core/_fileio.py:366: error: Unused "type: ignore" comment  [unused-ignore]
+ src/anyio/_core/_fileio.py:371: error: Unused "type: ignore" comment  [unused-ignore]

pytest (https://github.com/pytest-dev/pytest)
+ src/_pytest/_io/pprint.py:117: error: Right operand of "and" is never evaluated  [unreachable]
+ src/_pytest/_io/pprint.py:125: error: Statement is unreachable  [unreachable]

pydantic (https://github.com/samuelcolvin/pydantic)
+ pydantic/v1/json.py:80: error: No overload variant of "asdict" matches argument type "type[DataclassInstance]"  [call-overload]
+ pydantic/v1/json.py:80: note: Possible overload variants:
+ pydantic/v1/json.py:80: note:     def asdict(obj: DataclassInstance) -> dict[str, Any]
+ pydantic/v1/json.py:80: note:     def [_T] asdict(obj: DataclassInstance, *, dict_factory: Callable[[list[tuple[str, Any]]], _T]) -> _T
- pydantic/deprecated/json.py:98: error: Unused "type: ignore" comment  [unused-ignore]

streamlit (https://github.com/streamlit/streamlit)
+ lib/streamlit/runtime/caching/hashing.py: note: In member "_to_bytes" of class "_CacheFuncHasher":
+ lib/streamlit/runtime/caching/hashing.py:409:34: error: No overload variant of "asdict" matches argument type "Type[DataclassInstance]"  [call-overload]
+ lib/streamlit/runtime/caching/hashing.py:409:34: note: Possible overload variants:
+ lib/streamlit/runtime/caching/hashing.py:409:34: note:     def asdict(obj: DataclassInstance) -> Dict[str, Any]
+ lib/streamlit/runtime/caching/hashing.py:409:34: note:     def [_T] asdict(obj: DataclassInstance, *, dict_factory: Callable[[List[Tuple[str, Any]]], _T]) -> _T

hydra-zen (https://github.com/mit-ll-responsible-ai/hydra-zen)
- src/hydra_zen/structured_configs/_implementations.py:1129: error: Argument 2 to "builds" of "BuildsFn" has incompatible type "**dict[str, int | float | Path | DataClass_ | type[DataClass_] | Enum | Any | Sequence[HydraSupportedType] | Mapping[Any, HydraSupportedType] | None]"; expected "Literal[False] | None"  [arg-type]
- src/hydra_zen/structured_configs/_implementations.py:1129: error: Argument 2 to "builds" of "BuildsFn" has incompatible type "**dict[str, int | float | Path | DataClass_ | type[DataClass_] | Enum | Any | Sequence[HydraSupportedType] | Mapping[Any, HydraSupportedType] | None]"; expected "bool"  [arg-type]
- src/hydra_zen/structured_configs/_implementations.py:1129: error: Argument 2 to "builds" of "BuildsFn" has incompatible type "**dict[str, int | float | Path | DataClass_ | type[DataClass_] | Enum | Any | Sequence[HydraSupportedType] | Mapping[Any, HydraSupportedType] | None]"; expected "Builds[Callable[[Callable[..., Any]], Callable[..., Any]]] | ZenPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]] | HydraPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]] | Just[Callable[[Callable[..., Any]], Callable[..., Any]]] | type[Builds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[ZenPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[HydraPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[Just[Callable[[Callable[..., Any]], Callable[..., Any]]]] | Callable[[Callable[..., Any]], Callable[..., Any]] | str | None | Sequence[Builds[Callable[[Callable[..., Any]], Callable[..., Any]]] | ZenPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]] | HydraPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]] | Just[Callable[[Callable[..., Any]], Callable[..., Any]]] | type[Builds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[ZenPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[HydraPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[Just[Callable[[Callable[..., Any]], Callable[..., Any]]]] | Callable[[Callable[..., Any]], Callable[..., Any]] | str | None]"  [arg-type]
- src/hydra_zen/structured_configs/_implementations.py:1129: error: Argument 2 to "builds" of "BuildsFn" has incompatible type "**dict[str, int | float | Path | DataClass_ | type[DataClass_] | Enum | Any | Sequence[HydraSupportedType] | Mapping[Any, HydraSupportedType] | None]"; expected "Mapping[str, SupportedPrimitive] | None"  [arg-type]
- src/hydra_zen/structured_configs/_implementations.py:1129: error: Argument 2 to "builds" of "BuildsFn" has incompatible type "**dict[str, int | float | Path | DataClass_ | type[DataClass_] | Enum | Any | Sequence[HydraSupportedType] | Mapping[Any, HydraSupportedType] | None]"; expected "list[str | DataClass_ | type[DataClass_] | Mapping[str, str | Sequence[str] | None]] | None"  [arg-type]
- src/hydra_zen/structured_configs/_implementations.py:1129: error: Argument 2 to "builds" of "BuildsFn" has incompatible type "**dict[str, int | float | Path | DataClass_ | type[DataClass_] | Enum | Any | Sequence[HydraSupportedType] | Mapping[Any, HydraSupportedType] | None]"; expected "str | None"  [arg-type]
- src/hydra_zen/structured_configs/_implementations.py:1129: error: Argument 2 to "builds" of "BuildsFn" has incompatible type "**dict[str, int | float | Path | DataClass_ | type[DataClass_] | Enum | Any | Sequence[HydraSupportedType] | Mapping[Any, HydraSupportedType] | None]"; expected "tuple[type[DataClass_], ...]"  [arg-type]
- src/hydra_zen/structured_configs/_implementations.py:1129: error: Argument 2 to "builds" of "BuildsFn" has incompatible type "**dict[str, int | float | Path | DataClass_ | type[DataClass_] | Enum | Any | Sequence[HydraSupportedType] | Mapping[Any, HydraSupportedType] | None]"; expected "ZenConvert | None"  [arg-type]
- src/hydra_zen/structured_configs/_implementations.py:1129: error: Argument 2 to "builds" of "BuildsFn" has incompatible type "**dict[str, int | float | Path | DataClass_ | type[DataClass_] | Enum | Any | Sequence[HydraSupportedType] | Mapping[Any, HydraSupportedType] | None]"; expected "T"  [arg-type]
- src/hydra_zen/structured_configs/_implementations.py:1161: error: Incompatible types in assignment (expression has type "Just", variable has type "type[Builds[Any]]")  [assignment]

@cdce8p cdce8p requested a review from JelleZijlstra June 20, 2024 15:13
@JelleZijlstra JelleZijlstra merged commit 4ba2696 into python:master Jun 20, 2024
18 checks passed
@cdce8p cdce8p deleted the update-typeshed branch June 20, 2024 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants