Skip to content

Add types to _core._run #2733

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 60 commits into from
Aug 14, 2023
Merged

Add types to _core._run #2733

merged 60 commits into from
Aug 14, 2023

Conversation

TeamSpen210
Copy link
Contributor

@TeamSpen210 TeamSpen210 commented Aug 3, 2023

This adds types to the _run module (as much as can be done without TypeVarTuple). In particular I made TaskStatus a protocol, not a concrete class - since it's just documented as being a non-specific object with the method. Users shouldn't be depending on a specific type. Though the positional arguments can't be checked, this does make nursery.start(...) produce the right result, check you use TaskStatus correctly etc.

There's a number of interesting type errors - Clock._autojump() is called despite that not being defined on the ABC, and a bunch of places where _cancel_status is accessed without checking if it's None.

@TeamSpen210 TeamSpen210 added the typing Adding static types to trio's interface label Aug 3, 2023
@codecov
Copy link

codecov bot commented Aug 3, 2023

Codecov Report

Merging #2733 (137a05b) into master (4363a06) will increase coverage by 0.01%.
The diff coverage is 100.00%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2733      +/-   ##
==========================================
+ Coverage   98.91%   98.92%   +0.01%     
==========================================
  Files         113      113              
  Lines       16718    16708      -10     
  Branches     3030     3027       -3     
==========================================
- Hits        16536    16528       -8     
+ Misses        125      124       -1     
+ Partials       57       56       -1     
Files Changed Coverage Δ
trio/__init__.py 100.00% <ø> (ø)
trio/_core/__init__.py 100.00% <ø> (ø)
trio/_dtls.py 96.59% <ø> (ø)
trio/_tools/gen_exports.py 100.00% <ø> (ø)
trio/_util.py 100.00% <ø> (ø)
trio/lowlevel.py 100.00% <ø> (ø)
trio/_core/_io_epoll.py 100.00% <100.00%> (ø)
trio/_core/_io_kqueue.py 85.71% <100.00%> (+0.11%) ⬆️
trio/_core/_io_windows.py 98.78% <100.00%> (+0.01%) ⬆️
trio/_core/_local.py 100.00% <100.00%> (ø)
... and 4 more

Copy link
Member

@jakkdl jakkdl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The None cases can presumable be addressed with various assert x is not None, at least adding those (or type: ignore) makes it easier to see in the code review where it happens instead of having to cross-reference with the CI output.

I'm not sure how to handle _autojump in a way that's terribly helpful for end-users, as the mechanism by how it's enabled in the runner to start with is kinda weird. But since it requires explicitly setting runner.clock_autojump_treshold from outside the class, you're probably not hitting it unintentionally. Maybe just a try/except with a helpful error message directing them to MockClock, and a type: ignore.

@jakkdl
Copy link
Member

jakkdl commented Aug 3, 2023

Also: turn up the strictness in pyproject.toml (both for _run and _generated_run) - you've missed to add a hint for current_statistics (and maybe others?)

def start_soon(self, async_fn, *args, name=None):
def start_soon(
self,
# TODO: TypeVarTuple
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought I saw something about mypy being able to understand TypeVarTuple in one of the newer updates unless I am mistaken

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's got experimental support - definitions work, but when Mypy tries checking function calls it tends to crash with AssertionErrors.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's got experimental support - definitions work, but when Mypy tries checking function calls it tends to crash with AssertionErrors.

Hmm, does that mean that we could type our code with TypeVarTuple - but as long as we don't enable checking the code with mypy then we're fine, and users on other type checkers can make use of them? (and we probably want to add pyright or something to our CI with only enough checks added to catch incorrect calls to TypeVarTupled functions)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If so, we should leave it for a different PR that changes it in all files though and can figure out any issues.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have to pass the undocumented --enable-incomplete-feature=TypeVarTuple + =Unpack to enable the incomplete code. If not, you get a warning raised for the code, Unpack becomes Any. So maybe we should indeed use them, if we can suppress the error.

@TeamSpen210
Copy link
Contributor Author

While typing further, found a few more issues.

  • sortedcontainers doesn't have types, and it seems is unlikely to ever have typing - it does a lot of unusual things in the constructors apparently.
  • Looks like we need either my outcome PR or trio_typing in CI to have types for outcome, the errors are fine with that present.

Copy link
Member

@CoolCat467 CoolCat467 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great other than noted changes

# isort: skip_file
from __future__ import annotations

from typing import Any, Callable, Awaitable
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use from collections.abc import Awaitable, Callable instead, the typing versions are just aliases

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's no real reason not to use them either, once we drop py39 (?) support and run pyupgrade they'll all get replaced. I'm weakly in favor of using the non-aliases, but don't think it's worth delaying PR's over it.

Copy link
Contributor

@A5rocks A5rocks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment

Comment on lines +205 to +206
# derive() returns Any for some reason.
return excgroup.derive(exceptions) # type: ignore[no-any-return]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made an upstream issue! See agronholm/exceptiongroup#76

return task

def task_exited(self, task, outcome):
def task_exited(self, task: Task, outcome: Outcome[Any]) -> None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I .... think the Any might possibly be object-ified.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately no, Outcome can't be invariant because it uses them in both unwrap()'s return value and send()'s parameter.

Copy link
Member

@oremanj oremanj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This generally looks great; thank you for working on these!

@oremanj
Copy link
Member

oremanj commented Aug 11, 2023

Thanks! I have no further comments but will leave final approval to @A5rocks since they've been more involved in the process thus far.

Copy link
Member

@CoolCat467 CoolCat467 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am noticing a lot of missing annotations, in particular _io_ files

@jakkdl
Copy link
Member

jakkdl commented Aug 11, 2023

FTR: I cloned this PR and tried messing around with imports to see if I could resolve https://mypy.readthedocs.io/en/stable/error_code_list.html#check-that-type-of-target-is-known-has-type for _NO_SEND but to no avail.

Copy link
Member

@CoolCat467 CoolCat467 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good other than a few missing annotations on at least the windows one, but if not here there can always be more PRs. Also of course making sure the CI passes

@jakkdl
Copy link
Member

jakkdl commented Aug 14, 2023

Looks good other than a few missing annotations on at least the windows one, but if not here there can always be more PRs. Also of course making sure the CI passes

_io_windows is listed as a TODO in #2734, and not a focus of this PR

…pl uses TypeVarTuple, 3. we run mypy without -m
@jakkdl jakkdl merged commit 7090051 into python-trio:master Aug 14, 2023
@TeamSpen210 TeamSpen210 deleted the typed_run branch August 14, 2023 23:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
typing Adding static types to trio's interface
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants