Remaining tasks until --verifytypes passes and check_type_completeness.py can be removed

[jakkdl]

To get a bit of an overview of the remaining files until we hit Type Completeness ™️

Public API

• TypeVarTuple support - needs to be checked with pyright instead of mypy that they're correct, but mypy should be able to accept their presence.
  • not really possible to get mypy to accept their presence in my testing, we need to wait for mypy 1.7. Tracked in TypeVarTuple support #2859
• SocketType, see Handling _SocketType and SocketType #2720 - @jakkdl
• _core/
  • _mock_clock Add missing type annotations to _core/_mock_clock.py #2740
  • _run Add types to _core._run #2733
    • _generated_run Add types to _core._run #2733
  • _io_epoll type _io_epoll and _io_kqueue #2743
    • _generated_io_epoll type _io_epoll and _io_kqueue #2743
  • _io_kqueue type _io_epoll and _io_kqueue #2743
    • _generated_io_kqueue type _io_epoll and _io_kqueue #2743
  • _io_windows - some work in Fix various files so the entire trio/ directory can be type-checked #2763, but I would like somebody else to take over and finish.
    • _generated_io_windows
  • _thread_cache Add types to core._thread_cache #2741
  • _traps Fix various files so the entire trio/ directory can be type-checked #2763
• testing/ Add types to most of trio.testing #2747
• _highlevel
  • _socket
  • _ssl_helpers
  • _serve_listeners
  • _open_tcp_listeners Add typing for _highlevel_open_tcp_listeners.py #2724
  • _open_unix_stream
• _ssl Add type annotations for _ssl.py #2745
• _subprocess
• _threads
• _signals Add types to _signals, test_signals #2794 (_open_signal_receiver is clearly exported in trio/__init__.py, but for some reason doesn't show up in the file list when I run --verifytypes)

Possibly more files if running with a --pythonplatform other than linux.

Private or deprecated (lower priority)

May include files that should be in the above list, I didn't look through files in a fully complete way

• _core/
  • _asyncgens Add typing for _core/_asyncgens.py #2735
  • _multierror Add typing for _core/_multierror.py #2742
  • _io_common Fix various files so the entire trio/ directory can be type-checked #2763
  • _windows_cffi Add typing for _wait_for_object.py #2755
  • _ki Fix various files so the entire trio/ directory can be type-checked #2763
  • _wakeup_socketpair Fix various files so the entire trio/ directory can be type-checked #2763
• _tools/
  • _gen_exports.py Add typing to _tools/gen_exports.py #2738
• _windows_pipes Add type annotations for windows_pipes and its test #2812
• _wait_for_object (windows) Add typing for _wait_for_object.py #2755
• testing/
  • _fake_net Fix various files so the entire trio/ directory can be type-checked #2763 (This file should maybe be moved to _tests/ as it's only used internally)
• _subprocess_platform/waitid
• various files

Tests

Typing the tests will check that it's possible to use the interface in a reasonable way, as well as catch any possible bugs in the tests themself of course.

• trio/testing/_fake_net
• trio/_core/_tests/
• trio/_tests/

docstrings

• Add missing docstrings #2762

Cleanup

• Remove check_type_completeness and verify_types_*.json, minor fixes #2877
  • Remove references to py.typed in the code, from when we needed to generate it on-the-fly because it wasn't statically added in the package.
  • Remove check_type_completeness.py, verify_types_[windows,linux,darwin].json and their entries in check.sh
  • Add pyright --verifytypes with multiple platforms to check.sh, and/or pre-commit.

[CoolCat467]

_core/_asyncgens in #2735

[A5rocks]

FYI if you aren't aware, members of the org can edit issue comments! (I'm just saying, it would be cool to have people edit the initial comment in addition to their comment for notifications, so @jakkdl doesn't have to edit this all the time :-)

[jakkdl]

Clarified [_generated]_xxx into two separate listings for _xxx and a subheader for _generated_xxx.

[jakkdl]

Note that _multierror, _asyncgens, etc were intentionally left out of my listing as they're not required for the public interface - which should be the first priority. But I'll add a second list for non-public untyped files so those files can be coordinated on as well, since we do eventually want everything typed to catch any bugs in trio itself.

[jakkdl]

FYI if you aren't aware, members of the org can edit issue comments! (I'm just saying, it would be cool to have people edit the initial comment in addition to their comment for notifications, so @jakkdl doesn't have to edit this all the time :-)

Please make a comment when non-trivially editing the list though, so everybody gets a notification

[jakkdl]

Collapsed testing/, and added #2720 as a TODO

[jakkdl]

Ran --verifytypes on windows & darwin and split out files needed for those systems, where I noticed I missed a couple docstrings in #2743

EDIT: it seems I misplaced this edit, but handled in #2762

[jakkdl]

Noting down two small TODOS that weren't fully addressed in #2682

• lots of typing improvements #2682 (comment)
  

  trio/trio/_sync.py

  Lines 108 to 122 in d6d650f

  	# TODO: type this with a Protocol to get rid of type: ignore, see
  	# https://github.com/python-trio/trio/pull/2682#discussion_r1259097422
  	class AsyncContextManagerMixin:
  	@enable_ki_protection
  	async def __aenter__(self) -> None:
  	await self.acquire() # type: ignore[attr-defined]
  	@enable_ki_protection
  	async def __aexit__(
  	self,
  	exc_type: type[BaseException] | None,
  	exc_value: BaseException | None,
  	traceback: TracebackType | None,
  	) -> None:
  	self.release() # type: ignore[attr-defined]

  - addressed in Add types to test_io, test_run, various other fixes #2773

• use typing_extensions.Buffer in SendStream.send_all: lots of typing improvements #2682 (comment)

[jakkdl]

Checked off a couple dozen tasks, though didn't bother linking them to PRs. Also added a task for adding TypeVarTuple

[jakkdl]

I've started working on SocketType, and associated tests: test_highlevel_open_tcp_stream, test_highlevel_open_tcp_listeners, test_highlevel_socket, test_socket

[A5rocks]

For what it's worth, I don't think it's completely worth our time to completely type tests. I can imagine there's quite some dynamism going on there :P

I imagine everyone's got an opinion on this already, but maybe we can try just typing a few easy ones (no insane dynamic stuff) and try to aim for typing a breadth of usage (rather than comprehensively for every test)?

[TeamSpen210]

Fully dynamic stuff no, indeed the main benefit is having a lot of sample usages of all the various functions to confirm the types make sense.

[jakkdl]

For what it's worth, I don't think it's completely worth our time to completely type tests. I can imagine there's quite some dynamism going on there :P

I imagine everyone's got an opinion on this already, but maybe we can try just typing a few easy ones (no insane dynamic stuff) and try to aim for typing a breadth of usage (rather than comprehensively for every test)?

Yeah we could set up a separate set of settings for test files, or just be more lax with type: ignore and Any

[A5rocks]

Uh, well we have py.typed so I assume this issue is solely about typing the tests now, right?

[CoolCat467]

Uh, well we have py.typed so I assume this issue is solely about typing the tests now, right?

As far as I know, this is correct, except that _windows_pipes was still untyped, but #2812 is fixing that now.

[jakkdl]

yeah the title of the issue is definitely a misnomer, and with the excitement of adding types dying down as we're down to just the chore of typing the test files we don't really need this issue as a tracker either (the blacklist in pyproject.toml achieves it in a better way).

There's some cleanup that needs to be done though, there's 27 references to py.typed/py_typed which should be possible to remove now, and if anybody does anything with #2762 then we can fully get rid of check_type_completeness, the json output files and directly run pyright --verifytypes in CI.

[A5rocks]

Bump once again to make sure this is still relevant now that test files are typed too. (thanks again!)

[jakkdl]

There's some cleanup that needs to be done though, there's 27 references to py.typed/py_typed which should be possible to remove now, and if anybody does anything with #2762 then we can fully get rid of check_type_completeness, the json output files and directly run pyright --verifytypes in CI.

This is still the current status, I'll rename the issue.

[jakkdl]

Updated the tasks in the OP

[jakkdl]

🚀