-
-
Notifications
You must be signed in to change notification settings - Fork 335
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use less magic in constructing our public API exports, to make trio more intelligible to static analysis #542
Comments
FYI, there has been some work on this in the past. Maybe there could be a program to read import modules, introspect and spit out a new set of sources that are more static?
|
Yeah, we should probably give up on most of this cleverness and switch to doing the more boring approach. There was some discussion of this in the chat today also, starting about here: https://gitter.im/python-trio/general?at=5b1842e099fa7f4c064e93a0 This is probably best handled as an incremental set of cleanups, since there are a lot of different subtle bits of trickery here. "Does this cleanup help pylint/mypy/pycharm/jedi understand what's going on" is probably a good metric to use to decide where to start... Also relevant: #475, which I think is a change we probably should do, and will reduce the number of auto-generated wrapper functions in |
This comment describes how to write a strong test for pylint being able to understand our public API: #594 (comment) |
Closed by #594 🎉 |
I like your enthusiasm, but I'm afraid we're not done yet :-). #594 made the main trio namespace work with tab completion, but we still need to figure out what to do with the submodules, and think about whether we can get rid of any of the other runtime namespace manipulation that we do... |
Haha, I actually just went and updated my "add type hints" branch (master...Zac-HD:type-hints)... it's still mostly about ignoring analysis failures, so I was definitely too enthusiastic 🤣 |
So if I understand the status correctly: the main There are a few more public namespaces to look at:
Does that sounds like the current status? |
No way what? |
Yep, that's my understanding too.
Sounds good to me. We might also need to make the namespace construction less magical, but we need static imports first anyway and can see what actually has to change after that.
This also sounds good, though I'd be very cautious about statically listing dynamic constants - it might be the best way currently available, but we should leave some detailed comments about what's going on.
Yes, but only if we have CI always check that there is no difference between the current output of the generator and the file tracked by git. This catches both stale files when the generator has been updated, and contributors trying to update the files by hand - both recipes for bugs down the line! |
@Fuyukai good catch, thanks, I edited the post to add the missing words |
Just dropping a quick update on my type-hints branch: moving in the right direction, but plenty left to do. Current Mypy output
So once those things are sorted out, we can run Mypy in CI! Then:
|
Gah, misclick - sorry! |
Extending the tests from #594 to cover the changes in #656 would be a nice, self-contained contribution. |
This should make trio.socket more intelligible to static analysis tools (see gh-542).
Opened #699 to discuss the sub-issue of how to handle |
Updated todo list:
I think that's it for module-level exports. Then there are some classes that use dynamic shenanigans on the attribute level:
|
Hi I think I at least nominally covered the list. So if someone would like to take a look they are #751 #752 and #753 |
migrate trio.ssl from star import to explicit imports, to aid static analysis (issue #542)
Migrate hazmat to faked explicit importing and reexporting, to aid static analysis (issue #542)
#805 has landed! I think this means all our top-level stuff is now statically understandable, and all that's left are a few classes with dynamic attributes. Can any of our static analysis experts here give us an update on what you think the next step should be? |
@njsmith Just browsed again through the thread, but could not make out any task left. |
@jmfrank63 I think the next steps on static analysis stuff are:
|
Just leaving a note here because I was trying to figure out why the urwid tests wouldn't run. The upshot is an attribute error: |
@sarnold I don't think that's related to the static analysis that this thread is discussion -- it's just that |
(And there's even a urwid PR: urwid/urwid#439) |
The Trio loop was contributed to |
I've just run into the distinction between Is the distinction just to prevent people constructing class _SocketType(SocketType):
def __init__(self, sock, _internal=False):
if not _internal:
raise TypeError(...) Of course this makes it easier for someone to do what they're not meant to, but hopefully it's still clear that that's 'off label' usage and not to be relied on. |
No, we actually have a more standard solution for that. IIRC the weird So the goals were:
On further thought, maybe an alternative would be:
I guess this would need a deprecation b/c it would break any existing code that's doing |
Despite the well defined
__all__
, several important tools (e.g: pylint, mypy, jedi) cannot understand the code structure of trio. They then report error in the user code.E.G:
On the following snippet, pylint will report trio not having the
run()
member, jedi won't allow code completion, vscode intellisense won't let you go to definition and mypy won't report any error if I try to add+1
afterrun()
I understand this is inconvenient, as those tools should be able to pick up the
__all__
declaration and use that instead of requiring your to type it all again, but they don't. Since they are the most popular in the Python ecosystem, and are used by a lot of editors (vi, sublime text, vscode, etc) but also in tasks (tox, travis, git hooks, etc), it's important that they work.For now, that means putting
# noqa
a bit everywhere or disabling a lot of warning, which is not practical.The text was updated successfully, but these errors were encountered: