Names are re-exported from stubs

[ilevkivskyi]

This code passes silently, but obviously fails at runtime:

from collections import TypeVar
from weakref import Generic
from getopt import List

T = TypeVar('T')

class C(Generic[T], List[int]):
    ...

[gvanrossum]

Strange Why would that work? Normally when you do "from X import Y" inside a stub, Y is not exported further. Is this not working in the case that X is typing?

[ilevkivskyi]

Hm... I just tried this with other modules, and it looks like names from stubs are always exported further by stubs. Namely I have this:

ivan@X-333 ~/Devel $ cat tsta.py 
x = 1

ivan@X-333 ~/Devel $ cat tstb.pyi 
from tsta import x

y = 2

ivan@X-333 ~/Devel $ cat tstc.py 
from tstb import x

y: int = x

Then mypy tstc.py passes without errors.

[ilevkivskyi]

From what I read in semanal.py, the module_public attribute is used only by visit_import_all. And indeed, if I change the code in tstc.py to from tstb import *, then mypy correctly complains Name 'x' is not defined.

But I think names imported in stubs should be never implicitly re-exported (not only when used with import *). Is this right?

[gvanrossum]

What if you rename tsta.py to tsta.pyi?

[ilevkivskyi]

What if you rename tsta.py to tsta.pyi?

The behavior is the same as with .py: passes silently with import x and complains Name 'x' is not defined with import *.

I am almost sure now that the problem is module_public is used for two different purposes: for identifying real public symbols in the runtime sense of this word (i.e. not starting with an underscore or present in __all__), and for re-exporting names from stubs. Only the first purpose is fully reflected in semanal.py/visit_import_all currently. I think we need an additional flag (maybe stub_only?) for symbols that should not be imported by visit_import_from and visit_import.

[JukkaL]

I'm pretty sure that @ilevkivskyi is right and only from x import * is special right now for importing things defined in stubs.

[JukkaL]

The reason for the current behavior is that from x import * importing internal definitions used to break a lot of user code, so that was fixed. Importing an internal definition by name is a less serious issue and kind of similar how you can also import module internals at runtime, since they usually aren't protected in any way, so we haven't done anything about that yet. For stubs we can reliably(?) know which definitions are internal so we should be able to hide module internals more effectively. This could still break some user code, however.

[gvanrossum]

Wow. I've always believed that you had to write `from X import Y as Y` in a stub in order to explicitly export Y. That's also what PEP 484 says AFAICT. But you're right it seems to work only for 'import *'.

[ilevkivskyi]

I've always believed that you had to write from X import Y as Y in a stub in order to explicitly export Y. That's also what PEP 484 says AFAICT. But you're right it seems to work only for import *.

OK, then I will make a PR that will ensure this behaviour also for from mod import name. Running tests on typeshed will show how strictly the PEP rule from X import Y as Y was followed.