zipimporter misses namespace packages for implicit dirs

[jaraco]

BPO	36740
Nosy	@warsaw, @jaraco, @ericvsmith
Superseder	bpo-14905: zipimport needs to support namespace packages when no 'directory' entry exists

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-04-28.20:52:26.459>
created_at = <Date 2019-04-27.08:04:15.846>
labels = ['type-bug', 'library']
title = 'zipimporter misses namespace packages for implicit dirs'
updated_at = <Date 2019-04-30.22:54:18.989>
user = 'https://github.com/jaraco'

bugs.python.org fields:

activity = <Date 2019-04-30.22:54:18.989>
actor = 'barry'
assignee = 'none'
closed = True
closed_date = <Date 2019-04-28.20:52:26.459>
closer = 'jaraco'
components = ['Library (Lib)']
creation = <Date 2019-04-27.08:04:15.846>
creator = 'jaraco'
dependencies = []
files = []
hgrepos = []
issue_num = 36740
keywords = []
message_count = 2.0
messages = ['340975', '341033']
nosy_count = 3.0
nosy_names = ['barry', 'jaraco', 'eric.smith']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '14905'
type = 'behavior'
url = 'https://bugs.python.org/issue36740'
versions = []

[jaraco]

As discovered in pypa/packaging-problems#212, if a PEP-420 namespace package is represented by an implicit directory (that is, there's no explicit entry for the directory, only entries for the contents of the directory), that directory won't be picked up as a namespace package. The following code illustrates the issue:

zp $ cat make-pkgs.py                                                                                                                                                          
import zipfile


def make_pkgs():
    zf = zipfile.ZipFile('simple.zip', 'w')
    zf.writestr('pkg/__init__.py', b'')
    zf.close()

    zf = zipfile.ZipFile('namespace.zip', 'w')
    zf.writestr('ns/pkg/__init__.py', b'')
    zf.close()


__name__ == '__main__' and make_pkgs()
zp $ python make-pkgs.py                                                                                                                                                       
zp $ env PYTHONPATH=simple.zip python3.7 -c "import pkg"                                                                                                                       
zp $ env PYTHONPATH=namespace.zip python3.7 -c "import ns.pkg"                                                                                                                 
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'ns'

As you can see, in simple.zip, the pkg directory is implied, but despite that condition, pkg is importable.

However, with namespace.zip, the name ns is not visible even though it's present in the zipfile and would be importable if that zipfile were extracted to a file system.

zp $ unzip namespace.zip
Archive:  namespace.zip
 extracting: ns/pkg/__init__.py
zp $ python3.7 -c "import ns.pkg" && echo done
done

If you were to reconstruct that zip file on the file system using standard tools or explicitly include 'ns/' in the zip entries, the namespace package becomes visible:

zp $ rm namespace.zip                                                                                                                                                          
zp $ zip -r namespace.zip ns                                                                                                                                                   
  adding: ns/ (stored 0%)
  adding: ns/pkg/ (stored 0%)
  adding: ns/pkg/__init__.py (stored 0%)
  adding: ns/pkg/__pycache__/ (stored 0%)
  adding: ns/pkg/__pycache__/__init__.cpython-37.pyc (deflated 23%)
zp $ rm -r ns                                                                                                                                                                  
zp $ env PYTHONPATH=namespace.zip python3.7 -c "import ns.pkg" && echo done                                                                                                    
done

For consistency, the zip import logic should probably honor implicit directories in zip files.

[ericvsmith]

This is a duplicate of bpo-14905.