Cannot import requests/certifi from embedded zipfile since 2020.4.5.2 release #131
Comments
|
Hey @dstufft , what’s going on here? |
|
How was the zip file created? Does it have the pem file in it as expected?
…Sent from my iPhone
On Jun 9, 2020, at 2:19 PM, Cory Benfield ***@***.***> wrote:
Hey @dstufft , what’s going on here?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
The zip file was created by taking the standard python embeddable zip file and adding the requests module with dependencies to its standard library zip file (python37.zip) for embedded use in a C++ program. It contains the pem file as expected. |
|
I seem to be able to resolve a certificates file from the latest certifi when it's a wheel: |
|
The traceback you show there indicates that the right code is being called. And for whatever reason, Can you inspect the code at that point and ascertain why that call fails to find that resource? Alternately, can you suggest a way I could replicate the issue? Are you able to test easily with Python 3.8? Python 3.8 changed the way zipimport was implemented, so I'd like to eliminate that as a factor if it's not too much trouble. |
|
Finally got round to testing this with Python 3.8.3. Still get a stacktrace, pasted below. |
|
As for how to reproduce, I basically just add the requests module to the standard embeddable zipfile.
|
|
Didn't realised you can't attach files to these issues. Here's the little script "make_embedded_python.py", in any case. import sys, os, shutil
from zipfile import ZipFile, ZIP_DEFLATED
from glob import glob
def has_pyd_files(path):
for _, _, files in os.walk(path):
for fn in files:
if fn.endswith(".pyd"):
return True
return False
def add_directory_to_zip(zf, src, dst):
for root, _, files in os.walk(src):
for fn in files:
srcpath = os.path.join(root, fn)
srcrelpath = srcpath.replace(src + os.sep, "")
arcname = os.path.join(dst, srcrelpath)
zf.write(srcpath, arcname)
if __name__ == "__main__":
embed_zip = sys.argv[1]
packages_src = sys.argv[2]
dest_dir = sys.argv[3]
if os.path.isdir(dest_dir):
shutil.rmtree(dest_dir)
print(" ========= Creating", dest_dir)
os.makedirs(dest_dir)
with ZipFile(embed_zip, 'r') as zf:
zf.extractall(dest_dir)
packages_zip = glob(os.path.join(dest_dir, "python*.zip"))[0]
with ZipFile(packages_zip, 'a', ZIP_DEFLATED) as zf:
for fn in os.listdir(packages_src):
path = os.path.join(packages_src, fn)
if os.path.isfile(path):
if fn.endswith(".pyd"):
shutil.copy(path, dest_dir)
else:
zf.write(path, os.path.join("site-packages", fn))
elif not fn.endswith("-info"):
if has_pyd_files(path):
shutil.copytree(path, os.path.join(dest_dir, fn))
else:
add_directory_to_zip(zf, path, os.path.join("site-packages", fn)) |
|
I've been able to replicate the issue much more simply: Output The issue here appears to be related to the subdirectory on the zipfile ( This behavior appears to be a bug in importlib-resources. I'll dig deeper. |
|
I do note that the more modern construct of |
|
I filed python/importlib_resources#105 to track the issue in the package. It may prove difficult to include this behavior in Python 3.8 and impossible to fix in Python 3.7, so my best recommendation is for certifi to use the |
|
I think the bug is also in certifi: In /certifi/core.py there is a try/except, but the symbol can always be imported. Any implementation that relies on will always get the importlib.resources version, never the other version. So having ONE definition of where(), and a try except inside it would also be a solution. |
…n 3.9 and importlib_metadata 1.1. Add explicit temporary file cleanup behavior using atexit. Fixes certifi#131.
|
Recent changes to use |
Just a simple "import requests" no longer works in the context of embedded Python with the code in a zip file. It works fine in 2020.4.5.1.
The text was updated successfully, but these errors were encountered: