Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dill.dumps depends on 'site-packages' being substring of module path #331

Open
pdvyas opened this issue Aug 23, 2019 · 1 comment
Open

Comments

@pdvyas
Copy link

pdvyas commented Aug 23, 2019

dill.dumps tries to detect if the object is from a builtin (or an installed package?) by looking for 'site-packages' in module path (

builtin_mod = builtin_mod or 'site-packages' in obj.__file__
). It breaks for the way we package software with bazel where modules in the third party packages do not have 'site-packages' in their path. We are able to run it with the following patch (with pypi__ being a substring in our third party packages)

diff --git a/dill/_dill.py b/dill/_dill.py
index d52c486..13c6258 100644
--- a/dill/_dill.py
+++ b/dill/_dill.py
@@ -1247,7 +1247,7 @@ def save_module(pickler, obj):
                      "prefix", "real_prefix"]
             builtin_mod = any([obj.__file__.startswith(os.path.normpath(getattr(sys, name)))
                            for name in names if hasattr(sys, name)])
-            builtin_mod = builtin_mod or 'site-packages' in obj.__file__
+            builtin_mod = builtin_mod or 'pypi__' in obj.__file__ or 'site-packages' in obj.__file__
         else:
             builtin_mod = True
         if obj.__name__ not in ("builtins", "dill") \

Is it possible to not depend on the module path here?

@atleypnorth
Copy link

This PR #283 has been out there for a while proposing a similar fix to above - is it worth revisiting that?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants