Fix library duplication in tensorflow v.2.3 and avoid packaging unnecessary files #50
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
On some tensorflow versions (2.3.0 thus far), the _pywrap_tensorflow_internal extension module ends up duplicated. We pick it up as both an extension module (placed in `tensorflow/python` directory) and as a shared library (placed in program's root directory). This increases the program size, and also prevents the program from running under macOS. In the problematic versions, the extension module seems to be picked up as `tensorflow.python._pywrap_tensorflow_internal`, while in others it is picked up as just ˙_pywrap_tensorflow_internal`. So we try to work around the duplication by removing the problematic entry from `hiddenimports` and adding it to `excludedimports`. Fixes pyinstaller#49.
Avoid collecting tensorflow's development headers in `include` directory and XLA AOT runtime sources (`xla_aot_runtime_src`). Also prevent `libtensorflow_framework` shared library from being picked up as data to prevent its duplication (it should be correctly picked up as a shared library by dependency scanner).
rokm
force-pushed
the
tensorflow-duplicate-library
branch
from
79cb40d
to
70ae5e9
Compare
|
Is there any sense in adding the test code from #49 to the test suite? Or could we already reproduce with our current ones? |
|
The example given in #49 is actually (almost) identical as one of our tests (training on MNIST dataset). But the issue on macOS can be reproduced by just importing |
rokm
requested review from
a team and
bwoodsend
and removed request for
a team
bwoodsend
approved these changes
Sep 24, 2020
Legorooj
approved these changes
Sep 25, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a follow-up to the previous
tensorflowhook cleanup:tensorflow.python._pywrap_tensorflow_internal.libtensorflow_frameworkshared library (the unused copy picked up viadatasends up intensorflowfolder, while the one picked up by dependency analysis ends up in program root)