Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Not all changes to setup.py invalidate the cache. #620

Closed
jsirois opened this issue Nov 3, 2018 · 0 comments · Fixed by #788
Closed

Not all changes to setup.py invalidate the cache. #620

jsirois opened this issue Nov 3, 2018 · 0 comments · Fixed by #788
Assignees
@jsirois
Labels
bug

Comments

@jsirois
Copy link
Member

jsirois commented Nov 3, 2018

In particular, changing install_requires does not rebuild the local project and the resulting pex does not include the new install_requires dists.
For example, against the pex project itself:

$ rm -rf ~/.pex/
$ /tmp/pex-1.5.1.pex . -o pex-local.pex
$ vi setup.py
$ git diff setup.py
diff --git a/setup.py b/setup.py
index 3b7f3b4..6da5704 100644
--- a/setup.py
+++ b/setup.py
@@ -54,6 +54,7 @@ setup(
     'pex.commands',
   ],
   install_requires = [
+    'pytest',
     SETUPTOOLS_REQUIREMENT,
     WHEEL_REQUIREMENT,
   ],
$ /tmp/pex-1.5.1.pex . -o pex-local-edited.pex
$ diff <(zipinfo -1 pex-local.pex) <(zipinfo -1 pex-local-edited.pex)
$

To get the change to be picked up, we have to nuke the build cache:

$ rm -rf ~/.pex/build/
$ /tmp/pex-1.5.1.pex . -o pex-local-edited.pex
$ diff <(zipinfo -1 pex-local.pex) <(zipinfo -1 pex-local-edited.pex)
106a107,157
> .deps/atomicwrites-1.2.1-py2.py3-none-any.whl/atomicwrites-1.2.1.dist-info/METADATA
> .deps/atomicwrites-1.2.1-py2.py3-none-any.whl/atomicwrites-1.2.1.dist-info/RECORD
> .deps/atomicwrites-1.2.1-py2.py3-none-any.whl/atomicwrites-1.2.1.dist-info/WHEEL
> .deps/atomicwrites-1.2.1-py2.py3-none-any.whl/atomicwrites-1.2.1.dist-info/top_level.txt
> .deps/atomicwrites-1.2.1-py2.py3-none-any.whl/atomicwrites/__init__.py
> .deps/attrs-18.2.0-py2.py3-none-any.whl/attr/__init__.py
> .deps/attrs-18.2.0-py2.py3-none-any.whl/attr/__init__.pyi
> .deps/attrs-18.2.0-py2.py3-none-any.whl/attr/_compat.py
> .deps/attrs-18.2.0-py2.py3-none-any.whl/attr/_config.py
> .deps/attrs-18.2.0-py2.py3-none-any.whl/attr/_funcs.py
> .deps/attrs-18.2.0-py2.py3-none-any.whl/attr/_make.py
> .deps/attrs-18.2.0-py2.py3-none-any.whl/attr/converters.py
> .deps/attrs-18.2.0-py2.py3-none-any.whl/attr/converters.pyi
> .deps/attrs-18.2.0-py2.py3-none-any.whl/attr/exceptions.py
> .deps/attrs-18.2.0-py2.py3-none-any.whl/attr/exceptions.pyi
> .deps/attrs-18.2.0-py2.py3-none-any.whl/attr/filters.py
> .deps/attrs-18.2.0-py2.py3-none-any.whl/attr/filters.pyi
> .deps/attrs-18.2.0-py2.py3-none-any.whl/attr/py.typed
> .deps/attrs-18.2.0-py2.py3-none-any.whl/attr/validators.py
> .deps/attrs-18.2.0-py2.py3-none-any.whl/attr/validators.pyi
> .deps/attrs-18.2.0-py2.py3-none-any.whl/attrs-18.2.0.dist-info/LICENSE.txt
> .deps/attrs-18.2.0-py2.py3-none-any.whl/attrs-18.2.0.dist-info/METADATA
> .deps/attrs-18.2.0-py2.py3-none-any.whl/attrs-18.2.0.dist-info/RECORD
> .deps/attrs-18.2.0-py2.py3-none-any.whl/attrs-18.2.0.dist-info/WHEEL
> .deps/attrs-18.2.0-py2.py3-none-any.whl/attrs-18.2.0.dist-info/top_level.txt
> .deps/funcsigs-1.0.2-py2.py3-none-any.whl/funcsigs-1.0.2.dist-info/DESCRIPTION.rst
> .deps/funcsigs-1.0.2-py2.py3-none-any.whl/funcsigs-1.0.2.dist-info/METADATA
> .deps/funcsigs-1.0.2-py2.py3-none-any.whl/funcsigs-1.0.2.dist-info/RECORD
> .deps/funcsigs-1.0.2-py2.py3-none-any.whl/funcsigs-1.0.2.dist-info/WHEEL
> .deps/funcsigs-1.0.2-py2.py3-none-any.whl/funcsigs-1.0.2.dist-info/metadata.json
> .deps/funcsigs-1.0.2-py2.py3-none-any.whl/funcsigs-1.0.2.dist-info/pbr.json
> .deps/funcsigs-1.0.2-py2.py3-none-any.whl/funcsigs-1.0.2.dist-info/top_level.txt
> .deps/funcsigs-1.0.2-py2.py3-none-any.whl/funcsigs/__init__.py
> .deps/funcsigs-1.0.2-py2.py3-none-any.whl/funcsigs/version.py
> .deps/more_itertools-4.3.0-py2-none-any.whl/more_itertools-4.3.0.dist-info/METADATA
> .deps/more_itertools-4.3.0-py2-none-any.whl/more_itertools-4.3.0.dist-info/RECORD
> .deps/more_itertools-4.3.0-py2-none-any.whl/more_itertools-4.3.0.dist-info/WHEEL
> .deps/more_itertools-4.3.0-py2-none-any.whl/more_itertools-4.3.0.dist-info/top_level.txt
> .deps/more_itertools-4.3.0-py2-none-any.whl/more_itertools/__init__.py
> .deps/more_itertools-4.3.0-py2-none-any.whl/more_itertools/more.py
> .deps/more_itertools-4.3.0-py2-none-any.whl/more_itertools/recipes.py
> .deps/more_itertools-4.3.0-py2-none-any.whl/more_itertools/tests/__init__.py
> .deps/more_itertools-4.3.0-py2-none-any.whl/more_itertools/tests/test_more.py
> .deps/more_itertools-4.3.0-py2-none-any.whl/more_itertools/tests/test_recipes.py
> .deps/pathlib2-2.3.2-py2.py3-none-any.whl/pathlib2-2.3.2.dist-info/DESCRIPTION.rst
> .deps/pathlib2-2.3.2-py2.py3-none-any.whl/pathlib2-2.3.2.dist-info/METADATA
> .deps/pathlib2-2.3.2-py2.py3-none-any.whl/pathlib2-2.3.2.dist-info/RECORD
> .deps/pathlib2-2.3.2-py2.py3-none-any.whl/pathlib2-2.3.2.dist-info/WHEEL
> .deps/pathlib2-2.3.2-py2.py3-none-any.whl/pathlib2-2.3.2.dist-info/metadata.json
> .deps/pathlib2-2.3.2-py2.py3-none-any.whl/pathlib2-2.3.2.dist-info/top_level.txt
> .deps/pathlib2-2.3.2-py2.py3-none-any.whl/pathlib2/__init__.py
153a205,321
> .deps/pluggy-0.8.0-py2.py3-none-any.whl/pluggy-0.8.0.dist-info/LICENSE
> .deps/pluggy-0.8.0-py2.py3-none-any.whl/pluggy-0.8.0.dist-info/METADATA
> .deps/pluggy-0.8.0-py2.py3-none-any.whl/pluggy-0.8.0.dist-info/RECORD
> .deps/pluggy-0.8.0-py2.py3-none-any.whl/pluggy-0.8.0.dist-info/WHEEL
> .deps/pluggy-0.8.0-py2.py3-none-any.whl/pluggy-0.8.0.dist-info/top_level.txt
> .deps/pluggy-0.8.0-py2.py3-none-any.whl/pluggy/__init__.py
> .deps/pluggy-0.8.0-py2.py3-none-any.whl/pluggy/_tracing.py
> .deps/pluggy-0.8.0-py2.py3-none-any.whl/pluggy/_version.py
> .deps/pluggy-0.8.0-py2.py3-none-any.whl/pluggy/callers.py
> .deps/pluggy-0.8.0-py2.py3-none-any.whl/pluggy/hooks.py
> .deps/pluggy-0.8.0-py2.py3-none-any.whl/pluggy/manager.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py-1.7.0.dist-info/LICENSE
> .deps/py-1.7.0-py2.py3-none-any.whl/py-1.7.0.dist-info/METADATA
> .deps/py-1.7.0-py2.py3-none-any.whl/py-1.7.0.dist-info/RECORD
> .deps/py-1.7.0-py2.py3-none-any.whl/py-1.7.0.dist-info/WHEEL
> .deps/py-1.7.0-py2.py3-none-any.whl/py-1.7.0.dist-info/top_level.txt
> .deps/py-1.7.0-py2.py3-none-any.whl/py/__init__.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/__metainfo.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_builtin.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_code/__init__.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_code/_assertionnew.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_code/_assertionold.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_code/_py2traceback.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_code/assertion.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_code/code.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_code/source.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_error.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_io/__init__.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_io/capture.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_io/saferepr.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_io/terminalwriter.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_log/__init__.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_log/log.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_log/warning.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_path/__init__.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_path/cacheutil.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_path/common.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_path/local.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_path/svnurl.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_path/svnwc.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_process/__init__.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_process/cmdexec.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_process/forkedfunc.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_process/killproc.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_std.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_vendored_packages/__init__.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_vendored_packages/apipkg.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_vendored_packages/iniconfig.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_version.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/_xmlgen.py
> .deps/py-1.7.0-py2.py3-none-any.whl/py/test.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/__init__.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/_argcomplete.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/_code/__init__.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/_code/_py2traceback.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/_code/code.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/_code/source.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/_version.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/assertion/__init__.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/assertion/rewrite.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/assertion/truncate.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/assertion/util.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/cacheprovider.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/capture.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/compat.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/config/__init__.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/config/argparsing.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/config/exceptions.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/config/findpaths.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/debugging.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/deprecated.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/doctest.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/fixtures.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/freeze_support.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/helpconfig.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/hookspec.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/junitxml.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/logging.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/main.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/mark/__init__.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/mark/evaluate.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/mark/legacy.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/mark/structures.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/monkeypatch.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/nodes.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/nose.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/outcomes.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/pastebin.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/pathlib.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/pytester.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/python.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/python_api.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/recwarn.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/reports.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/resultlog.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/runner.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/setuponly.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/setupplan.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/skipping.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/terminal.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/tmpdir.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/unittest.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/warning_types.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/_pytest/warnings.py
> .deps/pytest-3.9.3-py2.py3-none-any.whl/pytest-3.9.3.dist-info/LICENSE
> .deps/pytest-3.9.3-py2.py3-none-any.whl/pytest-3.9.3.dist-info/METADATA
> .deps/pytest-3.9.3-py2.py3-none-any.whl/pytest-3.9.3.dist-info/RECORD
> .deps/pytest-3.9.3-py2.py3-none-any.whl/pytest-3.9.3.dist-info/WHEEL
> .deps/pytest-3.9.3-py2.py3-none-any.whl/pytest-3.9.3.dist-info/entry_points.txt
> .deps/pytest-3.9.3-py2.py3-none-any.whl/pytest-3.9.3.dist-info/top_level.txt
> .deps/pytest-3.9.3-py2.py3-none-any.whl/pytest.py
> .deps/scandir-1.9.0-cp27-cp27mu-linux_x86_64.whl/_scandir.so
> .deps/scandir-1.9.0-cp27-cp27mu-linux_x86_64.whl/scandir-1.9.0.dist-info/METADATA
> .deps/scandir-1.9.0-cp27-cp27mu-linux_x86_64.whl/scandir-1.9.0.dist-info/RECORD
> .deps/scandir-1.9.0-cp27-cp27mu-linux_x86_64.whl/scandir-1.9.0.dist-info/WHEEL
> .deps/scandir-1.9.0-cp27-cp27mu-linux_x86_64.whl/scandir-1.9.0.dist-info/top_level.txt
> .deps/scandir-1.9.0-cp27-cp27mu-linux_x86_64.whl/scandir.py
253a422,428
> .deps/six-1.11.0-py2.py3-none-any.whl/six-1.11.0.dist-info/DESCRIPTION.rst
> .deps/six-1.11.0-py2.py3-none-any.whl/six-1.11.0.dist-info/METADATA
> .deps/six-1.11.0-py2.py3-none-any.whl/six-1.11.0.dist-info/RECORD
> .deps/six-1.11.0-py2.py3-none-any.whl/six-1.11.0.dist-info/WHEEL
> .deps/six-1.11.0-py2.py3-none-any.whl/six-1.11.0.dist-info/metadata.json
> .deps/six-1.11.0-py2.py3-none-any.whl/six-1.11.0.dist-info/top_level.txt
> .deps/six-1.11.0-py2.py3-none-any.whl/six.py
@jsirois jsirois added the bug label Nov 3, 2018
This was referenced Nov 3, 2018
Closed
Closed
@jsirois jsirois mentioned this issue Dec 6, 2018
Closed
@jsirois jsirois mentioned this issue Jan 23, 2019
Closed
3 tasks
@jsirois jsirois mentioned this issue Feb 6, 2019
Closed
2 tasks
@jsirois jsirois mentioned this issue Mar 10, 2019
Closed
3 tasks
@jsirois jsirois mentioned this issue Mar 26, 2019
Closed
10 tasks
@jsirois jsirois mentioned this issue Nov 14, 2019
Merged
jsirois added a commit that referenced this issue Nov 14, 2019
@jsirois
Use pip for resolving and building distributions. (#788)
7509a89 
This changes Pex to vendor pip and defer all resolution and building to it.

As part of this change some ambiguous and / or broken features were changed and APIs adjusted:
+ The egg distribution format is no longer supported.
+ The deprecated `--interpreter-cache-dir` CLI option was removed.
+ The `--cache-ttl` CLI option and `cache_ttl` resolver API argument were removed.
+ The resolver API replaced `fetchers` with a list of `indexes` and a list of `find_links` repos.
+ The resolver API removed (http) `context` which is now automatically handled.
+ The resolver API removed `precedence` which is now pip default precedence - wheels when available and not ruled out via the `--no-wheel` CLI option or `use_wheel=False` API argument.
+ The `--platform` CLI option and `platform` resolver API argument now must be full platform strings that include platform, implementation, version and abi; e.g.: `--platform=macosx-10.13-x86_64-cp-36-m`.
+ The `--manylinux` CLI option and `use_manylinux` resolver API argument were removed. Instead, to resolve manylinux wheels for a foreign platform, specify the manylinux platform to use with an explicit `--platform` CLI flag or `platform` resolver API argument; e.g.: `--platform=manylinux2010-x86_64-cp-36-m`.

Fixes #781

Additionally:
Fixes #771
Fixes #763
Fixes #761
Fixes #735 
Fixes #694 
Fixes #660
Fixes #658 
Fixes #642
Fixes #641
Fixes #628
Fixes #620
Fixes #614
Fixes #611
Fixes #608
Fixes #439
Fixes #415
Fixes #387
Fixes #315
@jsirois jsirois self-assigned this Nov 14, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jsirois jsirois

Labels
bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Use pip for resolving and building distributions. jsirois/pex
1 participant
@jsirois