Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pex packaging code from git? (git+https support?) #439

Closed
itdaniher opened this issue Dec 22, 2017 · 5 comments · Fixed by #788
Closed

pex packaging code from git? (git+https support?) #439

itdaniher opened this issue Dec 22, 2017 · 5 comments · Fixed by #788

Comments

@itdaniher
Copy link

Like #429, I'm trying to use pex with Pipenv. Unlike #429, I'm trying package code from an upstream git repo into a pex file.

pipenv lock -r | awk -F ' --' '{print $1}' > requirements.txt

giving me a requirements.txt including lines like:

git+https://github.com/Blosc/python-blosc.git#egg=blosc
git+https://github.com/pyca/pynacl.git#egg=pynacl

which pex chokes on:

Traceback (most recent call last):
  File "/home/name/.pex/install/pex-1.2.15-py2.py3-none-any.whl.c57747ed87261721509db19020b7431dec741af3/pex-1.2.15-py2.py3-none-any.whl/pex/requirements.py", line 80, in requirements_from_lines
    resolvables.append(Resolvable.get(resolvable, builder))
  File "/home/name/.pex/install/pex-1.2.15-py2.py3-none-any.whl.c57747ed87261721509db19020b7431dec741af3/pex-1.2.15-py2.py3-none-any.whl/pex/resolvable.py", line 71, in get
    raise cls.InvalidRequirement('Unknown requirement type: %s' % resolvable_string)
pex.resolvable.InvalidRequirement: Unknown requirement type: git+https://github.com/Blosc/python-blosc.git#egg=blosc

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ".bootstrap/_pex/pex.py", line 365, in execute
  File ".bootstrap/_pex/pex.py", line 293, in _wrap_coverage
  File ".bootstrap/_pex/pex.py", line 325, in _wrap_profiling
  File ".bootstrap/_pex/pex.py", line 408, in _execute
  File ".bootstrap/_pex/pex.py", line 466, in execute_entry
  File ".bootstrap/_pex/pex.py", line 484, in execute_pkg_resources
  File "/home/name/.pex/install/pex-1.2.15-py2.py3-none-any.whl.c57747ed87261721509db19020b7431dec741af3/pex-1.2.15-py2.py3-none-any.whl/pex/bin/pex.py", line 648, in main
    pex_builder = build_pex(reqs, options, resolver_options_builder)
  File "/home/name/.pex/install/pex-1.2.15-py2.py3-none-any.whl.c57747ed87261721509db19020b7431dec741af3/pex-1.2.15-py2.py3-none-any.whl/pex/bin/pex.py", line 576, in build_pex
    resolvables.extend(requirements_from_file(requirements_txt, resolver_option_builder))
  File "/home/name/.pex/install/pex-1.2.15-py2.py3-none-any.whl.c57747ed87261721509db19020b7431dec741af3/pex-1.2.15-py2.py3-none-any.whl/pex/requirements.py", line 98, in requirements_from_file
    return requirements_from_lines(fp.readlines(), builder=builder, relpath=relpath)
  File "/home/name/.pex/install/pex-1.2.15-py2.py3-none-any.whl.c57747ed87261721509db19020b7431dec741af3/pex-1.2.15-py2.py3-none-any.whl/pex/requirements.py", line 82, in requirements_from_lines
    raise UnsupportedLine('Could not resolve line: %s (%s)' % (resolvable, e))
pex.requirements.UnsupportedLine: Could not resolve line: git+https://github.com/Blosc/python-blosc.git#egg=blosc (Unknown requirement type: git+https://github.com/Blosc/python-blosc.git#egg=blosc)

Is this a desired feature of pex? Am I doing something wrong?

Thanks!

@jcrowgey
Copy link

+1

@kevinkjt2000
Copy link

This would help out tremendously! Sometimes it is necessary to install from a git version of a package until a release is made available. Even worse, when a package is abandoned and it takes weeks to get control over it.

@TheoLechemia
Copy link

+1

@jsirois
Copy link
Member

jsirois commented Jul 10, 2018

I think like for #429 a PR implementing this support would be welcome.

@jaraco
Copy link
Contributor

jaraco commented Dec 20, 2018

Today I wanted to test this branch of importlib_metadata against pex, but couldn't readily due to this issue. I'd love to see this.

I was able to work around the issue by:

$ pip wheel --no-deps 'git+https://gitlab.com/python-devs/importlib_metadata@feature/find-local-distribution'
$ pex requests ./importlib_metadata*.whl -o requests.pex

@jsirois jsirois self-assigned this Nov 14, 2019
jsirois added a commit that referenced this issue Nov 14, 2019
This changes Pex to vendor pip and defer all resolution and building to it.

As part of this change some ambiguous and / or broken features were changed and APIs adjusted:
+ The egg distribution format is no longer supported.
+ The deprecated `--interpreter-cache-dir` CLI option was removed.
+ The `--cache-ttl` CLI option and `cache_ttl` resolver API argument were removed.
+ The resolver API replaced `fetchers` with a list of `indexes` and a list of `find_links` repos.
+ The resolver API removed (http) `context` which is now automatically handled.
+ The resolver API removed `precedence` which is now pip default precedence - wheels when available and not ruled out via the `--no-wheel` CLI option or `use_wheel=False` API argument.
+ The `--platform` CLI option and `platform` resolver API argument now must be full platform strings that include platform, implementation, version and abi; e.g.: `--platform=macosx-10.13-x86_64-cp-36-m`.
+ The `--manylinux` CLI option and `use_manylinux` resolver API argument were removed. Instead, to resolve manylinux wheels for a foreign platform, specify the manylinux platform to use with an explicit `--platform` CLI flag or `platform` resolver API argument; e.g.: `--platform=manylinux2010-x86_64-cp-36-m`.

Fixes #781

Additionally:
Fixes #771
Fixes #763
Fixes #761
Fixes #735 
Fixes #694 
Fixes #660
Fixes #658 
Fixes #642
Fixes #641
Fixes #628
Fixes #620
Fixes #614
Fixes #611
Fixes #608
Fixes #439
Fixes #415
Fixes #387
Fixes #315
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants