Un-deprecate `--process-dependency-links` until an alternative is implemented

[multimeric]

• Pip version: 8.1.1
• Python version: 3.5.2
• Operating System: Ubuntu 16.04

Description:

The --process-dependency-links was re-added to pip a while ago because there are a number of valid use-cases for the flag: #1955

For this reason, it should probably be un-deprecated until an implementation of PEP 508/440 is in pip. The dependency_links argument to setuptools's setup is also still documented and not deprecated: http://setuptools.readthedocs.io/en/latest/setuptools.html#dependencies-that-aren-t-in-pypi

What I've run:

$ pip install request --process-dependency-links
Collecting request
  Downloading request-0.0.12.tar.gz
  DEPRECATION: Dependency Links processing has been deprecated and will be removed in a future release.
... 

[Kxrr]

Agreed. Current behaviour of pip for dependency-links really sucks.
#4295
#3610

[jleclanche]

[Wrote in #3939, moving my comment here]

So, here's the main issue: If I don't want to use a requirements.txt file (because, say, I want declarative dependencies all specified in setup.cfg), how am I supposed to specify a URL as dependency?

This doesn't work:

[options]
install_requires = 
  requests==2.18.4
  https://github.com/example/example_lib/archive/master.zip

I also think dependency links are weird and fine to drop, but canonically how is this use case served if not with those?

[jleclanche]

Another issue is that dependency_links apparently cannot be specified in setup.cfg, only as setup.py arguments. If they are to be undeprecated, that should be fixed.

[robertour]

Is there any news regarding this? Is there any alternative in progress? dependency_links_ seems to be the only option for internal/private distribution of libraries.

Instead of deprecating it, it should be fixed, a lot of people forget or get confuse because they forget to put both the name-version of the library in the the install_requires and the dependency link that contains the tarbal together with the egg. As an alternative, I would love to see that @jleclanche has suggested, a simple link to the tarball in the install_requires (specifying the egg could be optional).

I don't think we are alone: https://stackoverflow.com/questions/3472430/how-can-i-make-setuptools-install-a-package-thats-not-on-pypi

[RonnyPfannschmidt]

@robertour tools like devpi support a more clear approach since years now

[RonnyPfannschmidt]

@dstufft btw - i would like to argue that the existene of devpi demonstrates a more clean solution to the problem making dependency links unnecessary

IMHO index inheritance + white listing/blacklisting is more consistent and more comprehensible at a ops level

[robertour]

@RonnyPfannschmidt, thanks for the suggestion, it seems I am missing something important (discussions about this deprecation have been going for too long) but generally speaking I cannot agree more with this comment from reddit:

devpi could be a solution, but bringing another service into the game that needs maintenance and having to wait for some administrator to setup, will not get me into working very soon. Also we still would have the need to build packages and then put them onto devip (or has devpi a feature to track git repositories??)

Ideally, I would like an alternative that won't take me more than adding a file in my library, or few lines in the setup.py.

I basically don't understand the underlying decision of removing it, I can see how it can lead to bad practices and insecurities, but as long as it is keep for internal use, it should not be a major issue.

[pradyunsg]

#4175 means that PEP 508 URL support will be there in pip 10. I think this can be closed.

Thoughts @pfmoore @dstufft?

[pfmoore]

I thought 86b0779 meant that the @ syntax cannot be used for dependencies, so this means #4175 isn't a solution for this issue. The comments over there were that we shouldn't implement @ support for dependencies until PyPI was able to block uploads that used it (because we don't want installing something from PyPI to be able to grab stuff from arbitrary URLs, presumably).

[rgerkin]

It shouldn't be deprecated until there is an alternative in place, i.e. PEP 508 support. Until then, too many people are using it.

What's the proper workflow for this?

Let 's say there's a library that I need to add a feature to. I fork it on Github. Feature isn't going to be merged any time soon, so I have a tool that I set to use my Github fork instead of PyPi.

Now all of my users have to add --process-dependency-links when installing my tool with pip, which is a deprecated and even once removed flag?

Is there some option in setup.py that I'm missing or is there really no way around this? It seems like the only viable way to do this is to push up a forked pypi package. That is just going to add to user's confusion once your pull request does get merged, anyway.