--no-deps flag inside requirements.txt

[sergei3000]

What's the problem this feature will solve?

I try to run pip install -r requirements.txt but I'd like one of the packages to be installed without dependencies.

Describe the solution you'd like

Make --no-deps flag work inside requirements.txt

I use AWS Elastic Beanstalk's Python platform. It installs all the packages from requirements.txt during deployment of user's code. One of the packages I use is aiopg and I don't want it to install psycopg2-binary since it is not recommended for use in production. I have the original psycopg2 package in requirements.txt along with another aiopg's dependency async-timeout and that should be enough for aiopg to work smoothly.

Alternative Solutions

I can't find any workaround for this problem.

Additional context

[uranusjr]

Sounds like a reasonable feature request to me. Hopefully someone is interested in implementing this.

[kumiDa]

@uranusjr and @sergei3000, how do you see this feature being enabled?
I was thinking about the following options:

• Include a tag against the module in the requirements.txt file like so:

PackageName==x.y.z --no-deps

• Include it as an option to the cli command like so:

pip install --requirement requirements.txt --no-deps PackageName

[sergei3000]

Hi @rahul-kumi
This tag in requirements.txt is exactly what I was thinking about how this could be implemented.
And your idea to implement a CLI option sounds awesome too!

[uranusjr]

This is not possible because --no-deps is already a flag that takes no arguments, and suddenly making it take one will break backwards compatibility.

The possible solution I envision would be

1. Allow setting --no-deps in a requirements file on its own line (same effect as pip install -r requirements.txt --no-deps).
2. Introduce a new option (e.g. --no-deps-for) that takes a value (comma-separated package names like --no-binary and --only-binary).

[MisterLin1995]

Hi guys, kindly ask when will this feature be available?

[uranusjr]

It would be avaiable if there is contribution. Feel free to work on a pull request.

[q0w]

It would be avaiable if there is contribution. Feel free to work on a pull request.

Is not #10837 an appropriate implementation?

[MisterLin1995]

It would be avaiable if there is contribution. Feel free to work on a pull request.

@uranusjr I'm directed here from #10837, please review it if possible.

[pradyunsg]

I'm curious -- everyone who is asking for this feature to be implemented ASAP, what is your usecase?

[adam-urbanczyk]

CI where the deps are installed via other means.

[Batev]

I'm curious -- everyone who is asking for this feature to be implemented ASAP, what is your usecase?

For outdated or conflicting dependency versions.

[pfmoore]

That says to me “to deliberately install a set of dependencies which don’t satisfy the metadata”. That’s not something pip supports (and never was, not even when the bugs in the old resolver meant we couldn’t enforce this).

[Franky1]

There are definitely use cases for this --no-deps per package option. Especially when one does not have full manual control over the runtime environment installation process. This is the case for example in some cloud environments e.g. Streamlit Cloud. In this context, this option can be helpful to resolve conflicting or obsolete dependencies package by package. Or to resolve dependencies that are intended for a GUI based environment but cannot be installed without errors in a headless environment.

[pradyunsg]

(Thanks @Franky1, @Batev and @adam-urbanczyk for the quick responses)

Ok, so all the use cases stated until now are basically a subset of #8076 -- that this is a somewhat stop-gap/pip-specific solution for IIUC. If that sounds about right, please 👍🏽 on this comment. If not, please drop a comment describing your usecase!

I wanna make sure pip maintainers understand what usecases people, who are pushing for us to add this, have. :)

[Franky1]

If not, please drop a comment describing your usecase!

I disagree.
My use case is not a subset of #8076 as far as i understand.

---

The use case I'm referring to is when you don't want to attract the original dependencies of a single package, but want to install those dependencies yourself because there are non-resolvable conflicts with those dependencies or there are non-installable dependencies in certain environments.
For example, the mediapipe package has as dependencies other non-headless packages that cannot be installed without problems in a headless cloud environment.

[pfmoore]

For example, the mediapipe package has as dependencies other non-headless packages that cannot be installed without problems in a headless cloud environment.

So you want to install mediapipe in a way that doesn't satisfy its own dependency metadata? That's exactly #8076. Yes, I understand that in a headless environment, some dependencies cannot be installed - but if mediapipe supports being used in a headless environment, then surely the "non-headless-only" dependencies should be specified as an extra? And if you're using mediapipe in an unsupported way, that's not a pip issue...

[Franky1]

So you want to install mediapipe in a way that doesn't satisfy its own dependency metadata?

Yes.

but if mediapipe supports being used in a headless environment, then surely the "non-headless-only" dependencies should be specified as an extra?

Yes that would be the ideal solution. But if the maintainers don't offer this extra, although it would be technically possible to install a headless version, you have to help yourself somehow and break and resolve the dependencies yourself...

[pfmoore]

Cool. So it's #8076. Thanks for clarifying.

Edit: I also note that there's google-ai-edge/mediapipe#3661 (which you are aware of, but I'm adding here for completeness), so like many of the use cases for #8076 it's needed as a temporary fix until the package authors address the problem. (Technically, an alternative fix as you can always do something like pip install --no-deps mediapipe; pip install the-thing-that-depends-on-mediapipe)

[nathanielwoodward]

Another example usecase: I am in a situation where users need a library that depends on kerberos tools (gssapi, krb5, etc) because that is one authentication method, but there are other auth methods that do no use kerberos. It runs just fine without it locally. However, we want to install it within an app on a k8s pod that does not currently let us install the external kerberos tools (we want to use the other auth methods instead). Currently I cannot do this as the wheel build in the app tries to install all dependencies of libraries included in requirements.txt and errors out on the kerberos stuff. Maintainers are backlogged and not prioritizing switching deps to extra and I need this thing working ASAP.

[DomhnallP]

(Thanks @Franky1, @Batev and @adam-urbanczyk for the quick responses)

Ok, so all the use cases stated until now are basically a subset of #8076 -- that this is a somewhat stop-gap/pip-specific solution for IIUC. If that sounds about right, please 👍🏽 on this comment. If not, please drop a comment describing your usecase!

I wanna make sure pip maintainers understand what usecases people, who are pushing for us to add this, have. :)

I've been looking for this for a slightly different use case to most other people. we have an AWS lambda that we need to migrate to the firebase-admin package, to move away from a soon to be deprecated API. however, firebase-admin installs a tonne of dependencies that not only do we not need, but it puts us way over the 250MB Limit for lambdas. as such, i'd like to install firebase-admin without dependencies, and then only install the dependencies we actually need manually (firebase-admin is a very broad package with tonnes of functionality but we only need one very specific part of it)

[eykd]

My own use case:

0. I deploy a web application to fly.io using https://github.com/paketo-buildpacks/pip-install, which doesn't allow specification of pip's command line flags, only a requirements file. I want a secure, repeatable install, with no surprises.
1. I use pip-compile to build my requirements.txt from the pyproject.toml dependencies list, which includes all sub-dependencies pinned & pre-resolved.
2. I have a separate requirements-production.txt with the following contents:

   --require-hashes
   --no-deps
   --only-binary :all:
   -r requirements.txt
   

3. I specify requirements-production.txt as the target for the pip-install buildpack.

Note that --no-deps is strongly recommended in James Bennett's recent article Use “pip install” safely, and I was frankly surprised to learn that the other flags worked inside a requirements file, but --no-deps didn't.

[leorochael]

My use case seems slightly different. Project A that I want to install depends on project B that has become uninstallable, for instance due to setuptools drift, and the original author is unresponsive.

Project B has been forked by its community as B-forked, which is installable with modern tooling, but Project A is also unresponsive.

So I'd like to add project A, along with its healthy sub-dependencies and project B-forked in my requirements.txt, while ignoring the sub-dependencies of project A.

Arguably the most famous case of package fork renaming was PIL/Pillow, but for a concrete example, the FuelSDK package depends on suds-jurko, but has been tested to work with it's fork/successor project suds-community, a.k.a. just suds these days.

(Yes, the project has come full circle, with the fork of the fork officially taking over the original name).

So I'd like to ignore Fuel-SDK dependencies, require them myself in requirements.txt, replacing suds-jurko with suds. It's not a case of just "relaxing" dependencies/constraints, as in #8076 (as far as I understand it), but ignoring them completely.

I can do a series of manual pip invocations that put my environment in the desired configuration, but I'd rather have a single pip install -r requirements.txt, which could contain something like:

# FuelSDK has broken sub-dependencies:
--no-deps-for FuelSDK
FuelSDK
# Corrected FuelSDK dependencies manually added here:
pyjwt >= 1.5.3
requests >= 2.18.4
suds >= 1.1.2

[leorochael]

For those looking to implement this, a quick look at the code tells me that it might be possible to implement this by passing a set of project names as a ignore_dependencies_for parameter for the pip._internal.resolution.resolvelib.resolver.py:Resolver class, which would pass this parameter into pip._internal.resolution.resolvelib.provider:PipProvider.

There, in the PipProvider.get_dependencies(), method, we could have this code instead:

    def get_dependencies(self, candidate: Candidate) -> Sequence[Requirement]:
        with_requires = (
            not self._ignore_dependencies
            and candidate.project_name not in self._ignore_dependencies_for
        )
        return [r for r in candidate.iter_dependencies(with_requires) if r is not None]

Making sure that the names in the ignore_dependencies_for set are properly normalized.

[leorochael]

Please take a look at PR #12790 if you think this is a worthwhile implementation.

[leorochael]

Please take a look at PR #12790 if you think this is a worthwhile implementation.

I now also added support for the global --no-deps to the requirements file to that PR.

[leorochael]

@uranusjr, now that there is a PR, should the state: awaiting PR be changed for something else?

[pfmoore]

@leorochael IMO there's no rush. The PR is linked from here, so people will find it. I've posted a few questions over on the PR which might push this back to "needs discussion" but I'd rather get the comments addressed before worrying about labels.

[q0w]

now that there is a PR

The pr was already here

upd: your pr adds only a global cli arg to ignore deps, but you can do the same now using --no-deps wit -r (where only one dependency is provided)