Allow a folder with a list of wheels to be a repository, mimic pip --find-links

[ThijsvandenBerg]

• I have searched the issues of this repo and believe that this is not a duplicate.
• I have searched the documentation and believe that my question is not covered.

Feature Request

The git --find-links features allows one to add a folder with a set of wheels as a repository. One can also configure git to include a folder with wheels as a repo, like so:

pip config set global.find-links N:\shared\repo

Could we have similar functionality for poetry? Have a feature that when the [[tool.poetry.source]] url is a local path like "/home/repo" or "C:\repo", or starts with "file://", then look for archives in that directory listing?

The git documentation for --find-links give a clear explanation of it's behavior:

If (the link to the repo is) a URL or path to an html file, then parse for links to archives such as sdist (.tar.gz) or wheel (.whl) files. If a local path or file:// URL that’s a directory, then look for archives in the directory listing.

[dimbleby]

duplicate #1391

[mkniewallner]

This is not exactly a duplicate of #1391 I think, since the issue here is about handling local directories as places to look into for wheels, while #1391 was specifically about handling HTTP single page repositories.

Unless Poetry supports that today (which I really don't think so), this does look like a valid feature request.

[mboisson]

Alternatively, having a ready made tool that takes a list of directories and generates an html file might fix this ?

[mboisson]

Alternatively, having a ready made tool that takes a list of directories and generates an html file might fix this ?
I think I just answered myself, the solution is to use https://pypi.org/project/dumb-pypi/ to generate an index that points to file:/// style URLs, and use a file:/// style URL within poetry configuration to point it to that index :
ComputeCanada/software-stack#114

[mboisson]

Unfortunately, that does not actually work for poetry because it uses requests, which is strictly HTTP, and does not support file:/// style URLs:

  InvalidSchema

  No connection adapters were found for 'file:///home/mboisson/tmp/generic-index/simple/torch/'

  at test_poetry/lib/python3.8/site-packages/requests/sessions.py:792 in get_adapter
      788│             if url.lower().startswith(prefix.lower()):
      789│                 return adapter
      790│
      791│         # Nothing matches :-/
    → 792│         raise InvalidSchema(f"No connection adapters were found for {url!r}")
      793│
      794│     def close(self):
      795│         """Closes all adapters and as such the session"""
      796│         for v in self.adapters.values():`

references: psf/requests#2732

[mklasby]

+1, this would be a useful feature for poetry when trying to build environments in high performance clusters that may prefer use of a local wheelhouse.

[jclerman]

Supporting this would effectively support the use-case where I want to mix a path= dependency-specification, and a version= restriction (a la #2693). I'd be able to specify the path part as a new [[tool.poetry.source]] stanza, and the version would remain in the dep-specification:

[tool.poetry.dependencies]

something-else-i-am-developing = "0.9.0"  # some version that has not yet been published in my HTTP repository(s)

[[tool.poetry.source]]
name = "private"
url = "https://repo.private.awesome.com/repository/pypi-awesome/simple"

[[tool.poetry.source]]
name = "local"
path = "../dists/"

[neersighted]

This is not actually the same thing as path dependencies -- since the dependency would be by name, and not by literal file. It's an important distinction, though many users just use path dependencies as a poor man's substitute for a custom repo.

[notkeo]

We are really looking forward to this feature to build projects in isolated environments (zero-internet access)

[alonme]

@neersighted - Would the project accept a PR for this issue?

[NicolasPA]

Similarly to @notkeo, we have to manage a python environment on a machine that is not connected to the internet for regulatory reasons.
So we copy the Python wheels ourselves to a network drive first and then install from this drive using:
pip install -r requirements.txt --no-index --find-links /path/to/python_wheels/ --disable-pip-version-check

We'd like to move to poetry to simplify our development setup and dependency management (recurrent issues with pip, venv, wheel, setup.py, requirements.txt...) but we can't because this feature is missing.

[radoering]

If the only issue is a machine that is not connected to the internet, you can setup an internal package index, e.g. pypiserver, which just takes a folder with wheels as input.

[mboisson]

If the only issue is a machine that is not connected to the internet, you can setup an internal package index, e.g. pypiserver, which just takes a folder with wheels as input.

This assumes the availability of infrastructure to host such long-running server. That is not typically the case in a cluster. Plain POSIX storage is much simpler than servers.

[notkeo]

The vendoring mechanism we are talking about is actually very popular. For example, this functionality is built into go mod

go mod vendor

https://go.dev/ref/mod#go-mod-vendor

So it would be very cool if this functionaliy appeared in poetry as well.

poetry vendor -f {path/to/packages}

[jg0110]

+1. Having this feature would've spared me the last few hours trying to use Poetry to replace my Pipenv workflow.

[soumendra]

+1

[zoonman]

Why just not do it like normal beings are doing it:

poetry add @vendor/package-name

where vendor is an alias for the sources name.

[jg0110]

See post from mboisson. "... which is strictly HTTP, and does not support file:/// style URLs"

[harrandt]

Unfortunately, that does not actually work for poetry because it uses requests, which is strictly HTTP, and does not support file:/// style URLs:

But would that be a reason to close the this feature request? Does Poetry use pip behind the scenes who could do the heavy lifting? Or would you have to implement a separate handler for the file:/// schema?

[seanswyi]

+1 I'm also trying to install PyTorch XLA using Poetry but am finding it difficult due to --find-links not being available.

[ported-pw]

This feature would make so much sense to simplify installation of local wheels per platform that GPT4o hallucinated it to me.

[theyorubayesian]

+1