pip install “offline mode”

[uranusjr]

What's the problem this feature will solve?
I started wondering whether this is possible when reading #8043. When pip initialises an isolated build environment, it currently access package repositories to get the build tools. This can however be cumbersome when the user does not have Internet access.

Describe the solution you'd like
An --offline flag that makes pip use only the network cache to install things. This includes sdists, wheels, and also the index/find-links pages. This would solve the network-less bootstrapping problem as long as the user has installed all the build tools before (and has not cleared the cache). The flag would also need to be passed to PEP 517 hooks.

Alternative Solutions
Currently the user can use --no-index --find-links and point pip to the relevent wheels. But this needs the user to plan ahead and download the wheels. pip’s network cache does not expose the file names, and is not practically usable as a --fine-links source.

As a hack, the wheels of setuptools and wheel are actually available in stdlib’s ensurepip package. But this does not work with other PEP 517 backends.

Additional context
This probably needs some works with improving pip’s network cache, and might even be impossible (I haven’t poked at the cache implementation). But I feel this would be a good feature if it’s possible to implement.

[kjpgit]

npm has a --prefer-offline mode that I think is even better.
It works great with say AWS Codebuild S3 bucket caching.
If a package is in cache, it uses it and does not access the network even to check for a newer version.
It only would access the network as a last resort.
So builds run reliably and don't break if pypi (or npm) is down.

[paugier]

Note that UV now has a global option

--offline                                    Disable network access [env: UV_OFFLINE=]

[zahlman]

This probably needs some works with improving pip’s network cache, and might even be impossible (I haven’t poked at the cache implementation). But I feel this would be a good feature if it’s possible to implement.

I did some investigation. "Impossible" is too strong, but without fundamentally changing the cache implementation, it would be probably quite a bit of work. Pip would have to start storing some metadata about the cache contents (perhaps in the root of the cache folder, or maybe it wouldn't cause a problem to add it to the http-v2 folder?) that it currently just doesn't have.

The basic problem is that all the stored files (wheels, sdists and now .metadata files) are named and stored according to a hash (SHA224) of the URL (without the fragment used to supply a hash for verification, if present) from which they were downloaded. So Pip doesn't actually know what's in there - it just uses cachecontrol to check the cache first when it wants to make an HTTP request. If you explore the http-v2 cache folder you can easily find e.g. the .whl files renamed as some hex digits and a .body extension (and a corresponding extension-less file with some HTTP header etc. information). But there's nothing to say which wheel it is, except to figure out the original URL.

The response header doesn't give the wheel name, but it does give the package name, version, and possibly the python-version wheel tag. So that can be used to make an API query, find candidate URLs for that package version (and add .metadata for those files), and brute-force the hash. And then the actual filename is directly included in the URL - at least for PyPI, anyway. (I think it needs to be to conform to standards now.)

I'm in the middle of some code now to take that approach, iterate over a Pip cache and migrate the contents to a Uv cache. Perhaps it will be useful as a model for anyone who wants to implement a Pip offline mode feature. Although I guess the practical approach is to just record an entry in a "wheel name -> [wheel cache key, metadata cache key]" database (could be a simple JSON or TOML file) whenever a file is downloaded.

[ichard26]

Thank you @zahlman for the preliminary investigation! It's appreciated. This is very much a low priority feature, but if someone does get to it, your comment will be useful!