Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is documented near the beginning of
https://packaging.python.org/en/latest/specifications/core-metadata/ and warehouse currently serves these as
content-type: application/octet-stream
with no encoding, so requests spends a surprising amount of time on what are generally lower-ascii files trying to guess an encoding.I think this is 100% safe -- files that can't be understood as utf-8 appear to not work in pip, which reads them as bytes, saves them as bytes and importlib always uses utf-8. A better API for pypi-simple might be to return bytes, but that's more disruptive.
Sample debug log pydantic-2.6.1-py3-none-any.whl.metadata showing more than 300ms spent trying to load the thing, only to give up and treat it as utf-8 anyway.
You might not notice this in a normal application, but I'm fetching these in a loop, with cachecontrol as the session, with a hot cache.
Here are traces for this change (the green block is
get_package_metadata
and the total time is visible top-right):Before:
After: