Skip to content

Commit

Permalink
lazier lazy_wheel
Browse files Browse the repository at this point in the history
- handle short files `416`; prefetch entire dist-info
- lazy_wheel: translate BadZipfile to InvalidWheel
- handle 501 on negative bytes range from pypi
- catch UnsupportedWheel
- make lazy wheel work against tensorflow-gpu
- link to pypi discussion on negative byte ranges
- check for case when server doesn't support byte ranges at all
- remove _check_zip() method since we do that in prefetch_dist_info() now
- clean up error handling code and don't warn when negative ranges fail
- remove TODO, as Cache-Control: no-cache is the correct behavior
- rearrange explanatory comments
- specify how we handle 200 OK and interpret 405 as no range requests
- rename LazyZipOverHTTP to LazyWheelOverHTTP because of the assumption of the structure of *.dist-info/
- stream the initial chunk request too, since it's 1MB
- add note about fast-deps being on by default
- add testing for request limiting
- fix download metadata testing
- factor out the laziness from the wheel-specific logic
- factor out LazyRemoteResource from FixedSizeLazyResource
- add metadata_first=True arg to make_wheel
- reduce initial chunk size to 10K
- remove hardcoded compilewheel and instead generate wheels
- catch new requests decoding error
- support NegativeRangeOverflowing
- support SneakilyCoerceNegativeRange
- time out test server thread joining within 3 seconds

Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>

- make FakePackageSource to abstract over generated and hardcoded whls
- ensure InvalidWheel retains context from inner errors

Co-authored-by: Randy Döring <30527984+radoering@users.noreply.github.com>

- add link to perf evaluation from radoering
  • Loading branch information
dholth authored and cosmicexplorer committed Aug 10, 2024
1 parent e98cc5c commit 0e9aee8
Show file tree
Hide file tree
Showing 10 changed files with 1,298 additions and 143 deletions.
2 changes: 2 additions & 0 deletions news/12208.feature.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Implement lazier lazy_wheel that avoids the HEAD request, fetching the metadata
for many wheels in 1 request.
6 changes: 4 additions & 2 deletions src/pip/_internal/exceptions.py
Original file line number Diff line number Diff line change
Expand Up @@ -323,12 +323,14 @@ class UnsupportedWheel(InstallationError):
class InvalidWheel(InstallationError):
"""Invalid (e.g. corrupt) wheel."""

def __init__(self, location: str, name: str):
def __init__(self, location: str, name: str, context: Optional[str] = None):
self.location = location
self.name = name
self.context = context

def __str__(self) -> str:
return f"Wheel '{self.name}' located at {self.location} is invalid."
suffix = f" ({self.context})" if self.context else "."
return f"Wheel '{self.name}' located at {self.location} is invalid{suffix}"


class MetadataInconsistent(InstallationError):
Expand Down
Loading

0 comments on commit 0e9aee8

Please sign in to comment.