Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perform 1-3 HTTP requests for each wheel using fast-deps #12208

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Commits on Aug 10, 2024

  1. lazier lazy_wheel

    - handle short files `416`; prefetch entire dist-info
    - lazy_wheel: translate BadZipfile to InvalidWheel
    - handle 501 on negative bytes range from pypi
    - catch UnsupportedWheel
    - make lazy wheel work against tensorflow-gpu
    - link to pypi discussion on negative byte ranges
    - check for case when server doesn't support byte ranges at all
    - remove _check_zip() method since we do that in prefetch_dist_info() now
    - clean up error handling code and don't warn when negative ranges fail
    - remove TODO, as Cache-Control: no-cache is the correct behavior
    - rearrange explanatory comments
    - specify how we handle 200 OK and interpret 405 as no range requests
    - rename LazyZipOverHTTP to LazyWheelOverHTTP because of the assumption of the structure of *.dist-info/
    - stream the initial chunk request too, since it's 1MB
    - add note about fast-deps being on by default
    - add testing for request limiting
    - fix download metadata testing
    - factor out the laziness from the wheel-specific logic
    - factor out LazyRemoteResource from FixedSizeLazyResource
    - add metadata_first=True arg to make_wheel
    - reduce initial chunk size to 10K
    - remove hardcoded compilewheel and instead generate wheels
    - catch new requests decoding error
    - support NegativeRangeOverflowing
    - support SneakilyCoerceNegativeRange
    - time out test server thread joining within 3 seconds
    
    Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>
    
    - make FakePackageSource to abstract over generated and hardcoded whls
    - ensure InvalidWheel retains context from inner errors
    
    Co-authored-by: Randy Döring <30527984+radoering@users.noreply.github.com>
    
    - add link to perf evaluation from radoering
    dholth authored and cosmicexplorer committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    0e9aee8 View commit details
    Browse the repository at this point in the history
  2. make all generated servers pick an arbitrary free port

    ideally, this should avoid collisions. no clue if this matters at all
    cosmicexplorer committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    e4b2aaf View commit details
    Browse the repository at this point in the history

Commits on Aug 11, 2024

  1. Configuration menu
    Copy the full SHA
    b06d73b View commit details
    Browse the repository at this point in the history