Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retry on tar extraction errors #9753

Merged
merged 1 commit into from
Dec 10, 2024
Merged

Retry on tar extraction errors #9753

merged 1 commit into from
Dec 10, 2024

Conversation

charliermarsh
Copy link
Member

@charliermarsh charliermarsh commented Dec 9, 2024

Summary

So the error here is:

ExtractError("cpython-3.11.11%2B20241206-aarch64-apple-darwin-install_only_stripped.tar.gz", Io(Custom { kind: UnexpectedEof, error: TarError { desc: "failed to unpack `/Users/crmarsh/.local/share/uv/python/.cache/.tmpkqFzqE/python/lib/libpython3.11.dylib`", io: Custom { kind: UnexpectedEof, error: TarError { desc: "failed to unpack `python/lib/libpython3.11.dylib` into `/Users/crmarsh/.local/share/uv/python/.cache/.tmpkqFzqE/python/lib/libpython3.11.dylib`", io: Custom { kind: UnexpectedEof, error: "unexpected end of file" } } } } }))

This isn't a Reqwest error, so we miss it in is_extended_transient_error.

We could add TarError or ExtractError here, but... should we? This PR just extends it to any error that has an IO source. I don't see much of a downside.

Closes #9747.

Test Plan

First, ran: uv run ./scripts/create-python-mirror.py --name cpython --arch aarch64 --os darwin.

Then, dropped this into ./scripts/mirror/server.py:

import os
import random
from http.server import SimpleHTTPRequestHandler, HTTPServer


class GlitchyStaticServer(SimpleHTTPRequestHandler):
    def do_GET(self):
        """Handle GET request."""
        file_path = self.translate_path(self.path)
        
        if not os.path.exists(file_path):
            self.send_error(404, "File not found")
            return
        
        try:
            with open(file_path, 'rb') as f:
                file_content = f.read()

            # Introduce an "unexpected end of file" glitch randomly
            if random.random() < 0.75:  # 75% chance of glitch
                glitch_point = random.randint(1, len(file_content) - 1)
                file_content = file_content[:glitch_point]

            self.send_response(200)
            self.send_header("Content-type", self.guess_type(file_path))
            self.send_header("Content-Length", len(file_content))
            self.end_headers()
            self.wfile.write(file_content)
        
        except Exception as e:
            self.send_error(500, f"Internal Server Error: {e}")
        

def run(server_class=HTTPServer, handler_class=GlitchyStaticServer, port=8080):
    """Run the server."""
    server_address = ('', port)
    httpd = server_class(server_address, handler_class)
    print(f"Serving on port {port} with glitchy behavior")
    httpd.serve_forever()


if __name__ == "__main__":
    run()

Then ran python server.py from that directory.

From there, ran UV_PYTHON_INSTALL_MIRROR="http://localhost:8080" cargo run python install 3.11 --reinstall --verbose to reliably test retries.

@charliermarsh charliermarsh added bug Something isn't working network Network connectivity e.g. proxies, DNS, and SSL labels Dec 9, 2024
Copy link
Member

@konstin konstin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

@charliermarsh charliermarsh merged commit eb21e4b into main Dec 10, 2024
64 checks passed
@charliermarsh charliermarsh deleted the charlie/py-retry branch December 10, 2024 12:33
tmeijn pushed a commit to tmeijn/dotfiles that referenced this pull request Dec 12, 2024
This MR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [astral-sh/uv](https://github.com/astral-sh/uv) | patch | `0.5.7` -> `0.5.8` |

MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot).

**Proposed changes to behavior should be submitted there as MRs.**

---

### Release Notes

<details>
<summary>astral-sh/uv (astral-sh/uv)</summary>

### [`v0.5.8`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#058)

[Compare Source](astral-sh/uv@0.5.7...0.5.8)

**This release does not include the `powerpc64le-unknown-linux-musl` target due to a build issue. See [#&#8203;9793](astral-sh/uv#9793) for details. If this change affects you, please file an issue with your use-case.**

##### Enhancements

-   Omit empty resolution markers in lockfile  ([#&#8203;9738](astral-sh/uv#9738))
-   Add `--install-dir` to to `uv python install` and `uninstall` commands ([#&#8203;7920](astral-sh/uv#7920))
-   Add `--show-urls` and `--only-downloads` to `uv python list` ([#&#8203;8062](astral-sh/uv#8062))
-   Add `uv python list --all-arches` ([#&#8203;9782](astral-sh/uv#9782))
-   Add `uv run --gui-script` flag for running Python scripts with `pythonw.exe` ([#&#8203;9152](astral-sh/uv#9152))
-   Allow `--gui-script` on Unix ([#&#8203;9787](astral-sh/uv#9787))
-   Allow download of Python distribution variants optimized for newer x86\_64 microarchitectures ([#&#8203;9781](astral-sh/uv#9781))
-   Allow execution of `pyw` files on Unix ([#&#8203;9759](astral-sh/uv#9759))
-   Allow users to specify URLs in `project.dependencies` and `tool.uv.sources` ([#&#8203;9718](astral-sh/uv#9718))
-   Encode mutually-incompatible pairs of markers ([#&#8203;9444](astral-sh/uv#9444))
-   Improve the error message when a Python install request is not valid ([#&#8203;9783](astral-sh/uv#9783))
-   Preserve directory-level standalone build symlinks ([#&#8203;9723](astral-sh/uv#9723))
-   Add support for `uv publish --index <name>` ([#&#8203;9694](astral-sh/uv#9694))
-   Reframe `--locked` and `--frozen` as `--check` operations for `uv lock` ([#&#8203;9662](astral-sh/uv#9662))
-   Rename Python install scratch directory from `.cache` -> `.temp` ([#&#8203;9756](astral-sh/uv#9756))
-   Enable `uv tool uninstall uv` on Windows ([#&#8203;8963](astral-sh/uv#8963))
-   Improve self-dependency hint to make shadowing clear ([#&#8203;9716](astral-sh/uv#9716))
-   Refactor unavailable metadata to shrink the resolver ([#&#8203;9769](astral-sh/uv#9769))
-   Show 'depends on itself' for proxy packages ([#&#8203;9717](astral-sh/uv#9717))
-   Show a dedicated error for missing subdirectories ([#&#8203;9761](astral-sh/uv#9761))
-   Show a dedicated hint for missing `git+` prefixes ([#&#8203;9789](astral-sh/uv#9789))

##### Performance

-   Eagerly error when parsing `pyproject.toml` requirements ([#&#8203;9704](astral-sh/uv#9704))
-   Use copy-on-write when normalizing paths ([#&#8203;9710](astral-sh/uv#9710))

##### Bug fixes

-   Avoid enforcing non-conflicts in `uv export` ([#&#8203;9751](astral-sh/uv#9751))
-   Don't drop comments between items in TOML tables ([#&#8203;9784](astral-sh/uv#9784))
-   Don't fail with `--no-build` when static metadata is available ([#&#8203;9785](astral-sh/uv#9785))
-   Don't filter non-patch registry version ([#&#8203;9736](astral-sh/uv#9736))
-   Don't read metadata from stale `.egg-info` files ([#&#8203;9760](astral-sh/uv#9760))
-   Enforce correctness of self-dependencies ([#&#8203;9705](astral-sh/uv#9705))
-   Fix projects's typo in resolver error messages ([#&#8203;9708](astral-sh/uv#9708))
-   Ignore `.` prefixed directories during managed Python installation discovery ([#&#8203;9786](astral-sh/uv#9786))
-   Improve handling of invalid virtual environments during interpreter discovery ([#&#8203;8086](astral-sh/uv#8086))
-   Normalize relative paths when `--project` is specified ([#&#8203;9709](astral-sh/uv#9709))
-   Respect self-constraints on recursive extras ([#&#8203;9714](astral-sh/uv#9714))
-   Respect user settings for tracing coloring ([#&#8203;9733](astral-sh/uv#9733))
-   Retry on tar extraction errors ([#&#8203;9753](astral-sh/uv#9753))
-   Add conflict markers to the lock file ([#&#8203;9370](astral-sh/uv#9370))
-   De-duplicate resolution markers ([#&#8203;9780](astral-sh/uv#9780))
-   Avoid 403 error hint for PyTorch URLs ([#&#8203;9750](astral-sh/uv#9750))
-   Avoid treating non-existent `--find-links` as relative URLs ([#&#8203;9720](astral-sh/uv#9720))
-   Omit Windows Store `python3.13.exe` et al ([#&#8203;9679](astral-sh/uv#9679))
-   Replace executables with broken symlinks during `uv python install` ([#&#8203;9706](astral-sh/uv#9706))

##### Documentation

-   Fix build failure links ([#&#8203;9740](astral-sh/uv#9740))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this MR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box

---

This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40NDAuNyIsInVwZGF0ZWRJblZlciI6IjM3LjQ0MC43IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJSZW5vdmF0ZSBCb3QiXX0=-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working network Network connectivity e.g. proxies, DNS, and SSL
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Downloading python from behind corporate proxy fails to download in uv but works with wget and rye
2 participants