Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python 3.12.4: Extract downloaded archive; gzip: stdin: not in gzip format #886

Closed
2 of 5 tasks
frenck opened this issue Jun 10, 2024 · 22 comments · Fixed by home-assistant/core#119454
Closed
2 of 5 tasks
Labels
bug Something isn't working

Comments

@frenck
Copy link

frenck commented Jun 10, 2024

Description:
A clear and concise description of what the bug is.

CleanShot 2024-06-10 at 07 51 57@2x

https://github.com/home-assistant/core/actions/runs/9442844577/job/26006478894?pr=119255

Action version:
Specify the action version

actions/setup-python@v5.1.0

Platform:

  • Ubuntu
  • macOS
  • Windows

Runner type:

  • Hosted
  • Self-hosted

Tools version:

3.12 (resulting in 3.12.4 right now)

Repro steps:
A description with steps to reproduce the issue. If your have a public example or repo to share, please provide the link.

      - name: Set up Python ${{ env.DEFAULT_PYTHON }}
        id: python
        uses: actions/setup-python@v5.1.0
        with:
          python-version: ${{ env.DEFAULT_PYTHON }}
          check-latest: true

https://github.com/home-assistant/core/blob/159503b8d37fbc46a353287416a640ad07c2cced/.github/workflows/ci.yaml#L230-L235

Expected behavior:
A description of what you expected to happen.

Actual behavior:
A description of what is actually happening.

@hugovk
Copy link
Contributor

hugovk commented Jun 10, 2024

Same thing with 3.13.0b2:

Run actions/setup-python@v5
Installed versions
  Version ~3.13.0-0 was not found in the local cache
  Version ~3.13.0-0 is available for downloading
  Download from "https://github.com/actions/python-versions/releases/download/3.13.0-beta.2-9442465578/python-3.13.0-beta.2-linux-22.04-x64.tar.gz"
  Extract downloaded archive
  /usr/bin/tar xz --warning=no-unknown-keyword --overwrite -C /home/runner/work/_temp/ce7441c9-c7c3-405b-8d85-f47fd5f8a722 -f /home/runner/work/_temp/8c32cd5a-e1e8-4874-ac7a-a0c200b8ba14
  
  gzip: stdin: not in gzip format
  /usr/bin/tar: Child returned status 1
  /usr/bin/tar: Error is not recoverable: exiting now
  Error: The process '/usr/bin/tar' failed with exit code 2

https://github.com/pypa/pip/actions/runs/9443536250/job/26007467646

Both 3.12.4 and 3.13.0-beta.2 were released on https://github.com/actions/python-versions a couple of hours ago:

@Foxboron
Copy link

Foxboron commented Jun 10, 2024

Seems like the header is incorrect?

λ Downloads » xxd python-3.12.4-linux-22.04-x64.tar.gz | less | head -1
00000000: 1fef bfbd 0800 0000 0000 0003 efbf bd3c  ...............<

1fef isn't a valid magic number for gzip (file is unaware of anything using it) so I'm not sure if this is a weird case of bitflip? Rest of the header seems fine.

@AGulev
Copy link

AGulev commented Jun 10, 2024

the same with macos and win archives :

  Version 3.12 was not found in the local cache
  Version 3.12 is available for downloading
  Download from "https://github.com/actions/python-versions/releases/download/3.12.4-9442300799/python-3.12.4-darwin-x64.tar.gz"
  Extract downloaded archive
  /usr/bin/tar xz -C /Users/runner/work/_temp/f2bc8ae4-bdba-46b8-823a-bd909c45dd1b -f /Users/runner/work/_temp/623e67b2-44e6-4a3d-8516-abe6e9d12231
  tar: Error opening archive: Unrecognized archive format
  Error: The process '/usr/bin/tar' failed with exit code 1

AGulev added a commit to defold/defold that referenced this issue Jun 10, 2024
@firewave
Copy link

Seems like the header is incorrect?

The file is completely garbled. It mostly consists of ef bf bd patterns.

@ay0o
Copy link

ay0o commented Jun 10, 2024

I have noticed this is working in ubuntu-latest runner, but not on my self-hosted runner, both using the same container: ubuntu:24.04 for testing purposes.

@MarcHagen
Copy link

MarcHagen commented Jun 10, 2024

I have noticed this is working in ubuntu-latest runner, but not on my self-hosted runner, both using the same container: ubuntu:24.04 for testing purposes.

The hosted runners have an cached toolchain with multiple versions of python already installed. The action is using that instead downloading an version.
The selfhosted image does not have sush toolchain.

GitHub hosted is running on 50Gb+ VMs in Azure to minimize download and install cycles.

Edit:
Info location hosted runner "image": https://github.com/actions/runner-images/blob/main/images/ubuntu/Ubuntu2404-Readme.md

akien-mga added a commit to akien-mga/godot that referenced this issue Jun 10, 2024
macOS runners can't extract the 3.12.4 tarball properly it seems.
actions/setup-python#886
firewave added a commit to danmar/cppcheck that referenced this issue Jun 10, 2024
@Foxboron
Copy link

The file is completely garbled. It mostly consists of ef bf bd patterns.

Ah, nice spotting.

However it's weird that the deflate header is kept as-is while the 1f bytes are correct.

@firewave
Copy link

However it's weird that the deflate header is kept as-is while the 1f bytes are correct.

The ef bf bd pattern is a specific symbol: https://www.cogsci.ed.ac.uk/~richard/utf-8.cgi?input=%EF%BF%BD&mode=char

That is the symbol you usually see when UTF-8 is being displayed as ASCII. It looks the files have been treated as text causing all the non-ASCII (i.e. binary) data being treated as unprintable characters thus corrupting it in the process.

@RobPasMue
Copy link

This might be related to the actions/upload-artifact new version (v4). We've had problems in the past with incompatibilities, wrong compression mechanisms etc. They upgraded the version not that long ago: actions/python-versions@c990e6d

This commit is dated on May 22nd whereas the last previous release is from May 9th

@HarithaVattikuti
Copy link
Contributor

Hello @frenck
Thankyou for creating this issue. we are working on the fix and will get back to you sooner.

@firewave
Copy link

firewave commented Jun 10, 2024

The hashes do not match at all.

3.13.0-beta.2

$ sha256sum -c hashes.sha256
python-3.13.0-beta.2-darwin-arm64.tar.gz: FAILED
python-3.13.0-beta.2-darwin-x64.tar.gz: FAILED
python-3.13.0-beta.2-linux-20.04-x64.tar.gz: FAILED
python-3.13.0-beta.2-linux-22.04-x64.tar.gz: FAILED
python-3.13.0-beta.2-linux-24.04-x64.tar.gz: FAILED
python-3.13.0-beta.2-win32-x64.zip: FAILED
python-3.13.0-beta.2-win32-x86.zip: FAILED
sha256sum: WARNING: 7 computed checksums did NOT match

3.12.4

$ sha256sum -c hashes.sha256
python-3.12.4-darwin-arm64.tar.gz: FAILED
python-3.12.4-darwin-x64.tar.gz: FAILED
python-3.12.4-linux-20.04-x64.tar.gz: FAILED
python-3.12.4-linux-22.04-x64.tar.gz: FAILED
python-3.12.4-linux-24.04-x64.tar.gz: FAILED
python-3.12.4-win32-x64.zip: FAILED
python-3.12.4-win32-x86.zip: FAILED
sha256sum: WARNING: 7 computed checksums did NOT match

The files are also double in size than the previous versions because of the corruption. The source archives are fine.

3.12.3

$ sha256sum -c hashes.sha256
python-3.12.3-darwin-arm64.tar.gz: OK
python-3.12.3-darwin-x64.tar.gz: OK
python-3.12.3-linux-20.04-x64.tar.gz: OK
python-3.12.3-linux-22.04-x64.tar.gz: OK
python-3.12.3-linux-24.04-x64.tar.gz: OK
python-3.12.3-win32-x64.zip: OK
python-3.12.3-win32-x86.zip: OK

@henryiii
Copy link

Download artifact will corrupt files if there are any matching file names when merging multiple, this looks suspiciously like that. actions/download-artifact#298

@mayeut
Copy link
Contributor

mayeut commented Jun 10, 2024

The manifest that holds broken releases has been reverted by @HarithaVattikuti : actions/python-versions#282

This removes 3.12.4 & 3.13.0b2. (but installing latest 3.12 or dev 3.13 now works again - using 3.12.3 & 3.13.0b1).

actions/python-versions#285 should resolve the issue.

AGulev added a commit to defold/defold that referenced this issue Jun 11, 2024
* Update xcode to 15.4

* update runners

* remove `arch -x86_64` for python

* use python 3.12.3 as a workaround for actions/setup-python#886

* do not specify arch
@aparnajyothi-y
Copy link
Contributor

Hello Everyone, we merged the fix for this issue. Please validate from your end and confirm.

@mayeut
Copy link
Contributor

mayeut commented Jun 12, 2024

Thanks @aparnajyothi-y, I can confirm the latest release / manifest fixes the issue in #863 (some GH-hosted workers) and #884 (all GH-hosted workers).

@AlexWaygood
Copy link

All looks good for us at flake8-pyi and typeshed now -- thanks @aparnajyothi-y!

@Avasam
Copy link

Avasam commented Jun 16, 2024

CI now passes at pywin32 for Python 3.13-dev as well! mhammond/pywin32#2260

@frenck
Copy link
Author

frenck commented Jun 16, 2024

Some good confirmations above. Can confirm it works on our end too. Closing the issue for that reason.

Thanks for the fix! ❤️

../Frenck

@frenck frenck closed this as completed Jun 16, 2024
AGulev added a commit to defold/defold that referenced this issue Jun 25, 2024
* Update xcode to 15.4

* update runners

* remove `arch -x86_64` for python

* use python 3.12.3 as a workaround for actions/setup-python#886

* do not specify arch
MewPurPur pushed a commit to MewPurPur/godot that referenced this issue Jul 11, 2024
macOS runners can't extract the 3.12.4 tarball properly it seems.
actions/setup-python#886
sorascode pushed a commit to sorascode/godot-soras-version that referenced this issue Jul 22, 2024
macOS runners can't extract the 3.12.4 tarball properly it seems.
actions/setup-python#886
2nafish117 pushed a commit to 2nafish117/godot that referenced this issue Aug 5, 2024
macOS runners can't extract the 3.12.4 tarball properly it seems.
actions/setup-python#886
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.