raise BadZipFile("Bad magic number for central directory") #215

craigstar · 2024-02-22T16:30:04Z

**Run pypa/gh-action-pypi-publish@release/v1**
/usr/bin/docker run --name bac05ded6ea2e388f648368fab0951ff555dd1_69c6d2 --label bac05d --workdir /github/workspace --rm -e "INPUT_SKIP-EXISTING" -e "INPUT_USER" -e "INPUT_PASSWORD" -e "INPUT_REPOSITORY-URL" -e "INPUT_REPOSITORY_URL" -e "INPUT_PACKAGES-DIR" -e "INPUT_PACKAGES_DIR" -e "INPUT_VERIFY-METADATA" -e "INPUT_VERIFY_METADATA" -e "INPUT_SKIP_EXISTING" -e "INPUT_VERBOSE" -e "INPUT_PRINT-HASH" -e "INPUT_PRINT_HASH" -e "HOME" -e "GITHUB_JOB" -e "GITHUB_REF" -e "GITHUB_SHA" -e "GITHUB_REPOSITORY" -e "GITHUB_REPOSITORY_OWNER" -e "GITHUB_REPOSITORY_OWNER_ID" -e "GITHUB_RUN_ID" -e "GITHUB_RUN_NUMBER" -e "GITHUB_RETENTION_DAYS" -e "GITHUB_RUN_ATTEMPT" -e "GITHUB_REPOSITORY_ID" -e "GITHUB_ACTOR_ID" -e "GITHUB_ACTOR" -e "GITHUB_TRIGGERING_ACTOR" -e "GITHUB_WORKFLOW" -e "GITHUB_HEAD_REF" -e "GITHUB_BASE_REF" -e "GITHUB_EVENT_NAME" -e "GITHUB_SERVER_URL" -e "GITHUB_API_URL" -e "GITHUB_GRAPHQL_URL" -e "GITHUB_REF_NAME" -e "GITHUB_REF_PROTECTED" -e "GITHUB_REF_TYPE" -e "GITHUB_WORKFLOW_REF" -e "GITHUB_WORKFLOW_SHA" -e "GITHUB_WORKSPACE" -e "GITHUB_ACTION" -e "GITHUB_EVENT_PATH" -e "GITHUB_ACTION_REPOSITORY" -e "GITHUB_ACTION_REF" -e "GITHUB_PATH" -e "GITHUB_ENV" -e "GITHUB_STEP_SUMMARY" -e "GITHUB_STATE" -e "GITHUB_OUTPUT" -e "RUNNER_OS" -e "RUNNER_ARCH" -e "RUNNER_NAME" -e "RUNNER_ENVIRONMENT" -e "RUNNER_TOOL_CACHE" -e "RUNNER_TEMP" -e "RUNNER_WORKSPACE" -e "ACTIONS_RUNTIME_URL" -e "ACTIONS_RUNTIME_TOKEN" -e "ACTIONS_CACHE_URL" -e "ACTIONS_ID_TOKEN_REQUEST_URL" -e "ACTIONS_ID_TOKEN_REQUEST_TOKEN" -e "ACTIONS_RESULTS_URL" -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/SophusPy/SophusPy":"/github/workspace" bac05d:ed6ea2e388f648368fab0951ff555dd1  "__token__" "" "" "" "" "true" "false" ""
Notice: Attempting to perform trusted publishing exchange to retrieve a temporary short-lived API token for authentication against https://upload.pypi.org/legacy/ due to __token__ username with no supplied password field
Traceback (most recent call last):
  File "/root/.local/bin/twine", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/root/.local/lib/python3.11/site-packages/twine/__main__.py", line 33, in main
    error = cli.dispatch(sys.argv[1:])
            ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/twine/cli.py", line 123, in dispatch
    return main(args.args)
           ^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/twine/commands/check.py", line 183, in main
    return check(parsed_args.dists, strict=parsed_args.strict)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/twine/commands/check.py", line 129, in check
    warnings, is_ok = _check_file(filename, render_warning_stream)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/twine/commands/check.py", line 73, in _check_file
    package = package_file.PackageFile.from_filename(filename, comment=None)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/twine/package.py", line 96, in from_filename
    meta = DIST_TYPES[dtype](filename)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/twine/wheel.py", line 42, in __init__
    self.extractMetadata()
  File "/root/.local/lib/python3.11/site-packages/pkginfo/distribution.py", line 123, in extractMetadata
    data = self.read()
           ^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/twine/wheel.py", line 64, in read
    archive = zipfile.ZipFile(fqn)
              ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/zipfile.py", line 1304, in __init__
    self._RealGetContents()
  File "/usr/local/lib/python3.11/zipfile.py", line 1401, in _RealGetContents
    raise BadZipFile("Bad magic number for central directory")
zipfile.BadZipFile: Bad magic number for central directory
Checking dist/sophuspy-1.0.0-cp310-cp310-macosx_10_9_x86_64.whl:

Github Actions:

name: Python application

# on:
#   push:
#     tags:
#       - 'v[0-9]+.[0-9]+.[0-9]+'

on: [push]

permissions:
  contents: read

jobs:
  # Build wheels on multi platforms
  build_wheels:
    name: Build wheels on ${{ matrix.os }}
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-latest, ubuntu-20.04, windows-latest, windows-2019, macos-latest, macos-11, macos-13]

    steps:
      - uses: actions/checkout@v4

      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v3
        with:  
          python-version: ${{ matrix.python-version }}

      - name: Install cibuildwheel
        run: python -m pip install cibuildwheel

      - name: Build wheels
        run: python -m cibuildwheel --output-dir wheelhouse

      - uses: actions/upload-artifact@v4
        with:
          name: cibw-wheels-${{ matrix.os }}-${{ strategy.job-index }}
          path: ./wheelhouse/*.whl

  build_sdist:
    name: Build source distribution
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Build sdist
        run: pipx run build --sdist

      - uses: actions/upload-artifact@v4
        with:
          name: cibw-sdist
          path: dist/*.tar.gz

  pypi_publish:
    needs: [build_wheels, build_sdist]
    runs-on: ubuntu-latest
    environment:
      name: pypi
      url: https://pypi.org/p/sophuspy
    permissions:
      id-token: write
    steps:
      - uses: actions/download-artifact@v4
        with:
          # unpacks all CIBW artifacts into dist/
          pattern: cibw-*
          path: dist
          merge-multiple: true

      - uses: pypa/gh-action-pypi-publish@release/v1
        with:
          skip-existing: true

And I tried - uses: pypa/gh-action-pypi-publish@release/v1.8, same problem.

Interesting thing is that:
if I trigger the os system separately, like below:
os: [ubuntu-latest, ubuntu-20.04]
os: [windows-latest, windows-2019]
os: [macos-latest, macos-11, macos-13]
these three times passed separately.

What happened? Please help.

The text was updated successfully, but these errors were encountered:

webknjaz · 2024-02-22T17:35:13Z

It's something in twine that we call to make this upload. Download the dist artifact and inspect it locally. Possibly, file an issue with twine.
Also, add twine check --strict in your pipelines as early as possible.

webknjaz · 2024-02-22T18:13:04Z

@henryiii is it possible that cibw produces broken wheels?

henryiii · 2024-02-22T18:23:12Z

I think there might be a bug with actions/download-artifact@v4, I've seen corrupted files from it in nox, too, in wntrblm/nox#772. I think it might be happening there because some of the files have the same file names when you merge.

Combined data file .coverage.3.10.tox.latest
Combined data file .coverage.3.11.tox.latest
Combined data file .coverage.3.8.tox.latest
Combined data file .coverage.3.9.tox.4
Traceback (most recent call last):
  File "/home/runner/work/nox/nox/.nox/cover/bin/coverage", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/runner/work/nox/nox/.nox/cover/lib/python3.11/site-packages/coverage/cmdline.py", line 970, in main
    status = CoverageScript().command_line(argv)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/nox/nox/.nox/cover/lib/python3.11/site-packages/coverage/cmdline.py", line 687, in command_line
    self.coverage.combine(data_paths, strict=True, keep=bool(options.keep))
  File "/home/runner/work/nox/nox/.nox/cover/lib/python3.11/site-packages/coverage/control.py", line 836, in combine
    combine_parallel_data(
  File "/home/runner/work/nox/nox/.nox/cover/lib/python3.11/site-packages/coverage/data.py", line 176, in combine_parallel_data
    data.update(new_data, aliases=aliases)
  File "/home/runner/work/nox/nox/.nox/cover/lib/python3.11/site-packages/coverage/sqldata.py", line 690, in update
    arcs = [
           ^
  File "/home/runner/work/nox/nox/.nox/cover/lib/python3.11/site-packages/coverage/sqldata.py", line 690, in <listcomp>
    arcs = [
           ^
sqlite3.DatabaseError: database disk image is malformed

There are a lot of repeated wheels with same file names if you have ubuntu-latest, ubuntu-20.04, windows-latest, windows-2019, macos-latest, macos-11, macos-13.

henryiii · 2024-02-22T18:26:25Z

As a first step, I'd reduce that list of platforms to ubuntu-latest, windows-latest, macos-latest. The only one that produces different wheels is macOS-14 (because it's ARM), which is not in the list above.

craigstar · 2024-02-23T03:12:28Z

As @henryiii recommended, I reduced down to os: [ubuntu-latest, windows-latest, macos-latest], and it passed. Thanks a lot.

FYI
Today I triggered
os: [ubuntu-latest, ubuntu-20.04]
os: [windows-latest, windows-2019]
os: [macos-latest, macos-11, macos-13]
separately again, two passed but os: [windows-latest, windows-2019] failed. Still the same reason:
raise BadZipFile("Bad magic number for central directory")

so I break os: [windows-latest, windows-2019] down to:
os: [windows-latest]
os: [windows-2019]
both of them succeeded.

henryiii · 2024-02-23T03:41:03Z

Don’t build with multiple image variations and see what happens. I’m pretty sure merge-multiple corrupts data if there are matching filenames. The version of the image you use doesn’t affect the wheels you produce.

craigstar · 2024-02-23T03:55:48Z

@henryiii os: [ubuntu-latest, windows-latest, macos-latest] works, Thank you! ( I didn't know these three is enough for all necessary wheels)

…ish#215

Borda · 2025-01-08T08:27:26Z

Don’t build with multiple image variations and see what happens. I’m pretty sure merge-multiple corrupts data if there are matching filenames. The version of the image you use doesn’t affect the wheels you produce.

I have a similar workflow and it seems to be randomly crashing with the very same error trace, when I download all particular artifacts locally and validate it all is fine:

Checking dist-packages-extras-b24688da5f969ab36922523e3174f55fe4eb6de7-build-0/lightning_utilities-0.12.0.dev0-py3-none-any.whl: PASSED
Checking dist-packages-extras-b24688da5f969ab36922523e3174f55fe4eb6de7-build-0/lightning_utilities-0.12.0.dev0.tar.gz: PASSED
jirka@Jiris-Laptop Downloads % twine check dist-packages-extras-b24688da5f969ab36922523e3174f55fe4eb6de7-build-1/*
Checking dist-packages-extras-b24688da5f969ab36922523e3174f55fe4eb6de7-build-1/lightning_utilities-0.12.0.dev0-py3-none-any.whl: PASSED
Checking dist-packages-extras-b24688da5f969ab36922523e3174f55fe4eb6de7-build-1/lightning_utilities-0.12.0.dev0.tar.gz: PASSED
jirka@Jiris-Laptop Downloads % twine check dist-packages-extras-b24688da5f969ab36922523e3174f55fe4eb6de7-build-2/*
Checking dist-packages-extras-b24688da5f969ab36922523e3174f55fe4eb6de7-build-2/lightning_utilities-0.12.0.dev0-py3-none-any.whl: PASSED
Checking dist-packages-extras-b24688da5f969ab36922523e3174f55fe4eb6de7-build-2/lightning_utilities-0.12.0.dev0.tar.gz: PASSED

and this is log from the download:

Filtering artifacts by pattern 'dist-packages-extras-b24688da5f969ab36922523e3174f55fe4eb6de7-build-*'
Preparing to download the following artifacts:
- dist-packages-extras-b24688da5f969ab36922523e3174f55fe4eb6de7-build-2 (ID: 2399978063, Size: 56161)
- dist-packages-extras-b24688da5f969ab36922523e3174f55fe4eb6de7-build-1 (ID: 2399977534, Size: 55766)
- dist-packages-extras-b24688da5f969ab36922523e3174f55fe4eb6de7-build-0 (ID: 2399976283, Size: 55698)
Redirecting to blob download url: https://productionresultssa15.blob.core.windows.net/actions-results/0af4dc60-3557-4087-ab2e-79ee9e024f56/workflow-job-run-8077a371-f2e9-560d-2640-26370e94f7f1/artifacts/037b204042f8879cdaa4cac098a018249f177740b4bc9a7[10](https://github.com/Lightning-AI/utilities/actions/runs/12666342800/job/35298281303?pr=347#step:2:11)14454c849017f77.zip
Starting download of artifact to: /home/runner/work/utilities/utilities/dist
Redirecting to blob download url: https://productionresultssa15.blob.core.windows.net/actions-results/0af4dc60-3557-4087-ab2e-79ee9e024f56/workflow-job-run-a4b50df8-5ee3-5593-91c7-833d58b9d8d4/artifacts/f14459896e679f03ba586f[11](https://github.com/Lightning-AI/utilities/actions/runs/12666342800/job/35298281303?pr=347#step:2:12)353425fe68c015cd0f6b5da948fd9c74ce7311c4.zip
Starting download of artifact to: /home/runner/work/utilities/utilities/dist
Redirecting to blob download url: https://productionresultssa15.blob.core.windows.net/actions-results/0af4dc60-3557-4087-ab2e-79ee9e024f56/workflow-job-run-f47de9fc-70e2-55fe-cdbd-5a25232bd95d/artifacts/2f82bc3340862884af86096d99e03a7e0d26f06e3431b83528ae176de238e727.zip
Starting download of artifact to: /home/runner/work/utilities/utilities/dist
(node:[17](https://github.com/Lightning-AI/utilities/actions/runs/12666342800/job/35298281303?pr=347#step:2:18)51) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
Artifact download completed successfully.
Artifact download completed successfully.
Artifact download completed successfully.
Total of 3 artifact(s) downloaded
Download artifact has finished successfully

also rerun the failed job crashes with same issue...

webknjaz · 2025-01-08T12:52:22Z

@Borda its recommended to also validate the metadata earlier in your workflows. In jobs that run in PRs, not just when you publish. This way, you won't be getting last-minute problems. Run twine check --strict for this.

As for why local check might be different, not sure. Maybe the versions of twine and CPython are different 🤷‍♂️.

FWIW, now that we switched to pre-cached containers, you should be able to grab the exact one running in the CI and attempt reproducing there.

henryiii mentioned this issue Feb 23, 2024

[bug] Matching names and merge-multiple causes data corruption actions/download-artifact#298

Open

webknjaz mentioned this issue Apr 11, 2024

twine check in action, but passes when dockerfile built locally #227

Closed

yhs0602 added a commit to yhs0602/CraftGround that referenced this issue Dec 27, 2024

💚 Fix ci for arm macos universal corrupt zip pypa/gh-action-pypi-publ…

7e0f112

…ish#215

mmacpherson added a commit to mmacpherson/tenforty that referenced this issue Jan 4, 2025

reduce # runners per pypa/gh-action-pypi-publish#215

2d3c985

mmacpherson added a commit to mmacpherson/tenforty that referenced this issue Jan 4, 2025

cut down per pypa/gh-action-pypi-publish#215

510f81e

mmacpherson added a commit to mmacpherson/tenforty that referenced this issue Jan 4, 2025

cut down per pypa/gh-action-pypi-publish#215 (#54)

6cf48dc

Borda mentioned this issue Jan 8, 2025

ci: debug pkg create on multi-OS Lightning-AI/utilities#347

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

raise BadZipFile("Bad magic number for central directory") #215

raise BadZipFile("Bad magic number for central directory") #215

craigstar commented Feb 22, 2024 •

edited

Loading

webknjaz commented Feb 22, 2024

webknjaz commented Feb 22, 2024

henryiii commented Feb 22, 2024 •

edited

Loading

henryiii commented Feb 22, 2024

craigstar commented Feb 23, 2024 •

edited

Loading

henryiii commented Feb 23, 2024

craigstar commented Feb 23, 2024 •

edited

Loading

Borda commented Jan 8, 2025 •

edited

Loading

webknjaz commented Jan 8, 2025

raise BadZipFile("Bad magic number for central directory") #215

raise BadZipFile("Bad magic number for central directory") #215

Comments

craigstar commented Feb 22, 2024 • edited Loading

webknjaz commented Feb 22, 2024

webknjaz commented Feb 22, 2024

henryiii commented Feb 22, 2024 • edited Loading

henryiii commented Feb 22, 2024

craigstar commented Feb 23, 2024 • edited Loading

henryiii commented Feb 23, 2024

craigstar commented Feb 23, 2024 • edited Loading

Borda commented Jan 8, 2025 • edited Loading

webknjaz commented Jan 8, 2025

craigstar commented Feb 22, 2024 •

edited

Loading

henryiii commented Feb 22, 2024 •

edited

Loading

craigstar commented Feb 23, 2024 •

edited

Loading

craigstar commented Feb 23, 2024 •

edited

Loading

Borda commented Jan 8, 2025 •

edited

Loading