upload-artifact double-zips a zip

[kcgen]

When upload-artifact is provided an existing zip file via its path: argument:

        with:
          name: asset-package
          path: asset-package.zip

It double-zips the zip, resulting a zip within a zip.

"You're using it wrong, don't provide a zip file!", might be a counter-argument, however zips might be preferred because:

• Some off-the-shelf workflows automatically produce a zipped result, therefore it would make sense to take it as-is.
• Re-zipping needlessly burns CPU, always results in a larger file, wastes user time, confuses users, and adds complexity to down-stream automation.
• Providing a zip can be faster than uploading a directory of files because it avoids the per-file latency multiplier that (possibly thousands of) individual transfers incur.

"We can't trust your zipfile; what if it's some other file in disguise!", could be another argument:

• Ok.. but the asset zipper already trusts any file anyway; so security already isn't a concern. People could already be feeding the assert zipper viruses and trojans for all we know.
• If this is a concern, a content/type check could be applied to confirm the file is a bonafide zip. If you still don't trust it, then the zip could be decompressed to /dev/null and confirmed good, as opposed to double-zipping the zip.

Yes, a work-around would be to unzip our resulting zip to a temporary directory, and then provide that to the asset uploader - however we're now wasting three rounds of zipping: initial zip, unzip, re-zip.

GitHub already gives developers full control over the content that they're uploading, so using a zipfile straight-away is just another means to provide the same content without a second layer of packaging.

[konradpabjan]

This is an issue on the download-artifact side with how the server downloads/streams the artifacts.

This will be fixed with the v2 versions of the actions that are currently in-development. The behavior right now for this is really annoying 😕

[kcgen]

@konradpabjan , thanks!

This is an issue on the download-artifact side with how the server downloads/streams the artifacts.

Very interesting knowing that the assets aren't being zipped by the upload action (potentially wasting space?), and instead the zipping is done by the downloader service.

Well, I'm not picky about what happens behind the curtain, but happy you're on this and that we won't have zipped-zips come v2!

[filips123]

@konradpabjan When will v2 be released?

[eine]

Ref #3 (comment)

[konradpabjan]

We're planning on having a v2-beta sometime around mid-January (possibly a little earlier than that but I don't want to over-promise 😊)

[mk-pmb]

I'd really appreciate if we could get uploading of verbatim single files going asap, so I can avoid setting up an interim FTP upload.

[pjoe]

Ran into this as well. Using 7zip as part of the build is order of magnitude faster than having upload-artifact handle the zipping.

For our specific repo timings are (3719 files, 150MB uncompressed, 44MB compressed):

Operation	Time
Use 7zip to compress	12.3s
Upload (and zip) .zip	15s
Use upload-artifact to zip and upload	2m 36s

[kcgen]

We're planning on having a v2-beta sometime around mid-January

Thanks @konradpabjan -- just checking in on how this is going; any news?

[konradpabjan]

The v2-preview is out now! You can see the pinned issue for more information.

Initially I thought we would get this done for v2 but unfortunately this is not the case 😞

Basically, this double-zip isn't caused by how the artifact get uploaded, but rather it's a limitation of the GitHub UI in terms of how you can later download the artifact. The long term fix is to offer a more enhanced experience in terms of how to download artifacts (browse and download individual components from the UI). For v2 we were considering some back-end changes/hack to prevent extra zipping if a zip file is already detected, but we steered away from that.

See my comment here for more information: #3 (comment) (same things applies to an individual file that gets uploaded, a zip is the only supported way that you can currently download an artifact from the UI)

There is GZip compression that is used during upload (which I initially though was responsible for this, apologies for some of my conflicting comments earlier).

[schultzern]

+1

[konradpabjan]

@afwn90cj93201nixr2e1re The artifact double zip is currently a UI limitation and it's independent of how the artifact gets uploaded. There is no zip that gets created during artifact upload.

We don't have anything yet to allow users to individually download files from the UI so a zip gets dynamically created when you click to download the artifact (this is done so that all files are downloaded at once). Long term we have plans to enhance this significantly so you individually download files (and browse them) without any extra zipping. At the moment this is an unfortunate limitation that we have 😕

[carrotIndustries]

Being able to download files without the additional ZIP wrapper would be highly appreciated as I use actions to build ready-to-extract zip files and having to extract that zip from another zip just confusion among users.

[jimis]

As an update, the issue still exists with v2. I use upload-artifact@v2 to upload a single .tar.gz file of about 300MB size. When trying to download it on the UI, it's only possible to download a ZIP archive of the already compressed tarball.

There is also about 30s lag between clicking and actually starting the download; I assume zipping happens on demand when trying to download it, as @konradpabjan mentioned.

[Xottab-DUTY]

I remember the times when this issue was fixed and we could upload our own archives. That were short and great times! (just few days, IIRC)
GitHub Team stop ignoring this issue, it is important!

People even create duplicates/related issues! (like #426)

[HunterZ]

This is the 4th-highest voted issue on this project, and relates to the highest-voted issue #14.

I just went to the trouble of setting up CPack for my C++ project to create a zip file, only to discover that it was a waste of time unless I want to live with a doubly-zipped artifact.

It's also bizarre that it only supports .zip and not .tar.gz. I think I saw another action that attempts that, however.

[paulocoutinhox]

+1 pls

[justinvforvendetta]

ok, glad im not losing my mind. yeah, all my artifacts are .zip.zip upon download.

Artifact verge-windows.zip.zip successfully finalized. Artifact ID 1263043011
Artifact verge-windows.zip has been successfully uploaded! Final size is 227342719 bytes. Artifact ID is 1263043011
Artifact download URL: https://github.com/vergecurrency/verge/actions/runs/7989724701/artifacts/1263043011

- uses: actions/upload-artifact@v4
        with:
          name: verge-windows.zip
          path: |
            /home/runner/work/verge/verge/src/verged.exe
            /home/runner/work/verge/verge/src/verge-cli.exe
            /home/runner/work/verge/verge/src/verge-tx.exe
            /home/runner/work/verge/verge/src/qt/verge-qt.exe

[tigrou23]

Please... This is a big issue for everyone

[justinvforvendetta]

you can check out my workflow here: https://github.com/vergecurrency/verge/blob/master/.github/workflows/build-all-and-release.yml

[make-github-pseudonymous-again]

See also #485.

[bsabiston]

It does the same thing with .7z files - please fix this!

[turkjanez]

still not fixed???

[romandolhai]

The strange thing is that official Docker manual suggests passing data through artifacts so that images can be loaded with .tar extension (e.g. docker load --input /tmp/myimage.tar)

I couldn't make it work. It appears that GitHub actions compress my .tar file into .zip!