Cloudflare Workers Support - Local file header not found

[baer]

First, thanks for all the work you've put into making this project complete. It's pretty cool to see this all come together on top of Web standards!

Background

I want to work with zip files in a Serverless environment. In that context, there are two options: container-based (e.g., Lambda) and Isolate-based (Cloudflare Workers, Deno, etc.). The latter group is just Chrome tabs in the cloud. Isolate-based platforms, like Zip.js, use Web standards instead of Node APIs.

Since Zip.js is based on Promise, TypedArray, and Streams, it should work out of the box for Isolate-based runtimes.

The issue

Running the Hello World! example from the README in Cloudflare Workers produces the error below.

• The line that produces the error (48) is:await firstEntry.getData(helloWorldStream.writable)
• [mf:err] is from Miniflare, Cloudflare's local runtime environment.

[mf:err] Error: Local file header not found
    at ZipEntry.getData (/private/.../node_modules/@zip.js/zip.js/lib/core/zip-reader.js:380:10)
    at async onRequest8 (/private/.../functions/api/v1/zip-test.ts:48:3)
    at async next (/private/.../node_modules/wrangler/templates/pages-template-worker.ts:152:22)
    at async Object.fetch (/private/.../node_modules/wrangler/templates/pages-template-worker.ts:171:11)
    at async jsonError (/private/.../node_modules/wrangler/templates/middleware/middleware-miniflare3-json-error.ts:22:10)
    at async jsonError2 (/.../node_modules/wrangler/templates/middleware/middleware-miniflare3-json-error.ts:22:10)

The Repro

This is just the Hello, World! example from the README pasted into a Worker, but I created a minimal repro project so you could run it easily, in case that's helpful.

https://github.com/baer/zip-cloudflare-repro

Things I've Looked at to Debug

Inspect the Blob as text

await new Response(zipFileBlob).text()

���V	-hello.txtUT�~�d
 �t�����t�����t�����H���W(�/�IQ��
                                ���V��
                                      	-hello.txtUT�~�d
 �t�����t�����t����PKd

Download the blob as a Zip to see if macOS will have any info
Here is the file.zip it creates, and the resulting Error message

Unable to expand "file.zip" into "Downloads"
(Error -1 - No such process.)

Inspect the Blob as hex

  const arrayBuffer = await new Response(zipFileBlob).arrayBuffer();
  const uint8Array = new Uint8Array(arrayBuffer);

  let hexDump = "";
  for (let byte of uint8Array) {
    hexDump += byte.toString(16).padStart(2, "0") + " ";
  }

50 4b 03 04 2d 00 08 08 08 00 87 46 ea 56 00 00 00 00 00 00 00 00 00 00 00 00 09 00 2d 00 68 65 6c 6c 6f 2e 74 78 74 55 54 05 00 01 2e 29 ac 64 0a 00 20 00 00 00 00 00 01 00 18 00 30 62 c9 7e 46 b3 d9 01 30 62 c9 7e 46 b3 d9 01 30 62 c9 7e 46 b3 d9 01 f3 48 cd c9 c9 57 28 cf 2f ca 49 51 04 00 95 19 85 1b 0e 00 00 00 00 00 00 00 0c 00 00 00 00 00 00 00 50 4b 01 02 14 00 2d 00 08 08 08 00 87 46 ea 56 95 19 85 1b ff ff ff ff 0c 00 00 00 09 00 39 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 68 65 6c 6c 6f 2e 74 78 74 01 00 08 00 0e 00 00 00 00 00 00 00 55 54 05 00 01 2e 29 ac 64 0a 00 20 00 00 00 00 00 01 00 18 00 30 62 c9 7e 46 b3 d9 01 30 62 c9 7e 46 b3 d9 01 30 62 c9 7e 46 b3 d9 01 50 4b 05 06 00 00 00 00 01 00 01 00 70 00 00 00 0e 00 00 00 00 00

Additional Info

• Cloudflare Workers supports "All of the standard built-in objects supported by the current Google Chrome stable release are supported" except eval(), new Function, and Date.now()
• Miniflare (Cloudflare's local dev runtime) only supports a subset of full V8. It's possible that this is a Miniflare issue, not a Workers issue, but I didn't try deploying the project to see.

---

Even if the fix ends up being beyond this project's scope, I thought I'd leave a few breadcrumbs in case anybody else decides to try something similar.

Thanks again for the work you do

[gildas-lormeau]

I analyzed your zip file manually and I confirm it is invalid. The bug is due to an invalid value for the "offset of start of central directory with respect to the starting disk number" record in the "end of central directory" of the zip file, see section 4.3.16 here: https://pkware.cachefly.net/webdocs/APPNOTE/APPNOTE-6.3.9.TXT. This is the offset of 50 4B 01 02 in the zip file. This value can be found at the offset 0xF8 (248) in the zip file and must be set to 0x76(118) in a similar valid zip file. Instead, it is set to 0x0E (14) in your zip file.

I've just spent some time trying to reproduce the bug in Deno, but I can't do it. I've also tried to track the assignment of this value in zip.js and so far I have no idea why it is set to 0x0E instead of 0x76. You can use https://hexed.it/ to edit the incorrect byte at the offset 0xF8 (248) if you want to do some tests.

Maybe setting the option useWebWorkers to false or useCompressionStream to false might help to circumvent this bug. That would be useful for me if you could do these tests. To do so, you have to call configure() exported from zip.js and pass as parameter an object with the options. It would also be useful if you can confirm this bug can be reproduced without Miniflare, in Workers. Finally, you can also update zip.js. I fixed some minor issues which should not be related to this bug but who knows?

[baer]

Amazing, thanks for looking. I tried both of those options, and in both cases, I got the Local file header not found error, which I presume to be the same thing.

You can repro the issue by running the little repo I created.

git clone https://github.com/baer/zip-cloudflare-repro.git
cd zip-cloudflare-repro
npm ci
npm start

This will fire up Cloudflare at http://127.0.0.1:8787/, and navigating to that URL will run the code.

As a control, I also created a file called node-test which pulls in the zip.js code without the Cloudflare handlers. In Node.js 20.2.0, everything is peachy. It seems to be specific to Cloudflare's runtime.

node node-test.mjs
# Hello world!

Is there anything else I can do to help track this down?

[gildas-lormeau]

Thank you for the additional info. I was able to identify some strange behaviors with typed arrays which are being emptied unexpectedly. This issue should be circumvented in the version 2.7.20 I've just published.