Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Zip build on windows are not POSIX compliant #512

Open
riderx opened this issue Jun 23, 2024 · 4 comments
Open

Zip build on windows are not POSIX compliant #512

riderx opened this issue Jun 23, 2024 · 4 comments

Comments

@riderx
Copy link

riderx commented Jun 23, 2024

I had an issue with my CLI when used on Windows the ZIP was not readable on certain picky env like JAVA.

So I created a test in GitHub action for that and found the issue

You can see the result here:
https://github.com/Cap-go/CLI/actions/runs/9634029990/job/26569222957
I created a zip with the lib in windows and tried with java and swift to open it, and it fails in Java.

My current fix is to use jszip for windows env, but it's not maintained, I would be glad if we managed to fix it here and I keep only using adm-zip.

the java test i run

import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.BufferedInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;

public class VerifyZip {
    public static void main(String[] args) {
        if (args.length < 1) {
            System.out.println("Usage: java VerifyZip <zip-file>");
            System.exit(1);
        }

        String zipFilePath = args[0];
        File zipFile = new File(zipFilePath);
        File targetDirectory = new File("extracted");

        if (!zipFile.exists()) {
            System.out.println("File not found: " + zipFilePath);
            System.exit(1);
        }

        try (
            BufferedInputStream bis = new BufferedInputStream(new FileInputStream(zipFile));
            ZipInputStream zis = new ZipInputStream(bis)
        ) {
            int count;
            int bufferSize = 8192;
            byte[] buffer = new byte[bufferSize];
            long lengthTotal = zipFile.length();
            long lengthRead = bufferSize;
            int percent = 0;

            ZipEntry entry;
            while ((entry = zis.getNextEntry()) != null) {
                if (entry.getName().contains("\\")) {
                    System.out.println("Windows path is not supported: " + entry.getName());
                    System.exit(1);
                }
                File file = new File(targetDirectory, entry.getName());
                String canonicalPath = file.getCanonicalPath();
                String canonicalDir = targetDirectory.getCanonicalPath();
                File dir = entry.isDirectory() ? file : file.getParentFile();

                if (!canonicalPath.startsWith(canonicalDir)) {
                    System.out.println("SecurityException, Failed to ensure directory is the start path: " +
                            canonicalDir + " of " + canonicalPath);
                    System.exit(1);
                }

                if (!dir.isDirectory() && !dir.mkdirs()) {
                    System.out.println("Failed to ensure directory: " + dir.getAbsolutePath());
                    System.exit(1);
                }

                if (entry.isDirectory()) {
                    continue;
                }

                try (FileOutputStream outputStream = new FileOutputStream(file)) {
                    while ((count = zis.read(buffer)) != -1) {
                        outputStream.write(buffer, 0, count);
                    }
                }

                int newPercent = (int) ((lengthRead / (float) lengthTotal) * 100);
                if (lengthTotal > 1 && newPercent != percent) {
                    percent = newPercent;
                }

                lengthRead += entry.getCompressedSize();
            }
            System.out.println("ZIP file is valid: " + zipFilePath);
        } catch (IOException e) {
            System.out.println("Failed to process ZIP file: " + zipFilePath);
            e.printStackTrace();
            System.exit(1);
        }
    }
}

The github action who could be adapted

name: Check POSIX Paths in Zip File

on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main

jobs:
  create-valid-zip-linux:
    if: ${{ !startsWith(github.event.head_commit.message, 'chore(release):') }}
    runs-on: ubuntu-latest

    steps:
    - name: Checkout repository
      uses: actions/checkout@v4

    - name: Setup bun
      uses: oven-sh/setup-bun@v1.2.2
      with:
        bun-version: latest

    - name: Install dependencies
      id: install_code
      run: bun install --frozen-lockfile

    - name: Build code
      id: build_code
      run: bun run build

    - name: Create a valid zip test
      id: create_zip
      run: node ./dist/index.js bundle zip --path test/test_upload -n build-linux.zip

    - name: Check build directory contents
      run: |
        echo "Listing contents of the build directory..."
        ls -R ./dist

    - name: Check ZIP file contents
      run: |
        echo "Listing contents of the ZIP file..."
        unzip -l build-linux.zip

    - name: Upload build-linux.zip artifact
      uses: actions/upload-artifact@v4
      with:
        name: build-zip-linux
        path: build-linux.zip

  check-posix-paths-windows:
    runs-on: ${{ matrix.os }}
    if: ${{ !startsWith(github.event.head_commit.message, 'chore(release):') }}

    strategy:
      matrix:
        os: [windows-2019, windows-2022]

    steps:
    - name: Checkout repository
      uses: actions/checkout@v4

    - name: Setup bun
      uses: oven-sh/setup-bun@v1.2.2
      with:
        bun-version: latest

    - name: Install dependencies
      id: install_code
      run: bun install --frozen-lockfile

    - name: Build code
      id: build_code
      run: bun run build

    - name: Create a zip test
      id: create_zip
      run: node ./dist/index.js bundle zip --path test/test_upload -n build-${{ matrix.os }}.zip

    - name: Upload build.zip artifact
      uses: actions/upload-artifact@v4
      with:
        name: build-zip-${{ matrix.os }}
        path: build-${{ matrix.os }}.zip

  check-posix-paths-unix:
    runs-on: ubuntu-latest
    needs: [create-valid-zip-linux, check-posix-paths-windows]

    steps:
    - name: Checkout repository
      uses: actions/checkout@v4

    - name: Download build-linux.zip artifact
      uses: actions/download-artifact@v4
      with:
        name: build-zip-linux

    - name: List the files
      run: ls -lh

    - name: Check file size of Linux build
      run: ls -lh build-linux.zip

    - name: Verify ZIP file integrity for Linux build with zipinfo
      run: |
        echo "Verifying ZIP file integrity for Linux build with zipinfo..."
        zipinfo ./build-linux.zip || (echo "ZIP file is corrupted: build-linux.zip" && exit 1)

    - name: Verify POSIX paths for Linux build
      run: |
        unzip build-linux.zip -d extracted-linux
        if find extracted-linux -type f | grep -qE '\\\\'; then
          echo "Non-POSIX paths detected in build-linux.zip."
          exit 1
        else
          echo "All paths are POSIX compliant in build-linux.zip."
        fi

    - name: Setup Java
      uses: actions/setup-java@v4
      with:
        distribution: 'zulu' 
        java-version: '17'

    - name: Compile VerifyZip.java
      run: javac ./test/VerifyZip.java

    - name: Verify ZIP file integrity for Linux build with Java
      run: java -cp ./test VerifyZip build-linux.zip

    - name: Download build-windows-2019.zip artifact
      uses: actions/download-artifact@v4
      with:
        name: build-zip-windows-2019

    - name: Download build-windows-2022.zip artifact
      uses: actions/download-artifact@v4
      with:
        name: build-zip-windows-2022

    - name: List the files
      run: ls -lh

    - name: Check file sizes of Windows builds
      run: |
        echo "Checking file sizes..."
        ls -lh build-windows-2019.zip
        ls -lh build-windows-2022.zip

    - name: Verify ZIP file integrity for Windows 2019 build with zipinfo
      run: |
        echo "Verifying ZIP file integrity for Windows 2019 build with zipinfo..."
        zipinfo ./build-windows-2019.zip || (echo "ZIP file is corrupted: build-windows-2019.zip" && exit 1)

    - name: Verify ZIP file integrity for Windows 2022 build with zipinfo
      run: |
        echo "Verifying ZIP file integrity for Windows 2022 build with zipinfo..."
        zipinfo ./build-windows-2022.zip || (echo "ZIP file is corrupted: build-windows-2022.zip" && exit 1)

    - name: Verify POSIX paths for Windows 2019 build
      run: |
        unzip build-windows-2019.zip -d extracted-2019
        if find extracted-2019 -type f | grep -qE '\\\\'; then
          echo "Non-POSIX paths detected in build-windows-2019.zip."
          exit 1
        else
          echo "All paths are POSIX compliant in build-windows-2019.zip."
        fi
    
    - name: Verify POSIX paths for Windows 2022 build
      run: |
        unzip build-windows-2022.zip -d extracted-2022
        if find extracted-2022 -type f | grep -qE '\\\\'; then
          echo "Non-POSIX paths detected in build-windows-2022.zip."
          exit 1
        else
          echo "All paths are POSIX compliant in build-windows-2022.zip."
        fi

    - name: Verify ZIP file integrity for Windows 2019 build with Java
      run: java -cp ./test VerifyZip build-windows-2019.zip

    - name: Verify ZIP file integrity for Windows 2022 build with Java
      run: java -cp ./test VerifyZip build-windows-2022.zip

  check-posix-paths-macos:
    runs-on: macos-latest
    needs: [create-valid-zip-linux, check-posix-paths-windows]

    steps:
    - name: Checkout repository
      uses: actions/checkout@v4

    - name: Download build-linux.zip artifact
      uses: actions/download-artifact@v4
      with:
        name: build-zip-linux
   
    - name: Download build-windows-2019.zip artifact
      uses: actions/download-artifact@v4
      with:
        name: build-zip-windows-2019

    - name: Download build-windows-2022.zip artifact
      uses: actions/download-artifact@v4
      with:
        name: build-zip-windows-2022

    - name: List the files
      run: ls -lh

    - name: Check file size of Linux build
      run: ls -lh build-linux.zip

    - name: Setup Swift
      uses: swift-actions/setup-swift@v2

    - name: Get swift version
      run: swift --version # Swift 5.10

    - name: Compile test executable
      run: swift build -c release
      working-directory: ./test/test_zip_swift/

    - name: Run the swift test
      run: ./test/test_zip_swift/.build/release/MyCLI --zip-files build-linux.zip build-windows-2019.zip build-windows-2022.zip
      ```
@riderx riderx changed the title Zip build on windows are build with wrong non unix path Zip build on windows are not POSIX compliant Jun 23, 2024
@riderx
Copy link
Author

riderx commented Jun 23, 2024

Could be linked to #497
Maybe it could be nice if we could force the behavior like in JSZIP:
CleanShot 2024-06-23 at 15 27 11@2x

@5saviahv
Copy link
Collaborator

Hmm, can you take code directly from github repo.

I believe #497 was fixed in repo, but this fix is not released in npm

@JaymzZh
Copy link

JaymzZh commented Jul 2, 2024

May I ask when will it be released? @5saviahv

@5saviahv
Copy link
Collaborator

5saviahv commented Aug 1, 2024

May I ask when will it be released? @5saviahv

Sadly, I have no good aswer for that. Currently only @cthackers can make releases in npm.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants