feat!: add Windows support with standalone binary (#474)

This change provides Windows support with a standalone binary included as a new release artifact. This is the implementation of the research spike in #472, which settled on `nuitka` for generating the binary.

The binary is created with [Nuitka](https://nuitka.net) as a single file and included as an additional release artifact. The file is a self-extracting executable that expands to a version-specific user cache directory and runs from there, saving startup time for subsequent runs. Unlike other install options, this one does not require Python on the system to run.

Like other supported platforms, if an existing supported instance of the Phylum CLI is not found on the system, the latest Windows binary CLI is downloaded and placed in the same self-extracted cache directory used by the phylum-ci binary. This allows for reduced prerequisites and better testing.

Additional changes made include:

* Add a new Poetry dependency group named `compile`
* Break up `Release` workflow into separate jobs
  * A build job to create the Python distributions
  * A build job to create the Windows binary
  * A release job to create/publish the GitHub release and Docker images
* Update the `Preview` workflow to add option to create Windows binary
* Add a Phylum `favicon.ico` to allow the binary to have an icon
* Ensure internal calls to `phylum-init` don't reset log level
* Ensure all `subprocess.run` commands are cross-platform compatible
  * Specify `encoding` and `errors` where it matters
* Add ability to bypass full CI detection
  * Set `PHYLUM_BYPASS_CI_DETECTION` envvar to get `CINone` instead
  * Helpful for testing while still providing basic functionality
* Allow `phylum-init` to use a release candidate when `latest` specified
  * This only happens when `latest` is not a supported version
* Add method for ensuring the Phylum settings file is always populated
* Account for Windows device drive prefix (e.g., `\\?\`) in paths
* Ensure internal `ci` extension has cross-platform read permissions set
* Add new installation method to README
* Update tests
* Refactor and format throughout
  * Use envvar instead of matrix to define Python version in workflows
  * Use GitHub step and job outputs to pass information in workflows
  * Reduce verbosity on `python-semantic-release` calls in workflows
  * Update log messages
  * Add missing type hints to `src/phylum/init/cli.py`
  * Remove `rich` markup emojis from log entries

BREAKING CHANGE: Phylum CLI installs before v7.1.0-rc1 are no longer supported. That release is the first one providing full Windows support.

.github/workflows/auto_updates.yml

@@ -14,10 +14,9 @@ jobs:
   workflow-auto-updates:
     name: Update dependencies and hooks
     runs-on: ubuntu-latest
-    strategy:
-      matrix:
-        # It's only one Python version specified in a "matrix", but on purpose to stay DRY
-        python-version: ["3.12"]
+    env:
+      PYTHON_VERSION: "3.12"
+      POETRY_VERSION: "1.8.3"
     defaults:
       run:
         shell: bash
@@ -35,20 +34,20 @@ jobs:
           git_commit_gpgsign: true
 
       - name: Install poetry
-        run: pipx install poetry==1.8.3
+        run: pipx install poetry==${{ env.POETRY_VERSION }}
 
       - name: Configure poetry
         run: poetry config virtualenvs.in-project true
 
       - name: Set up Python
         uses: actions/setup-python@f677139bbe7f9c59b41e40162b753c062f5d49a3 # v5.2.0
         with:
-          python-version: ${{ matrix.python-version }}
+          python-version: ${{ env.PYTHON_VERSION }}
           cache: 'poetry'
 
       - name: Install the project with poetry
         run: |
-          poetry env use python${{ matrix.python-version }}
+          poetry env use python${{ env.PYTHON_VERSION }}
           poetry check --lock
           poetry lock --no-update --no-cache
           poetry install --verbose --no-root --sync --with qa

.github/workflows/docker.yml

@@ -45,6 +45,7 @@ jobs:
       DOCKER_BUILDKIT: 1
     steps:
       - name: Get latest phylum-ci release version
+        id: get_vers
         # The API is called directly here instead of using git commands because the repo is not checked out yet
         run: |
           REL_VER_WITH_v=$( \
@@ -62,19 +63,19 @@ jobs:
           )
           REL_VER_WITHOUT_v="${REL_VER_WITH_v//v/}"
           echo "${REL_VER_WITH_v}" "${REL_VER_WITHOUT_v}"
-          echo "REL_VER_WITH_v=${REL_VER_WITH_v}" >> "${GITHUB_ENV}"
-          echo "REL_VER_WITHOUT_v=${REL_VER_WITHOUT_v}" >> "${GITHUB_ENV}"
+          echo "REL_VER_WITH_v=${REL_VER_WITH_v}" >> "${GITHUB_OUTPUT}"
+          echo "REL_VER_WITHOUT_v=${REL_VER_WITHOUT_v}" >> "${GITHUB_OUTPUT}"
 
       - name: Checkout the repo
         uses: actions/checkout@d632683dd7b4114ad314bca15554477dd762a938 # v4.2.0
         with:
           # This will ensure the checkout matches the tag for the latest release
-          ref: ${{ env.REL_VER_WITH_v }}
+          ref: ${{ steps.get_vers.outputs.REL_VER_WITH_v }}
 
       - name: Get phylum wheel from latest release
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-        run: gh release download ${{ env.REL_VER_WITH_v }} --pattern '*.whl'
+        run: gh release download ${{ steps.get_vers.outputs.REL_VER_WITH_v }} --pattern '*.whl'
 
       - name: Build default docker image with latest phylum wheel
         run: |
@@ -112,6 +113,8 @@ jobs:
         run: docker login --username ${{ github.actor }} --password ${{ secrets.GITHUB_TOKEN }} ghcr.io
 
       - name: Create specific docker tags and push them
+        env:
+          REL_VER_WITHOUT_v: ${{ steps.get_vers.outputs.REL_VER_WITHOUT_v }}
         run: |
           CLI_REL_VER=$(docker run --rm phylum-ci phylum --version | sed 's/phylum //')
           docker tag phylum-ci "phylumio/phylum-ci:${{ env.REL_VER_WITHOUT_v }}-CLI${CLI_REL_VER}"

.github/workflows/preview.yml

@@ -1,6 +1,7 @@
 # This is a workflow for previewing packages. It can be used for testing before a release to the "production" systems.
 # It will automatically create developmental release builds and make them available for all pushes to `main`. There is
-# also an ability to manually trigger this workflow, with an additional option to publish the package to TestPyPI.
+# also an ability to manually trigger this workflow, with additional options to (1) publish the package to TestPyPI and
+# (2) build, test, and make available a Windows standalone binary.
 ---
 name: Preview
 
@@ -13,22 +14,29 @@ on:
         type: boolean
         required: true
         default: false
+      CompileWindows:
+        description: "Create Windows binary"
+        type: boolean
+        required: true
+        default: true
 
   push:
     branches:
       - main
 
+env:
+  PYTHON_VERSION: "3.12"
+  POETRY_VERSION: "1.8.3"
+
 jobs:
   publish_preview:
     name: Build and Publish for Preview
     runs-on: ubuntu-latest
-    strategy:
-      matrix:
-        # It's only one Python version specified in a "matrix", but on purpose to stay DRY
-        python-version: ["3.12"]
     defaults:
       run:
         shell: bash
+    outputs:
+      next_ver: ${{ steps.dev_ver.outputs.next_ver }}
     steps:
       - name: Checkout the repo
         uses: actions/checkout@d632683dd7b4114ad314bca15554477dd762a938 # v4.2.0
@@ -37,7 +45,7 @@ jobs:
           fetch-depth: 0
 
       - name: Install poetry
-        run: pipx install poetry==1.8.3
+        run: pipx install poetry==${{ env.POETRY_VERSION }}
 
       - name: Configure poetry
         run: |
@@ -47,25 +55,27 @@ jobs:
       - name: Set up Python
         uses: actions/setup-python@f677139bbe7f9c59b41e40162b753c062f5d49a3 # v5.2.0
         with:
-          python-version: ${{ matrix.python-version }}
+          python-version: ${{ env.PYTHON_VERSION }}
           cache: 'poetry'
 
       - name: Install the project with poetry
         run: |
-          poetry env use python${{ matrix.python-version }}
+          poetry env use python${{ env.PYTHON_VERSION }}
           poetry check --lock
           poetry lock --no-update --no-cache
           poetry install --verbose --no-root --sync --with test,ci
 
       - name: Make developmental release version
+        id: dev_ver
         # poetry version rules do not provide for developmental releases as specified in PEP440.
         # It can be pieced together with these commands.
         run: |
           curr_ver=$(poetry version --short)
-          next_ver=$(poetry run semantic-release -vv version --print)
+          next_ver=$(poetry run semantic-release -v version --print)
           if [ "${curr_ver}" = "${next_ver}" ]; then
-            next_ver=$(poetry run semantic-release -vv version --print --patch)
+            next_ver=$(poetry run semantic-release -v version --print --patch)
           fi
+          echo "next_ver=${next_ver}" >> "${GITHUB_OUTPUT}"
           poetry version "${next_ver}.dev${GITHUB_RUN_NUMBER}"
 
       - name: Run tox via poetry
@@ -80,8 +90,98 @@ jobs:
           name: dist
           path: ./dist/
           if-no-files-found: error
-          retention-days: 7
 
       - name: Publish to TestPyPI
         if: inputs.TestPyPI
         run: poetry publish --repository testpypi --username __token__ --password ${{ secrets.TESTPYPI_API_TOKEN }}
+
+  build_windows:
+    name: Build Windows standalone binary
+    if: inputs.CompileWindows
+    needs: publish_preview
+    runs-on: windows-latest
+    steps:
+      - name: Checkout the repo
+        uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
+
+        # Nuitka needs the packaged form and not the editable install Poetry provides
+        # Ref: https://github.com/Nuitka/Nuitka/issues/2965
+      - name: Download build artifacts
+        uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
+
+      - name: Set up Python
+        uses: actions/setup-python@f677139bbe7f9c59b41e40162b753c062f5d49a3 # v5.2.0
+        with:
+          python-version: ${{ env.PYTHON_VERSION }}
+
+      - name: Install poetry
+        run: pipx install --python ${{ env.PYTHON_VERSION }} poetry==${{ env.POETRY_VERSION }}
+
+      - name: Configure poetry
+        run: poetry config virtualenvs.in-project true
+
+      - name: Install the project with poetry
+        run: |
+          poetry check --lock
+          poetry lock --no-update --no-cache
+          poetry install --verbose --no-root --sync --with compile
+          poetry run python -m pip install --find-links dist --no-index phylum
+
+      - name: Compile binary with Nuitka
+        env:
+          PREVIEW_VER: ${{ needs.publish_preview.outputs.next_ver }}
+        run: |
+          poetry run python -m nuitka `
+            --onefile `
+            --output-dir=build `
+            --output-filename="phylum-ci.exe" `
+            --include-package=phylum `
+            --include-package-data=phylum `
+            --include-distribution-metadata=phylum `
+            --onefile-tempdir-spec="{CACHE_DIR}/{PRODUCT}/{VERSION}" `
+            --product-name=phylum-ci `
+            --product-version=${env:PREVIEW_VER} `
+            --file-version=${env:GITHUB_RUN_NUMBER} `
+            --company-name="Phylum, Inc." `
+            --copyright="Copyright (C) 2024 Phylum, Inc." `
+            --file-description="Analyze dependencies in CI with Phylum" `
+            --windows-icon-from-ico="docs/img/favicon.ico" `
+            --warn-implicit-exceptions `
+            --warn-unusual-code `
+            --assume-yes-for-downloads `
+            --report=nuitka-compilation-report.xml `
+            --deployment `
+            src/phylum/ci/cli.py
+
+      - name: Confirm operation of binary
+        env:
+          PHYLUM_API_KEY: ${{ secrets.PHYLUM_TOKEN }}
+          PHYLUM_BYPASS_CI_DETECTION: true
+        run: |
+          ./build/phylum-ci.exe -h
+          ./build/phylum-ci.exe -vvaf
+
+      - name: Upload standalone binary
+        if: always()
+        uses: actions/upload-artifact@50769540e7f4bd5e21e526ee35c689e35e0d6874 # v4.4.0
+        with:
+          name: phylum-ci.exe
+          path: ./build/phylum-ci.exe
+          if-no-files-found: error
+
+      - name: Upload compilation report
+        if: always()
+        uses: actions/upload-artifact@50769540e7f4bd5e21e526ee35c689e35e0d6874 # v4.4.0
+        with:
+          name: nuitka-compilation-report.xml
+          path: ./nuitka-compilation-report.xml
+          if-no-files-found: warn
+
+      # Nuitka will create a crash report with a static name when there are failures
+      - name: Upload crash report
+        if: always()
+        uses: actions/upload-artifact@50769540e7f4bd5e21e526ee35c689e35e0d6874 # v4.4.0
+        with:
+          name: nuitka-crash-report.xml
+          path: ./nuitka-crash-report.xml
+          if-no-files-found: ignore