Switch to browser-based bundle size measurement using Playwright

Co-authored-by: adamziel <205419+adamziel@users.noreply.github.com>

Author: Copilot

.github/workflows/bundle-size-check.yml

@@ -23,8 +23,20 @@ jobs:
             - name: Build website
               run: npx nx build playground-website
 
-            - name: Analyze current bundle size
-              run: node tools/scripts/analyze-bundle-size.mjs
+            - name: Start preview server
+              run: |
+                  npx nx preview playground-website &
+                  # Wait for server to be ready
+                  timeout 60 bash -c 'until curl -s http://localhost:5400 > /dev/null; do sleep 1; done'
+
+            - name: Install Playwright browsers
+              run: npx playwright install chromium --with-deps
+
+            - name: Measure current bundle size
+              run: node tools/scripts/measure-bundle-size-browser.mjs
+
+            - name: Stop preview server
+              run: pkill -f "nx preview" || true
 
             - name: Save current report
               run: cp bundle-size-report.json bundle-size-report-current.json
@@ -41,8 +53,17 @@ jobs:
             - name: Build base branch website
               run: npx nx build playground-website
 
-            - name: Analyze base bundle size
-              run: node tools/scripts/analyze-bundle-size.mjs
+            - name: Start preview server for base branch
+              run: |
+                  npx nx preview playground-website &
+                  # Wait for server to be ready
+                  timeout 60 bash -c 'until curl -s http://localhost:5400 > /dev/null; do sleep 1; done'
+
+            - name: Measure base bundle size
+              run: node tools/scripts/measure-bundle-size-browser.mjs
+
+            - name: Stop preview server
+              run: pkill -f "nx preview" || true
 
             - name: Save base report
               run: cp bundle-size-report.json bundle-size-report-base.json

tools/scripts/README.md

@@ -1,136 +1,161 @@
 # Bundle Size Tracking
 
-This directory contains scripts for tracking and reporting bundle size changes in WordPress Playground.
+This directory contains scripts for tracking and reporting bundle size changes in WordPress Playground using real browser measurements.
 
 ## Overview
 
-The bundle size tracking system helps ensure that changes to the codebase don't significantly increase the download size required for:
+The bundle size tracking system uses Playwright to measure actual download sizes at key stages during page load:
 
-1. **First Paint**: Assets needed to display the initial WordPress Playground interface
-2. **Offline Mode**: Assets cached for offline functionality
+1. **First Paint**: Assets downloaded until the progress bar is visible
+2. **WordPress Loaded**: Assets downloaded until WordPress site is ready (nested iframes loaded)
+3. **Offline Mode Ready**: All assets downloaded after network activity settles
+
+This approach provides real-world measurements instead of static file analysis.
 
 ## Scripts
 
-### `analyze-bundle-size.mjs`
+### `measure-bundle-size-browser.mjs`
 
-Analyzes the build output and generates a detailed report of asset sizes.
+Uses Playwright to measure bundle size by monitoring actual browser network requests.
 
 **Usage:**
-
 ```bash
-npm run build:website
-node tools/scripts/analyze-bundle-size.mjs
-```
+# Start the development server
+npm run dev
 
-**Output:**
-
--   `bundle-size-report.json`: Detailed JSON report with size information for all assets
+# In another terminal, run the measurement
+node tools/scripts/measure-bundle-size-browser.mjs
+```
 
 **What it measures:**
+- Total bytes transferred at each stage
+- Number of files loaded
+- Time to each milestone
+- Top 10 largest files at each stage
+- Breakdown by resource type (script, stylesheet, image, etc.)
 
--   Total size and gzipped size for first paint assets
--   Total size and gzipped size for offline mode assets
--   Individual file sizes
--   Top 10 largest files in each category
+**Output:**
+- `bundle-size-report.json`: Detailed JSON report with measurements
 
 ### `compare-bundle-size.mjs`
 
 Compares two bundle size reports and generates a markdown report suitable for GitHub PR comments.
 
 **Usage:**
-
 ```bash
 node tools/scripts/compare-bundle-size.mjs [base-report] [current-report]
 ```
 
 **Default paths:**
-
--   `base-report`: `bundle-size-report-base.json`
--   `current-report`: `bundle-size-report.json`
+- `base-report`: `bundle-size-report-base.json`
+- `current-report`: `bundle-size-report.json`
 
 **Output:**
-
--   `bundle-size-comment.md`: Markdown-formatted comparison report
--   GitHub Actions outputs for workflow automation
+- `bundle-size-comment.md`: Markdown-formatted comparison report
+- GitHub Actions outputs for workflow automation
 
 ## CI Workflow
 
 The bundle size check runs automatically on pull requests via the `.github/workflows/bundle-size-check.yml` workflow.
 
 ### How it works
 
-1. **Build Current Branch**: Builds the website from the PR branch and analyzes the bundle size
-2. **Build Base Branch**: Checks out and builds the base branch (usually `trunk`) and analyzes its bundle size
-3. **Compare**: Generates a comparison report showing size changes
-4. **Comment**: If size changes exceed 50 KB (gzipped) in either category, posts a comment on the PR
+1. **Build & Start Current Branch**: 
+   - Builds the website from the PR branch
+   - Starts the preview server
+   - Installs Playwright
+   - Measures bundle size with real browser
 
-### Comment Threshold
+2. **Build & Start Base Branch**: 
+   - Checks out and builds the base branch (usually `trunk`)
+   - Starts the preview server
+   - Measures its bundle size with real browser
 
-A PR comment is posted when:
+3. **Compare**: 
+   - Generates a comparison report showing size changes at each stage
+   - Includes time delta as well as size delta
 
--   First paint assets change by more than ±50 KB (gzipped), OR
--   Offline mode assets change by more than ±50 KB (gzipped)
+4. **Comment**: 
+   - If any stage changes by more than 50 KB, posts a comment on the PR
+
+### Comment Threshold
+
+A PR comment is posted when any of these change by more than ±50 KB:
+- First paint downloads
+- WordPress loaded downloads
+- Offline mode ready downloads
 
 ### Comment Format
 
-The PR comment includes:
+The PR comment includes three sections:
+
+#### 🎨 First Paint (Progress Bar Visible)
+- Current vs. base size and load time
+- Delta in bytes and time
+- Top 10 largest files
+
+#### ✅ WordPress Loaded (Site Ready)
+- Current vs. base size and load time
+- Delta in bytes and time
+- Top 10 largest files
+
+#### 💾 Offline Mode Ready (All Downloads Settled)
+- Current vs. base size and load time
+- Delta in bytes and time
+- Top 10 largest files
 
--   **Size Comparison**: Current vs. base size with delta
--   **File Count**: Number of files in each category
--   **Files with Largest Changes**: Top 10 files with the biggest size deltas
--   **Top 10 Largest Files**: Current largest files in each category
--   **Status Indicators**:
-    -   🆕 New file
-    -   🗑️ Removed file
-    -   📈 Size increased
-    -   📉 Size decreased
-    -   ➡️ No change
+**Status Indicators**:
+- 📈 Size increased
+- 📉 Size decreased
+- ➡️ No change
 
-## First Paint Assets
+## Measurement Stages
 
-Files considered critical for the first paint include:
+### First Paint (Progress Bar Visible)
 
--   `index.html` and `remote.html`
--   Core JavaScript bundles in `/assets/` (excluding optional chunks)
--   Core CSS files
--   Service worker
--   Manifest files
+Measures all downloads until the progress bar becomes visible. This represents the minimum assets needed for users to see that the page is loading.
 
-**Excluded from first paint:**
+**Key signals:**
+- Progress bar element visible
+- Falls back to DOMContentLoaded if no progress bar found
 
--   Optional chunks (e.g., CodeMirror extensions in `/assets/optional/`)
--   PHP WASM files (loaded on demand)
--   WordPress build ZIPs (loaded on demand)
--   SQLite integration (loaded on demand)
--   Demos and builder assets
+### WordPress Loaded (Site Ready)
 
-## Offline Mode Assets
+Measures all downloads until WordPress is fully loaded in the nested iframe, indicating the site is interactive and ready to use.
 
-Files required for offline functionality are determined by the `assets-required-for-offline-mode.json` manifest, which is automatically generated during the build process by the `listAssetsRequiredForOfflineMode` Vite plugin.
+**Key signals:**
+- WordPress iframe body element is attached
+- Falls back to window load event if iframe not found
 
-See `packages/vite-extensions/vite-list-assets-required-for-offline-mode.ts` for details on how this manifest is generated.
+### Offline Mode Ready (All Downloads Settled)
+
+Measures all downloads after network activity settles (5 seconds of no new requests). This represents all assets that would be cached for offline use.
+
+**Key signals:**
+- No network requests for 5 consecutive seconds
+- Includes all lazy-loaded assets
 
 ## Local Development
 
 To test bundle size changes locally:
 
 ```bash
-# Build the website
-npm run build:website
+# Terminal 1: Start the dev server
+npm run dev
 
-# Analyze current build
-node tools/scripts/analyze-bundle-size.mjs
+# Terminal 2: Measure current build
+node tools/scripts/measure-bundle-size-browser.mjs
 
 # Save as base for comparison
 cp bundle-size-report.json bundle-size-report-base.json
 
 # Make your changes...
 
-# Build again
-npm run build:website
+# Restart dev server if needed
+npm run dev
 
-# Analyze new build
-node tools/scripts/analyze-bundle-size.mjs
+# Measure new build
+node tools/scripts/measure-bundle-size-browser.mjs
 
 # Compare
 node tools/scripts/compare-bundle-size.mjs
@@ -143,13 +168,24 @@ If your PR triggers a bundle size increase:
 1. **Check for new dependencies**: Large libraries can significantly increase bundle size
 2. **Use code splitting**: Move non-critical code to lazy-loaded chunks
 3. **Optimize assets**: Compress images, minify code
-4. **Review bundle composition**: Use tools like `vite-bundle-visualizer` to understand what's taking up space
+4. **Review network tab**: Use browser DevTools to see what's being loaded
 5. **Consider alternatives**: Look for lighter-weight alternatives to heavy dependencies
+6. **Analyze resource types**: Check if images, scripts, or styles are the main contributor
+
+## Browser-Based Measurement Benefits
+
+Using real browser measurements instead of static file analysis provides:
+
+- **Realistic data**: Measures what users actually download
+- **Network behavior**: Captures caching, compression, and HTTP/2 multiplexing effects
+- **Load timing**: Shows when assets are downloaded relative to page milestones
+- **Resource prioritization**: Reflects browser's actual loading strategy
+- **Accurate offline assets**: Measures what's actually cached, not estimates
 
 ## Artifacts
 
 The workflow uploads the following artifacts for debugging:
 
--   `bundle-size-report.json`: Current branch analysis
--   `bundle-size-report-base.json`: Base branch analysis
--   `bundle-size-comment.md`: Generated PR comment
+- `bundle-size-report.json`: Current branch measurements
+- `bundle-size-report-base.json`: Base branch measurements
+- `bundle-size-comment.md`: Generated PR comment