Skip to content

Add E2E Testing GitHub Action with comprehensive documentation#1

Merged
kiyarose merged 5 commits intomainfrom
copilot/add-e2e-testing-suite
Oct 24, 2025
Merged

Add E2E Testing GitHub Action with comprehensive documentation#1
kiyarose merged 5 commits intomainfrom
copilot/add-e2e-testing-suite

Conversation

Copy link
Contributor

Copilot AI commented Oct 24, 2025

  • Create .github/workflows directory structure
  • Create the E2E testing workflow example file
  • Update README with comprehensive documentation
  • Fix npm cache dependency error by removing cache: 'npm' from workflow
  • Add action.yml for GitHub Marketplace support with:
    • Configurable inputs (port, node-version, build-command, preview-command, test-files, screenshot-prefix)
    • Composite action implementation
    • Branding (camera icon, purple color)
  • Update README with:
    • Usage as GitHub Action (recommended method)
    • Action inputs documentation
    • Examples with custom inputs
  • Rename workflow to .example extension to prevent execution in action repo
Original prompt

Make an github actions repo based on ```name: E2E Testing Suite (Screenshots & Accessibility)

concurrency:
group: ${{ github.event_name == 'pull_request' && format('e2e-pr-{0}', github.event.pull_request.number) || format('e2e-{0}', github.ref) }}
cancel-in-progress: false # Don't cancel immediately - wait for timeout

permissions:
contents: write # needed to push to gh-pages
pages: write # deploy with actions/deploy-pages
id-token: write # required by actions/deploy-pages
pull-requests: write
issues: write

on:
push:
branches:
- main
pull_request:

jobs:
e2e-tests:
name: E2E Tests (Screenshots & A11y)
runs-on: ubuntu-latest
env:
PORT: 4173 # Port for preview server and Playwright tests

steps:
  - name: Checkout
    uses: actions/checkout@v4

  # Simple notification for PR-specific concurrency
  - name: PR Test Info
    if: github.event_name == 'pull_request'
    uses: actions/github-script@v7
    with:
      script: |
        const prNumber = context.payload.pull_request.number;
        core.info(`Starting E2E tests for PR ${prNumber} with isolated concurrency group.`);
        core.info(`This PR will publish screenshots to: /(Dev-Shots)/PR-${prNumber}/`);

  - name: Use Node
    uses: actions/setup-node@v4
    with:
      node-version: '20'
      cache: 'npm'

  - name: Install deps
    run: npm ci

  - name: Build site
    run: npm run build

  # --- Added steps to start and wait for the preview server ---
  - name: Start preview server
    run: npm run preview -- --port $PORT &

  - name: Wait for server to be ready
    run: npx wait-on http://localhost:$PORT
  # -----------------------------------------------------------

  - name: Cache Playwright browsers
    id: playwright-cache
    uses: actions/cache@v4
    with:
      path: ~/.cache/ms-playwright
      key: ${{ runner.os }}-playwright-${{ hashFiles('package-lock.json') }}

  - name: Install Playwright browsers
    if: steps.playwright-cache.outputs.cache-hit != 'true'
    run: npx playwright install --with-deps

  - name: Run Playwright tests (screenshots & accessibility)
    run: |
      npx playwright test tests/e2e.spec.ts tests/a11y.spec.ts

  - name: Upload artifacts
    uses: actions/upload-artifact@v4
    with:
      name: e2e-test-results
      path: |
        playwright-logs/portfolio-web-light.png
        playwright-logs/portfolio-web-dark.png
        playwright-logs/portfolio-mobile-light.png
        playwright-logs/portfolio-mobile-dark.png
        playwright-report/**
        axe-report.json

  - name: Add screenshot to job summary
    run: |
      web_light="playwright-logs/portfolio-web-light.png"
      web_dark="playwright-logs/portfolio-web-dark.png"
      mobile_light="playwright-logs/portfolio-mobile-light.png"
      mobile_dark="playwright-logs/portfolio-mobile-dark.png"

      has_any=false
      if [ -f "$web_light" ] || [ -f "$web_dark" ] || [ -f "$mobile_light" ] || [ -f "$mobile_dark" ]; then
        has_any=true
      fi

      if [ "$has_any" = true ]; then
        {
          echo "## Web Render"
          if [ -f "$web_light" ] || [ -f "$web_dark" ]; then
            echo
            echo "### Web"
            if [ -f "$web_light" ]; then
              echo
              echo "#### Light"
              echo
              echo "![Web Light Render]($web_light)"
            fi
            if [ -f "$web_dark" ]; then
              echo
              echo "#### Dark"
              echo
              echo "![Web Dark Render]($web_dark)"
            fi
          fi
          if [ -f "$mobile_light" ] || [ -f "$mobile_dark" ]; then
            echo
            echo "### Mobile"
            if [ -f "$mobile_light" ]; then
              echo
              echo "#### Light"
              echo
              echo "![Mobile Light Render]($mobile_light)"
            fi
            if [ -f "$mobile_dark" ]; then
              echo
              echo "#### Dark"
              echo
              echo "![Mobile Dark Render]($mobile_dark)"
            fi
          fi
          echo
          echo "_This screenshot was generated by Playwright during the latest workflow run._"
        } >> "$GITHUB_STEP_SUMMARY"
      else
        echo "⚠️ Screenshots not found." >> "$GITHUB_STEP_SUMMARY"
      fi

  - name: Prepare Pages payload and workspace archive
    id: prep_pages
    run: |
      # Use PR-specific directory structure instead of run-based
      if [ "${{ github.event_name }}" = "pull_request" ]; then
        pr_number="${{ github.event.pul...

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Co-authored-by: kiyarose <75678535+kiyarose@users.noreply.github.com>
Copilot AI changed the title [WIP] Add E2E Testing suite for screenshots and accessibility Add E2E Testing GitHub Action with comprehensive documentation Oct 24, 2025
Copilot AI requested a review from kiyarose October 24, 2025 12:45
@kiyarose kiyarose marked this pull request as ready for review October 24, 2025 12:56
kiyarose

This comment was marked as outdated.

Co-authored-by: kiyarose <75678535+kiyarose@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Oct 24, 2025

@copilot
Error: Dependencies lock file is not found in /home/runner/work/E2E/E2E. Supported file patterns: package-lock.json,npm-shrinkwrap.json,yarn.lock on run.
my full original code was...

Fixed in commit 9a789f6:

Issue 1: npm cache error - Removed cache: 'npm' from the workflow since this is a reusable action template that won't have a package-lock.json file in the action repository itself.

Issue 2: Marketplace ready - Added action.yml file with:

  • Composite action definition
  • Configurable inputs (port, node-version, build-command, preview-command, test-files, screenshot-prefix)
  • Branding (camera icon, purple color)

The action can now be used directly from the Marketplace:

- uses: SillyLittleTech/E2E@v1
  with:
    port: '4173'
    node-version: '20'

Updated README with usage examples, action inputs table, and marketplace publishing instructions.

@kiyarose
Copy link
Member

@copilot we don't need marketplace information directly in the readme, we can leave that out. you can yap to me about how to actually get started with marketplace in a comment here.

Copilot AI requested a review from kiyarose October 24, 2025 13:04
@kiyarose
Copy link
Member

@copilot additionally, we are still failing.

npm error code EUSAGE
npm error
npm error The `npm ci` command can only install with an existing package-lock.json or
npm error npm-shrinkwrap.json with lockfileVersion >= 1. Run an install with npm@5 or
npm error later to generate a package-lock.json file, then try again.
npm error
npm error Clean install a project
npm error
npm error Usage:
npm error npm ci
npm error
npm error Options:
npm error [--install-strategy <hoisted|nested|shallow|linked>] [--legacy-bundling]
npm error [--global-style] [--omit <dev|optional|peer> [--omit <dev|optional|peer> ...]]
npm error [--include <prod|dev|optional|peer> [--include <prod|dev|optional|peer> ...]]
npm error [--strict-peer-deps] [--foreground-scripts] [--ignore-scripts] [--no-audit]
npm error [--no-bin-links] [--no-fund] [--dry-run]
npm error [-w|--workspace <workspace-name> [-w|--workspace <workspace-name> ...]]
npm error [-ws|--workspaces] [--include-workspace-root] [--install-links]
npm error
npm error aliases: clean-install, ic, install-clean, isntall-clean
npm error
npm error Run "npm help ci" for more info
npm error A complete log of this run can be found in: /home/runner/.npm/_logs/2025-10-24T13_03_46_107Z-debug-0.log
Error: Process completed with exit code 1.

Co-authored-by: kiyarose <75678535+kiyarose@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Oct 24, 2025

@copilot we don't need marketplace information directly in the readme, we can leave that out. you can yap to me about how to actually get started with marketplace in a comment here.

Removed marketplace info from README in commit e0a560b.

Getting Started with GitHub Actions Marketplace:

  1. Prerequisites: You have action.yml ready (already done ✓)

  2. Create a Release:

    • Go to your repository → Releases → "Create a new release"
    • Create a new tag (e.g., v1.0.0) following semantic versioning
    • Title: "v1.0.0" or descriptive name
    • Description: Release notes
  3. Publish to Marketplace:

    • During release creation, check ☑️ "Publish this Action to the GitHub Marketplace"
    • Primary category: Choose "Testing" or "Continuous Integration"
    • Add up to 2 more categories if desired
    • The action will be automatically listed after publishing the release
  4. Version Tags (create these after your first release):

    git tag -f v1 v1.0.0
    git push origin v1 --force

    This lets users reference @v1 for the latest 1.x version

  5. Users can then use:

    - uses: SillyLittleTech/E2E@v1

The action will appear at: https://github.com/marketplace/actions/e2e-testing-suite

Co-authored-by: kiyarose <75678535+kiyarose@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Oct 24, 2025

@copilot additionally, we are still failing.

npm error code EUSAGE

...

Fixed in commit 9b29826.

The workflow file was trying to run in this action repository (which has no package.json). Renamed .github/workflows/e2e-testing.yml to .github/workflows/e2e-testing.yml.example so it won't auto-execute here.

For users:

  • Option 1 (Recommended): Use the action directly via uses: SillyLittleTech/E2E@v1 - no workflow file needed
  • Option 2: Copy the .example file to their repository (renamed without .example) for full GitHub Pages integration

The action repository itself no longer has executable workflows, preventing the npm ci error.

@kiyarose kiyarose merged commit 52c9db1 into main Oct 24, 2025
@kiyarose kiyarose deleted the copilot/add-e2e-testing-suite branch October 24, 2025 13:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Comments