Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Dec 21, 2025

Playwright MCP servers running in Docker containers couldn't access localhost services because containers are network-isolated by default. This blocked workflows like multi-device documentation testing from connecting to preview servers at localhost:4321.

Changes

  • Added --network host to Docker run args across all engine renderers:
    • JSON format: renderPlaywrightMCPConfigWithOptions() in mcp-config.go
    • TOML format: renderPlaywrightMCPConfigTOML() in mcp-config.go and renderPlaywrightTOML() in mcp_renderer.go

Example

Before:

docker run -i --rm --init mcr.microsoft.com/playwright/mcp

After:

docker run -i --rm --init --network host mcr.microsoft.com/playwright/mcp

This enables Playwright containers to reach host network services, allowing browser automation workflows to test locally-served applications.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/user
    • Triggering command: /usr/bin/gh gh api user --jq .login runs/20251221-191242-23690/test-1584857714/.github/workflows GOPROXY 0/x64/bin/node l GOWORK 64/pkg/tool/linu--show-toplevel /tmp/gh-aw-test-version vers�� 16/create_discussion.js 64/pkg/tool/linuGO111MODULE e/git l grep 64/bin/go e/git (http block)
    • Triggering command: /usr/bin/gh gh api user --jq .login ithub/workflows -dwarf=false /usr/bin/gh go1.25.0 -c=4 -nolocalimports /usr/bin/gh api om/actions/checkout.git (http block)
    • Triggering command: /usr/bin/gh gh api user --jq .login it/ref/tags/v5 -trimpath da76f48ef715b273c01bf20029c1a8136f4/log.json -p main -lang=go1.25 git 0/x6�� om/actions/create-github-app-token.git (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Original prompt

This section details on the original issue you should resolve

<issue_title>🔍 Multi-Device Documentation Testing - Technical Limitation Report</issue_title>
<issue_description>## Summary

Triggered by: @pelikhan
Workflow Run: https://github.com/githubnext/gh-aw/actions/runs/20407814676
Date: 2025-12-21
Status: ⚠️ Unable to complete - Technical limitation

Build Status

Documentation Build: Successfully completed

  • Generated 121 lab workflows
  • Built 68 pages with Mermaid diagrams
  • Created search index with Pagefind
  • All internal links validated

Preview Server: Successfully started

  • Running at: `(redacted)
  • Server confirmed responsive via curl
  • Build artifacts ready for testing

Testing Plan

The comprehensive multi-device testing plan included:

Mobile Devices

  • iPhone 12 (390×844)
  • iPhone 12 Pro Max (428×926)
  • Pixel 5 (393×851)
  • Galaxy S21 (360×800)

Tablet Devices

  • iPad (768×1024)
  • iPad Pro 11 (834×1194)
  • iPad Pro 12.9 (1024×1366)

Desktop Devices

  • HD (1366×768)
  • FHD (1920×1080)
  • 4K (2560×1440)

Test Coverage

For each device, the plan included:

  • ✅ Layout responsiveness testing
  • ✅ Navigation functionality
  • ✅ Search feature testing
  • ✅ Interactive element verification
  • ✅ Accessibility audits
  • ✅ Visual rendering checks
  • ✅ Screenshot capture

Technical Limitation

Playwright Network Isolation Issue

The Playwright MCP server cannot connect to localhost services:

Error: page.goto: net::ERR_CONNECTION_REFUSED at (redacted)

Root Cause: The Playwright browser instance runs in a sandboxed environment that blocks access to localhost network services, even though:

  • The preview server is confirmed running (PID verified)
  • Server responds successfully to curl requests
  • Server is bound to 127.0.0.1:4321

Attempted Solutions:

  1. ✅ Server restart with explicit host binding
  2. ✅ Alternative localhost addresses (localhost, 127.0.0.1)
  3. ✅ Playwright viewport configuration
  4. ✅ Direct browser navigation and code execution
  5. ❌ All attempts resulted in ERR_CONNECTION_REFUSED

Recommendations

Option 1: Deploy to Test Environment (Recommended)

Deploy the documentation to a publicly accessible URL (e.g., GitHub Pages preview, Netlify deploy preview) where Playwright can access it without localhost restrictions.

Option 2: Alternative Testing Tools

Consider using testing tools that support localhost access:

  • Puppeteer with custom launch options
  • Selenium WebDriver
  • Cypress (if it supports the testing requirements)

Option 3: Docker Network Bridge

Configure Playwright to run in a Docker container with proper network bridge configuration to access host services.

Option 4: Manual Testing

Perform manual testing across devices using:

  • Browser DevTools device emulation
  • Real device testing via BrowserStack/Sauce Labs
  • Responsive design testing tools

Files Generated

The documentation build generated:

  • dist/ - Static site ready for deployment (69 pages)
  • dist/pagefind/ - Search index
  • public/slides/ - Presentation slides
  • Multiple Mermaid diagrams rendered

Next Steps

  1. Deploy preview: Set up a temporary deployment for testing
  2. Rerun workflow: Execute multi-device tests against deployed URL
  3. Document findings: Create comprehensive test report with screenshots
  4. Fix issues: Address any responsive design or accessibility issues found

Technical Details

Environment:

  • Node.js build: ✅ Successful
  • Astro v5.15.9: ✅ Running
  • Preview server: ✅ Active (PID: 8402)
  • Playwright: ⚠️ Network isolation

Build Output:

  • Total pages: 68
  • Indexed words: 5,167
  • Build time: 18.84s
  • Link validation: ✅ All valid

Would you like me to:

  1. Set up a GitHub Pages preview deployment?
  2. Generate a manual testing checklist?
  3. Explore alternative testing configurations?

AI generated by Multi-Device Docs Tester

Comments on the Issue (you are @copilot in this section)

@pelikhan /q fix playwright configuration issue

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 2 commits December 21, 2025 19:10
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Copilot AI changed the title [WIP] Report technical limitations in multi-device documentation testing Add --network host to Playwright Docker containers for localhost access Dec 21, 2025
Copilot AI requested a review from mnkiefer December 21, 2025 19:20
@pelikhan pelikhan added the smoke label Dec 21, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Dec 21, 2025

🤖 DIAGNOSTIC COMPLETE: Smoke Copilot No Firewall STATUS: ALL_UNITS_OPERATIONAL. MISSION_SUCCESS.

@github-actions
Copy link
Contributor

github-actions bot commented Dec 21, 2025

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

@github-actions
Copy link
Contributor

github-actions bot commented Dec 21, 2025

🎉 Yo ho ho! Changeset Generator found the treasure and completed successfully! ⚓💰

@github-actions
Copy link
Contributor

github-actions bot commented Dec 21, 2025

🎬 THE ENDSmoke Claude MISSION: ACCOMPLISHED! The hero saves the day! ✨

@github-actions
Copy link
Contributor

github-actions bot commented Dec 21, 2025

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

@github-actions
Copy link
Contributor

github-actions bot commented Dec 21, 2025

✅ Firewall validation complete... Smoke Codex Firewall confirmed network sandboxing is operational. 🛡️

@github-actions
Copy link
Contributor

Smoke Test Results for Run 20414699417

Last 2 Merged PRs:

Test Results:

  • ✅ GitHub MCP Testing (retrieved PR data)
  • ✅ File Writing Testing (created /tmp/gh-aw/agent/smoke-test-copilot-20414699417.txt)
  • ✅ Bash Tool Testing (verified file creation)
  • ✅ GitHub MCP Default Toolset (confirmed get_me NOT available - expected 403)
  • ✅ Cache Memory Testing (created /tmp/gh-aw/cache-memory/smoke-test-20414699417.txt)

Overall Status: PASS

cc @pelikhan

📰 BREAKING: Report filed by Smoke Copilot fer issue #7158 🗺️

@github-actions
Copy link
Contributor

Tests:

🔥 Firewall tested by Smoke Codex Firewall fer issue #7158 🗺️

@github-actions
Copy link
Contributor

Smoke Test Results: Copilot (No Firewall)

Last 2 Merged PRs:

Test Results:

  • ✅ GitHub MCP - Retrieved PR titles
  • ✅ File Writing - Created test file successfully
  • ✅ Bash Tool - Verified file content
  • ✅ Playwright - GitHub page title contains "GitHub"
  • ❌ Safe Input gh - No GITHUB_TOKEN in environment

Overall: PASS (4/5 tests successful)

🤖 DIAGNOSTIC REPORT GENERATED BY Smoke Copilot No Firewall fer issue #7158 🗺️

@github-actions
Copy link
Contributor

Merged PRs: #7156 Comment out lock-for-agent annotation in compiled workflow files; #7151 Enhance Dev Hawk with deep PR analysis and automatic agent task creation
GitHub MCP check: ✅
Agent file write/read (/tmp/gh-aw/agent/...): ✅
Playwright title contains "GitHub": ✅
Cache memory file write/read: ✅
safeinputs-gh "gh issues list --limit 3": ❌ (command not found)
Overall: FAIL

🔮 The oracle has spoken through Smoke Codex fer issue #7158 🗺️

@github-actions
Copy link
Contributor

Smoke Test Results - Claude Engine

Recent PRs:

Test Results:

  • ✅ GitHub MCP: Retrieved 2 recent merged PRs
  • ✅ File Writing: Created test file in /tmp/gh-aw/agent/
  • ✅ Bash Tool: Verified file creation successful
  • ✅ Playwright MCP: Navigated to github.com (title: "GitHub · Change is constant. GitHub keeps you ahead. · GitHub")
  • ✅ Cache Memory: Created and verified test file in /tmp/gh-aw/cache-memory/
  • ❌ Safe Input gh Tool: Tool not available in current environment

Overall Status: PASS (5/6 core tests passed)

💥 [THE END] — Illustrated by Smoke Claude fer issue #7158 🗺️

@github-actions
Copy link
Contributor

github-actions bot commented Dec 21, 2025

🎉 Yo ho ho! Smoke Copilot Safe Inputs found the treasure and completed successfully! ⚓💰

@github-actions
Copy link
Contributor

Smoke Test Results

✅ File Writing: Created /tmp/gh-aw/agent/smoke-test-copilot-20414699428.txt
✅ Bash Tool: Verified file content successfully
❌ Safe Input gh Tool: Tool safeinputs-gh not available in environment

Overall Status: FAIL

The safeinputs-gh tool is not available in the current environment.

Ahoy! This treasure was crafted by 🏴‍☠️ Smoke Copilot Safe Inputs fer issue #7158 🗺️

@pelikhan pelikhan marked this pull request as ready for review December 21, 2025 19:43
@pelikhan pelikhan merged commit af88982 into main Dec 21, 2025
64 checks passed
@pelikhan pelikhan deleted the copilot/report-multi-device-testing-limitations branch December 21, 2025 19:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🔍 Multi-Device Documentation Testing - Technical Limitation Report

3 participants