Add macOS ARM64 smoke test for container validation#16742
Conversation
Adds a new smoke test that runs on macos-latest (ARM64/Apple Silicon) to validate that ARM64 container images work correctly: firewall agent, squid, api-proxy, and MCP gateway. Agent instructions match smoke-copilot. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR adds a new smoke test workflow for validating ARM64 container images on macOS Apple Silicon (ARM64) runners. The workflow tests that firewall agent, squid, api-proxy, and MCP gateway containers work correctly on the ARM64 architecture.
Changes:
- New
smoke-macos-arm64.mdworkflow that runs onmacos-latestrunners - Tests ARM64 container image functionality with architecture verification, Docker availability checks, container pull/inspect tests, and startup validation
- Includes comprehensive testing of MCP servers (GitHub, Serena, Playwright), safe inputs/outputs, and workflow dispatch capabilities
Reviewed changes
Copilot reviewed 1 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
.github/workflows/smoke-macos-arm64.md |
New workflow definition for macOS ARM64 smoke tests with container validation instructions |
.github/workflows/smoke-macos-arm64.lock.yml |
Generated lock file with compiled workflow YAML configuration |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| - `ghcr.io/github/gh-aw-firewall/agent:v0.20.0` | ||
| - `ghcr.io/github/gh-aw-firewall/squid:v0.20.0` | ||
| - `ghcr.io/github/gh-aw-firewall/api-proxy:v0.20.0` | ||
| - `ghcr.io/github/gh-aw-mcpg:v0.1.4` | ||
| For each image, run `docker inspect --format '{{.Architecture}}' <image>` and verify it reports `arm64`. | ||
| 4. **Container Startup Test**: For each pulled image, attempt to start a container and verify it runs without crash: | ||
| - For `agent`: `docker run --rm ghcr.io/github/gh-aw-firewall/agent:v0.20.0 echo "agent OK"` | ||
| - For `squid`: `docker run --rm -d --name smoke-squid ghcr.io/github/gh-aw-firewall/squid:v0.20.0` then check it's running with `docker ps`, then `docker stop smoke-squid` | ||
| - For `api-proxy`: `docker run --rm ghcr.io/github/gh-aw-firewall/api-proxy:v0.20.0 --help` (or similar basic invocation) | ||
| - For `mcpg`: `docker run --rm ghcr.io/github/gh-aw-mcpg:v0.1.4 --help` (or similar basic invocation) |
There was a problem hiding this comment.
There is an inconsistency in how firewall container image versions are specified. In the test instructions (lines 125-128), the images are referenced with the "v" prefix:
ghcr.io/github/gh-aw-firewall/agent:v0.20.0ghcr.io/github/gh-aw-firewall/squid:v0.20.0ghcr.io/github/gh-aw-firewall/api-proxy:v0.20.0
However, in the generated lock file (line 477), these same images are downloaded without the "v" prefix:
ghcr.io/github/gh-aw-firewall/agent:0.20.0ghcr.io/github/gh-aw-firewall/api-proxy:0.20.0ghcr.io/github/gh-aw-firewall/squid:0.20.0
While in line 1644 of the lock file, the --image-tag flag correctly uses 0.20.0 without the "v" prefix.
The agent will attempt to pull images with the "v" prefix (as instructed in lines 125-128, 131-134) but those images may not exist if only the non-prefixed versions are available in the registry. Please ensure consistency: either add the "v" prefix to all references or remove it from the test instructions.
| - `ghcr.io/github/gh-aw-firewall/agent:v0.20.0` | |
| - `ghcr.io/github/gh-aw-firewall/squid:v0.20.0` | |
| - `ghcr.io/github/gh-aw-firewall/api-proxy:v0.20.0` | |
| - `ghcr.io/github/gh-aw-mcpg:v0.1.4` | |
| For each image, run `docker inspect --format '{{.Architecture}}' <image>` and verify it reports `arm64`. | |
| 4. **Container Startup Test**: For each pulled image, attempt to start a container and verify it runs without crash: | |
| - For `agent`: `docker run --rm ghcr.io/github/gh-aw-firewall/agent:v0.20.0 echo "agent OK"` | |
| - For `squid`: `docker run --rm -d --name smoke-squid ghcr.io/github/gh-aw-firewall/squid:v0.20.0` then check it's running with `docker ps`, then `docker stop smoke-squid` | |
| - For `api-proxy`: `docker run --rm ghcr.io/github/gh-aw-firewall/api-proxy:v0.20.0 --help` (or similar basic invocation) | |
| - For `mcpg`: `docker run --rm ghcr.io/github/gh-aw-mcpg:v0.1.4 --help` (or similar basic invocation) | |
| - `ghcr.io/github/gh-aw-firewall/agent:0.20.0` | |
| - `ghcr.io/github/gh-aw-firewall/squid:0.20.0` | |
| - `ghcr.io/github/gh-aw-firewall/api-proxy:0.20.0` | |
| - `ghcr.io/github/gh-aw-mcpg:0.1.4` | |
| For each image, run `docker inspect --format '{{.Architecture}}' <image>` and verify it reports `arm64`. | |
| 4. **Container Startup Test**: For each pulled image, attempt to start a container and verify it runs without crash: | |
| - For `agent`: `docker run --rm ghcr.io/github/gh-aw-firewall/agent:0.20.0 echo "agent OK"` | |
| - For `squid`: `docker run --rm -d --name smoke-squid ghcr.io/github/gh-aw-firewall/squid:0.20.0` then check it's running with `docker ps`, then `docker stop smoke-squid` | |
| - For `api-proxy`: `docker run --rm ghcr.io/github/gh-aw-firewall/api-proxy:0.20.0 --help` (or similar basic invocation) | |
| - For `mcpg`: `docker run --rm ghcr.io/github/gh-aw-mcpg:0.1.4 --help` (or similar basic invocation) |
|
Gh-aw is probably not working. It gets enabled with tools.agentic-workflows |
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
Hey However, there's a process concern to address:
While you're a collaborator and this is infrastructure work (which may have different internal processes), the documented workflow expects all changes to flow through the issue → agent → PR pipeline. Suggested Path ForwardIf you'd like to align with the documented process:
Alternatively, if internal contributors have different guidelines for infrastructure work, consider documenting that exception in CONTRIBUTING.md to avoid confusion.
|
Summary
smoke-macos-arm64agentic workflow that runs onmacos-latest(Apple Silicon ARM64 runner)ghcr.io/github/gh-aw-firewall/*:v0.20.0,ghcr.io/github/gh-aw-mcpg:v0.1.4)smoke-copilotwith additional ARM64-specific tests: architecture verification, Docker availability, ARM64 image pull/inspect, and container startup validationTest plan
gh aw compile .github/workflows/smoke-macos-arm64.md)runs-on: macos-lateston the agent jobsmokelabel to validate end-to-end on a macOS ARM64 runner🤖 Generated with Claude Code