diff --git a/.github/workflows/smoke-claude.lock.yml b/.github/workflows/smoke-claude.lock.yml index cabb94e516..3972367653 100644 --- a/.github/workflows/smoke-claude.lock.yml +++ b/.github/workflows/smoke-claude.lock.yml @@ -31,7 +31,7 @@ # - shared/mcp/tavily.md # - shared/reporting.md # -# frontmatter-hash: ce8a3d0dc475ddadadf2f7a44f9c96c4686d5dab00e046595fbbcfd7c2bcdebd +# frontmatter-hash: 78071d0b5aa3dd18c2e0a148703429c5c3859fc89d33b92309116cbbfdcd314c name: "Smoke Claude" "on": @@ -1737,7 +1737,7 @@ jobs: # - mcp__playwright__browser_type # - mcp__playwright__browser_wait_for # - mcp__tavily - timeout-minutes: 15 + timeout-minutes: 10 run: | set -o pipefail sudo -E awf --enable-chroot --tty --env-all --container-workdir "${GITHUB_WORKSPACE}" --allow-domains '*.githubusercontent.com,anthropic.com,api.anthropic.com,api.github.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,cdn.playwright.dev,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,files.pythonhosted.org,ghcr.io,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.com,github.githubassets.com,go.dev,golang.org,goproxy.io,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,mcp.tavily.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,pkg.go.dev,playwright.download.prss.microsoft.com,ppa.launchpad.net,proxy.golang.org,pypi.org,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,sentry.io,statsig.anthropic.com,sum.golang.org,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com' --log-level info --proxy-logs-dir /tmp/gh-aw/sandbox/firewall/logs --enable-host-access --image-tag 0.13.7 --skip-pull \ diff --git a/.github/workflows/smoke-claude.md b/.github/workflows/smoke-claude.md index 8d7b147c88..fad5b3daa2 100644 --- a/.github/workflows/smoke-claude.md +++ b/.github/workflows/smoke-claude.md @@ -67,7 +67,7 @@ safe-outputs: run-started: "💥 **WHOOSH!** [{workflow_name}]({run_url}) springs into action on this {event_type}! *[Panel 1 begins...]*" run-success: "🎬 **THE END** — [{workflow_name}]({run_url}) **MISSION: ACCOMPLISHED!** The hero saves the day! ✨" run-failure: "💫 **TO BE CONTINUED...** [{workflow_name}]({run_url}) {status}! Our hero faces unexpected challenges..." -timeout-minutes: 15 +timeout-minutes: 10 --- # Smoke Test: Claude Engine Validation. @@ -91,20 +91,25 @@ timeout-minutes: 15 - Extract the discussion number from the result (e.g., if the result is `{"number": 123, "title": "...", ...}`, extract 123) - Use the `add_comment` tool with `discussion_number: ` to add a fun, comic-book style comment stating that the smoke test agent was here 10. **Agentic Workflows MCP Testing**: - - Use the `agentic-workflows` MCP tool with the `status` method to query the status of the "smoke-claude" workflow in ${{ github.repository }} - - Extract key information: total runs, recent success/failure status, last run time - - Write a summary of the smoke-claude workflow status to `/tmp/gh-aw/agent/smoke-claude-status-${{ github.run_id }}.txt` - - Use bash to display the file contents + - Call the `agentic-workflows` MCP tool using the `status` method with workflow name `smoke-claude` to query workflow status + - If the tool returns an error or no results, mark this test as ❌ and note "Tool unavailable or workflow not found" but continue to the Output section + - If the tool succeeds, extract key information from the response: total runs, success/failure counts, last run timestamp + - Write a summary of the results to `/tmp/gh-aw/agent/smoke-claude-status-${{ github.run_id }}.txt` (create directory if needed) + - Use bash to verify the file was created and display its contents ## Output -1. **Create an issue** with a summary of the smoke test run: +**CRITICAL: You MUST create an issue regardless of test results - this is a required safe output.** + +1. **ALWAYS create an issue** with a summary of the smoke test run: - Title: "Smoke Test: Claude - ${{ github.run_id }}" - Body should include: - Test results (✅ or ❌ for each test) - Overall status: PASS or FAIL - Run URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} - Timestamp + - If ANY test fails, include error details in the issue body + - This issue MUST be created before any other safe output operations 2. **Only if this workflow was triggered by a pull_request event**: Use the `add_comment` tool to add a **very brief** comment (max 5-10 lines) to the triggering pull request (omit the `item_number` parameter to auto-target the triggering PR) with: - PR titles only (no descriptions) @@ -112,5 +117,6 @@ timeout-minutes: 15 - Overall status: PASS or FAIL 3. Use the `add_comment` tool with `item_number` set to the discussion number you extracted in step 9 to add a **fun comic-book style comment** to that discussion - be playful and use comic-book language like "💥 WHOOSH!" + - If step 9 failed to extract a discussion number, skip this step If all tests pass, use the `add_labels` tool to add the label `smoke-claude` to the pull request (omit the `item_number` parameter to auto-target the triggering PR if this workflow was triggered by a pull_request event). diff --git a/docs/src/content/docs/agent-factory-status.mdx b/docs/src/content/docs/agent-factory-status.mdx index 9ded2e3614..28473d868d 100644 --- a/docs/src/content/docs/agent-factory-status.mdx +++ b/docs/src/content/docs/agent-factory-status.mdx @@ -121,7 +121,6 @@ These are experimental agentic workflows used by the GitHub Next team to learn, | [Schema Consistency Checker](https://github.com/github/gh-aw/blob/main/.github/workflows/schema-consistency-checker.md) | claude | [![Schema Consistency Checker](https://github.com/github/gh-aw/actions/workflows/schema-consistency-checker.lock.yml/badge.svg)](https://github.com/github/gh-aw/actions/workflows/schema-consistency-checker.lock.yml) | - | - | | [Scout](https://github.com/github/gh-aw/blob/main/.github/workflows/scout.md) | claude | [![Scout](https://github.com/github/gh-aw/actions/workflows/scout.lock.yml/badge.svg)](https://github.com/github/gh-aw/actions/workflows/scout.lock.yml) | - | `/scout` | | [Security Compliance Campaign](https://github.com/github/gh-aw/blob/main/.github/workflows/security-compliance.md) | copilot | [![Security Compliance Campaign](https://github.com/github/gh-aw/actions/workflows/security-compliance.lock.yml/badge.svg)](https://github.com/github/gh-aw/actions/workflows/security-compliance.lock.yml) | - | - | -| [Security Fix PR](https://github.com/github/gh-aw/blob/main/.github/workflows/security-fix-pr.md) | copilot | [![Security Fix PR](https://github.com/github/gh-aw/actions/workflows/security-fix-pr.lock.yml/badge.svg)](https://github.com/github/gh-aw/actions/workflows/security-fix-pr.lock.yml) | - | - | | [Security Guard Agent 🛡️](https://github.com/github/gh-aw/blob/main/.github/workflows/security-guard.md) | copilot | [![Security Guard Agent 🛡️](https://github.com/github/gh-aw/actions/workflows/security-guard.lock.yml/badge.svg)](https://github.com/github/gh-aw/actions/workflows/security-guard.lock.yml) | - | - | | [Security Review Agent 🔒](https://github.com/github/gh-aw/blob/main/.github/workflows/security-review.md) | copilot | [![Security Review Agent 🔒](https://github.com/github/gh-aw/actions/workflows/security-review.lock.yml/badge.svg)](https://github.com/github/gh-aw/actions/workflows/security-review.lock.yml) | - | `/security` | | [Semantic Function Refactoring](https://github.com/github/gh-aw/blob/main/.github/workflows/semantic-function-refactor.md) | claude | [![Semantic Function Refactoring](https://github.com/github/gh-aw/actions/workflows/semantic-function-refactor.lock.yml/badge.svg)](https://github.com/github/gh-aw/actions/workflows/semantic-function-refactor.lock.yml) | - | - |