Add automated Claude workflow for test failure analysis

[Copilot]

When PR tests fail, automatically trigger Claude to analyze failures and post diagnostic comments with suggested fixes.

Changes

• New workflow .github/workflows/claude-on-test-failure.yml

  • Triggers on workflow_run: completed for "Run Tests" workflow
  • Runs only when conclusion == 'failure'
  • Waits for all test jobs to complete before analyzing
  • Uses GitHub MCP tools to fetch workflow logs and failure details
  • Posts structured PR comment with root cause and actionable solution
  • Concurrency controls: Only one analysis runs at a time per branch, with older jobs automatically cancelled
  • Loop detection: Claude checks for bot comment loops (e.g., CodeRabbit ↔ Claude cycles) and exits with a warning if detected

• Documentation updated in AGENTS.md

Workflow Behavior

When tests fail, Claude:

1. Fetches failed job logs via get_job_logs and summarize_run_log_failures
2. Analyzes failures against codebase context
3. Checks PR comments for bot loops before posting
4. Posts comment with:
   • Failure summary
   • Root cause analysis
   • Specific file changes needed
   • Detailed logs in collapsible sections

Follows existing Claude workflow patterns (same MCP servers, similar prompt structure, consistent tool allowlist).

Safety Features

• Concurrency management: cancel-in-progress: true ensures efficient resource usage by canceling superseded analysis runs
• Loop prevention: Explicit instructions for Claude to detect and break bot interaction loops

<issue_title>Add Claude Workflow for failed tests</issue_title>
><issue_description>When a PR check fails, we should trigger Claude automatically to assess the failure and post a comment with what it thinks the solution to the failure is.
>
> Can we have it wait until all checks on the PR report success or failure?
>
></issue_description>
>
> ## Comments on the Issue (you are @copilot in this section)
>
>
><comment_new>@strawgate
> @claude please make a pull request</comment_new>
>
>

• Fixes Add Claude Workflow for failed tests #195

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Add Claude Workflow for failed tests</issue_title>
<issue_description>When a PR check fails, we should trigger Claude automatically to assess the failure and post a comment with what it thinks the solution to the failure is.

Can we have it wait until all checks on the PR report success or failure?

</issue_description>

Comments on the Issue (you are @copilot in this section)

@strawgate @claude please make a pull request

• Fixes Add Claude Workflow for failed tests #195

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[coderabbitai]

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch copilot/add-claude-workflow-for-tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[sonarqubecloud]

Please retry analysis of this Pull-Request directly on SonarQube Cloud

[strawgate]

@copilot i think we want to set concurrency controls here so that older jobs get cancelled, only one runs at a time, etc. We also should make it very clear in the prompt that if it detects a loop (code rabbit triggers claude triggers code rabbit triggers claude sorta thing) it must exist.

[Copilot]

@copilot i think we want to set concurrency controls here so that older jobs get cancelled, only one runs at a time, etc. We also should make it very clear in the prompt that if it detects a loop (cod...

Added in commit 914a4de:

1. Concurrency controls: Added concurrency group with cancel-in-progress: true to ensure only one analysis runs at a time per branch, canceling older jobs automatically.

2. Loop detection: Added a critical section to the prompt instructing Claude to:

   • Check PR comments for bot comment loops before posting
   • Detect if CodeRabbit/other bots triggered a cycle
   • Exit immediately with a brief warning comment if a loop is detected
   • Not post another full analysis if loops are detected

[sonarqubecloud]

Quality Gate failed

Failed conditions
3 Security Hotspots

See analysis details on SonarQube Cloud