Skip to content

Refactor Dev Hawk from PR monitor to workflow run inspector#7194

Closed
Copilot wants to merge 2 commits intomainfrom
copilot/update-hawk-agent-workflow
Closed

Refactor Dev Hawk from PR monitor to workflow run inspector#7194
Copilot wants to merge 2 commits intomainfrom
copilot/update-hawk-agent-workflow

Conversation

Copy link
Contributor

Copilot AI commented Dec 22, 2025

Dev Hawk was analyzing PR diffs and correlating errors with code changes. Now it inspects workflow run logs independently, focusing on error patterns and root causes from log evidence only.

Changes

Workflow Configuration

  • Updated description to reflect inspection role
  • Removed bash tool (no longer creating agent tasks)
  • Updated safe-output messages

Inspection Process

  • Removed: PR diff analysis, file change correlation, agent task creation
  • Added: Systematic error categorization (Build/Test/Lint/Runtime/Infrastructure/Timeout/Tool), anomaly detection (timing, resources, flaky behavior), root cause tracing from logs
  • Implemented 10-step inspection methodology

Reporting

  • New structured comment templates with:
    • Error categorization and severity
    • Job/Step breakdown with failures
    • Anomalies detected section
    • Performance insights
    • Recommendations (not fix instructions)
  • Added explicit "What NOT to Do" guidelines forbidding code analysis

Example Comment Structure

# ⚠️ Dev Hawk Inspection - Failure Analysis

## 🔍 Inspection Findings

### Error Summary
Build failure in compilation step

### Root Cause Analysis
**Error Category**: Build
**What Failed**: `make build` step in main job
**Why It Failed**: Go compilation error - undefined reference

**Key Error Messages**:

./pkg/workflow/compiler.go:123: undefined: validateStrictMode


### Detailed Findings
#### Job/Step Breakdown
- **Build**: Failed (2m 34s)
  - Error: Compilation failed at line 123
  - Impact: Blocks all downstream jobs

### Performance Insights
- Total Duration: 2m 45s
- Failed At: 2m 34s (main job)

The workflow now answers "What failed in the logs?" rather than "Which code change caused this?"

Original prompt

Update dev Hawk agentic workflow prompt to behave as a workflow run inspector that looks for errors, anomalies, issues in the run log and provides insights on the root cause.

The goal is dev hawk is NOT to summarize the PR.

Custom agent used: create-agentic-workflow
Design agentic workflows using GitHub Agentic Workflows (gh-aw) extension with interactive guidance on triggers, tools, and security best practices.


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Update dev Hawk agentic workflow to inspect runs for issues Refactor Dev Hawk from PR monitor to workflow run inspector Dec 22, 2025
Copilot AI requested a review from pelikhan December 22, 2025 05:10
@pelikhan pelikhan closed this Dec 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants