From 1f297a28291ca0dcd729b15a0658b201f3bed791 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 23 Jan 2026 19:54:12 +0000 Subject: [PATCH 1/2] Initial plan From 941469ffe07762a7720726346ef98b94be25555d Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 23 Jan 2026 20:05:28 +0000 Subject: [PATCH 2/2] Add report formatting guidelines to agent-performance-analyzer workflow Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com> --- .../agent-performance-analyzer.lock.yml | 140 +++++++++++++++++- .../workflows/agent-performance-analyzer.md | 64 ++++++++ 2 files changed, 202 insertions(+), 2 deletions(-) diff --git a/.github/workflows/agent-performance-analyzer.lock.yml b/.github/workflows/agent-performance-analyzer.lock.yml index 6aabf2ca0b..20fe03f86d 100644 --- a/.github/workflows/agent-performance-analyzer.lock.yml +++ b/.github/workflows/agent-performance-analyzer.lock.yml @@ -20,6 +20,10 @@ # For more information: https://github.com/githubnext/gh-aw/blob/main/.github/aw/github-agentic-workflows.md # # Meta-orchestrator that analyzes AI agent performance, quality, and effectiveness across the repository +# +# Resolved workflow manifest: +# Imports: +# - shared/reporting.md name: "Agent Performance Analyzer - Meta-Orchestrator" "on": @@ -709,6 +713,76 @@ jobs: PROMPT_EOF cat << 'PROMPT_EOF' >> "$GH_AW_PROMPT" + ## Report Structure Guidelines + + ### 1. Header Levels + **Use h3 (###) or lower for all headers in your issue report to maintain proper document hierarchy.** + + When creating GitHub issues or discussions: + - Use `###` (h3) for main sections (e.g., "### Test Summary") + - Use `####` (h4) for subsections (e.g., "#### Device-Specific Results") + - Never use `##` (h2) or `#` (h1) in reports - these are reserved for titles + + ### 2. Progressive Disclosure + **Wrap detailed test results in `
Section Name` tags to improve readability and reduce scrolling.** + + Use collapsible sections for: + - Verbose details (full test logs, raw data) + - Secondary information (minor warnings, extra context) + - Per-item breakdowns when there are many items + + Always keep critical information visible (summary, critical issues, key metrics). + + ### 3. Report Structure Pattern + + 1. **Overview**: 1-2 paragraphs summarizing key findings + 2. **Critical Information**: Show immediately (summary stats, critical issues) + 3. **Details**: Use `
Section Name` for expanded content + 4. **Context**: Add helpful metadata (workflow run, date, trigger) + + ### Design Principles (Airbnb-Inspired) + + Reports should: + - **Build trust through clarity**: Most important info immediately visible + - **Exceed expectations**: Add helpful context like trends, comparisons + - **Create delight**: Use progressive disclosure to reduce overwhelm + - **Maintain consistency**: Follow patterns across all reports + + ### Example Report Structure + + ```markdown + ### Summary + - Key metric 1: value + - Key metric 2: value + - Status: ✅/⚠️/❌ + + ### Critical Issues + [Always visible - these are important] + +
+ View Detailed Results + + [Comprehensive details, logs, traces] + +
+ +
+ View All Warnings + + [Minor issues and potential problems] + +
+ + ### Recommendations + [Actionable next steps - keep visible] + ``` + + ## Workflow Run References + + - Format run IDs as links: `[§12345](https://github.com/owner/repo/actions/runs/12345)` + - Include up to 3 most relevant run URLs at end under `**References:**` + - Do NOT add footer attribution (system adds automatically) + {{#runtime-import? .github/shared-instructions.md}} # Agent Performance Analyzer - Meta-Orchestrator @@ -719,6 +793,68 @@ jobs: As a meta-orchestrator for agent performance, you assess how well AI agents are performing their tasks, identify patterns in agent behavior, detect quality issues, and recommend improvements to the agent ecosystem. + ## Report Formatting Guidelines + + When creating performance reports as issues or discussions: + + **1. Header Levels** + - Use h3 (###) or lower for all headers in your reports to maintain proper document hierarchy + - Never use h2 (##) or h1 (#) in report bodies - these are reserved for titles + + **2. Progressive Disclosure** + - Wrap detailed analysis sections in `
Section Name` tags to improve readability and reduce scrolling + - Always keep critical findings visible (quality issues, failing agents, urgent recommendations) + - Use collapsible sections for: + - Full performance metrics tables + - Agent-by-agent detailed breakdowns + - Historical trend charts + - Comprehensive quality analysis + - Detailed effectiveness metrics + + **3. Report Structure Pattern** + + Follow this structure for performance reports: + + ```markdown + ### Performance Summary + - Total agents analyzed: [N] + - Overall effectiveness score: [X%] + - Critical issues found: [N] + + ### Critical Findings + [Always visible - quality issues, failing agents, urgent recommendations] + +
+ View Detailed Quality Analysis + + [Full quality metrics, agent-by-agent scores, trend charts] + +
+ +
+ View Effectiveness Metrics + + [Task completion rates, decision quality, resource efficiency tables] + +
+ +
+ View Behavioral Patterns + + [Detailed pattern analysis, collaboration metrics, coverage gaps] + +
+ + ### Recommendations + [Actionable next steps - keep visible] + ``` + + **Design Principles** + - **Build trust through clarity**: Most important findings (critical issues, overall health) immediately visible + - **Exceed expectations**: Add helpful context like trend comparisons, historical performance + - **Create delight**: Use progressive disclosure to present complex data without overwhelming + - **Maintain consistency**: Follow the same patterns as other meta-orchestrator reports + ## Responsibilities ### 1. Agent Output Quality Analysis @@ -1043,6 +1179,8 @@ jobs: - Efficient resource usage - Example outputs: #123, #456, #789 + PROMPT_EOF + cat << 'PROMPT_EOF' >> "$GH_AW_PROMPT" 2. **Agent Name 2** (Quality: 90/100, Effectiveness: 88/100) - Clear, well-documented outputs - Good collaboration with other agents @@ -1182,8 +1320,6 @@ jobs: - Overall agent quality: XX/100 (↑ +5 from last week) - Average effectiveness: XX/100 (→ stable) - Output volume: XXX outputs (↑ +10% from last week) - PROMPT_EOF - cat << 'PROMPT_EOF' >> "$GH_AW_PROMPT" - PR merge rate: XX% (↑ +3% from last week) - Resource efficiency: XX min average (↓ -2 min from last week) diff --git a/.github/workflows/agent-performance-analyzer.md b/.github/workflows/agent-performance-analyzer.md index 59117af04e..7253c910ba 100644 --- a/.github/workflows/agent-performance-analyzer.md +++ b/.github/workflows/agent-performance-analyzer.md @@ -16,6 +16,8 @@ tools: branch-name: memory/meta-orchestrators file-glob: "**" max-file-size: 102400 # 100KB +imports: + - shared/reporting.md safe-outputs: create-issue: max: 5 @@ -38,6 +40,68 @@ You are an AI agent performance analyst responsible for evaluating the quality, As a meta-orchestrator for agent performance, you assess how well AI agents are performing their tasks, identify patterns in agent behavior, detect quality issues, and recommend improvements to the agent ecosystem. +## Report Formatting Guidelines + +When creating performance reports as issues or discussions: + +**1. Header Levels** +- Use h3 (###) or lower for all headers in your reports to maintain proper document hierarchy +- Never use h2 (##) or h1 (#) in report bodies - these are reserved for titles + +**2. Progressive Disclosure** +- Wrap detailed analysis sections in `
Section Name` tags to improve readability and reduce scrolling +- Always keep critical findings visible (quality issues, failing agents, urgent recommendations) +- Use collapsible sections for: + - Full performance metrics tables + - Agent-by-agent detailed breakdowns + - Historical trend charts + - Comprehensive quality analysis + - Detailed effectiveness metrics + +**3. Report Structure Pattern** + +Follow this structure for performance reports: + +```markdown +### Performance Summary +- Total agents analyzed: [N] +- Overall effectiveness score: [X%] +- Critical issues found: [N] + +### Critical Findings +[Always visible - quality issues, failing agents, urgent recommendations] + +
+View Detailed Quality Analysis + +[Full quality metrics, agent-by-agent scores, trend charts] + +
+ +
+View Effectiveness Metrics + +[Task completion rates, decision quality, resource efficiency tables] + +
+ +
+View Behavioral Patterns + +[Detailed pattern analysis, collaboration metrics, coverage gaps] + +
+ +### Recommendations +[Actionable next steps - keep visible] +``` + +**Design Principles** +- **Build trust through clarity**: Most important findings (critical issues, overall health) immediately visible +- **Exceed expectations**: Add helpful context like trend comparisons, historical performance +- **Create delight**: Use progressive disclosure to present complex data without overwhelming +- **Maintain consistency**: Follow the same patterns as other meta-orchestrator reports + ## Responsibilities ### 1. Agent Output Quality Analysis