Skip to content

🚨 URGENT: PR Merge Crisis Week 2 - 0% Merge Rate Despite 97% Quality #10788

@github-actions

Description

@github-actions

Problem Statement

CRITICAL: Week 2 of complete PR merge blockage despite excellent agent quality.

The agent ecosystem is producing high-quality pull requests that are not being merged, resulting in:

  • 0% PR merge rate (0 out of 100 PRs merged in last 7 days)
  • 97% PR quality (6.8/7 average quality score)
  • Zero code contributions reaching main branch from agents
  • Agent effectiveness at 8/100 despite high activity

Evidence

By the Numbers

  • 93 PRs created by agents (Copilot SWE Agent) in last 7 days
  • 0 merged (0% merge rate)
  • 96 closed without merge
  • 4 remain open

By Category (All 0% Merge Rate)

  • Bugfix: 32 PRs
  • Other: 32 PRs
  • Feature: 25 PRs
  • Maintenance: Recompile: 5 PRs
  • Security: 4 PRs
  • Testing: 1 PR
  • Documentation: 1 PR

Quality Indicators

PR quality is NOT the problem:

  • Average quality score: 6.8/7 (97%)
  • 100% have descriptions (20/20 sampled)
  • 95% have structured sections (19/20)
  • 90% include code examples or tests (18/20)
  • 85% have clear context and rationale (17/20)

Example high-quality PRs:

Impact

Agent Ecosystem

  • Complete value delivery breakdown - agents producing zero impact
  • Wasted agent effort - high-quality work not reaching main
  • Campaign effectiveness at 0% - all code-contributing campaigns blocked
  • Quality/effectiveness divergence - Quality ⬆️ +33 points, Effectiveness ⬇️ -2 points

Development Velocity

  • Bug fixes not being merged
  • Security fixes not being merged (4 security PRs blocked)
  • Feature improvements not being merged
  • Maintenance updates not being merged

Root Cause Analysis

NOT an agent quality problem - PR quality is excellent (97%).

Likely a process/approval bottleneck. Potential causes:

  1. Maintainer bandwidth?

    • Not enough reviewers for volume of agent PRs?
    • Single maintainer bottleneck?
    • Review queue backlog?
  2. CI/test failures?

    • Are agent PRs failing tests invisibly?
    • Are there undocumented test requirements?
    • Is CI blocking merges without clear signals?
  3. Undocumented merge criteria?

    • Are there requirements agents aren't meeting?
    • Missing labels or approvals?
    • Undocumented review process?
  4. Policy or freeze?

    • Is there a feature freeze in effect?
    • Policy restriction on agent PRs?
    • Requires human approval that's not happening?

Investigation Plan

Phase 1: Identify Bottleneck (2-3 hours)

  1. Interview maintainers

    • Why aren't agent PRs being reviewed?
    • What's the review process for agent PRs?
    • What would make agent PRs auto-mergeable?
  2. Analyze PR status

    • Check CI/test status on open agent PRs
    • Review closed-without-merge reasons
    • Identify common blockers
  3. Review merge criteria

    • Document current requirements
    • Identify gaps in agent PR metadata
    • Check for missing approvals or labels

Phase 2: Quick Wins (1-2 hours)

  1. Identify auto-mergeable categories

    • Which PR types are safe to auto-merge?
    • Define safety criteria (tests pass, no conflicts, etc.)
    • Recompile workflows? Documentation updates?
  2. Unblock obvious PRs

    • Merge any PRs that meet all criteria
    • Close any PRs that are truly invalid
    • Provide feedback on blocked PRs

Phase 3: Long-term Solution (8-16 hours)

  1. Create PR triage workflow

    • Auto-assign reviewers based on file changes
    • Label PRs by category and complexity
    • Flag PRs ready for auto-merge
    • Track time-in-review metrics
  2. Establish auto-merge criteria

    • Define which categories can auto-merge
    • Implement safety checks (tests, conflicts, etc.)
    • Document process for agents
  3. Monitor and iterate

    • Track PR merge rate improvement
    • Adjust criteria based on results
    • Target: 50-80% merge rate

Success Metrics

Target outcomes:

  • PR merge rate: 50-80% (healthy ecosystem level)
  • Time to merge: <48 hours for auto-mergeable categories
  • Agent effectiveness: 50-80/100 (up from 8/100)
  • Zero value delivery gap between quality and effectiveness

Measurement:

  • Track daily PR merge rate
  • Monitor time-to-merge by category
  • Measure agent effectiveness scores
  • Survey maintainer satisfaction

Timeline

  • Phase 1 (Investigation): 2-3 hours
  • Phase 2 (Quick wins): 1-2 hours
  • Phase 3 (Long-term): 8-16 hours
  • Total: 11-21 hours over 3-5 days

Priority Justification

P0 (Critical) because:

  1. Week 2 of zero value delivery from agent ecosystem
  2. 97% quality wasted due to process issue
  3. All code-contributing campaigns blocked
  4. Security fixes not reaching production (4 security PRs)
  5. Agent effectiveness collapsed to 8/100

This is the #1 blocker for agent ecosystem value delivery.

Related Issues

Recommended Owners

  • Investigation: @pelikhan or repository maintainers
  • PR Triage Workflow: Development team
  • Auto-merge Criteria: Repository administrators
  • Monitoring: Agent Performance Analyzer (automated)

AI generated by Agent Performance Analyzer - Meta-Orchestrator

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions