Skip to content

Investigate Documentation PR 0% Merge Rate (47 PRs closed without merge) #15727

@github-actions

Description

@github-actions

Problem Statement

In the past 7 days (February 7-14, 2026), documentation agents created 47 pull requests with a 0% merge rate - all PRs were closed without merging. This represents a significant decline from the previous period's 70% merge rate.

Affected Workflows:

  • Daily Doc Updater
  • Instructions Janitor
  • Documentation Unbloat
  • Workflow Normalizer

Evidence

From Agent Performance Analysis (§22021394730):

| Metric | Current (Feb 7-14) | Previous | Change |
|--------|-------------------|----------|--------|
| PRs Created | 47 | N/A | High volume |
| PR Merge Rate | 0% | 70% | ↓ -70% CRITICAL |

Sample Closed PRs (all without merge):

Hypotheses for Root Cause

  1. Timing: PRs too recent to be reviewed (need more time)
  2. Manual superseding: Maintainers making manual fixes before reviewing agent PRs
  3. Quality issues: PRs not meeting unstated quality requirements
  4. Alignment problems: Changes not aligning with maintainer priorities or vision
  5. Process issues: Wrong workflow for documentation changes (should be issues/discussions?)

Investigation Required

Phase 1: Data Collection (2 hours)

  1. Review all 47 closed PRs:

    • Read close comments and reasons
    • Identify patterns in closure (timing, quality, maintainer feedback)
    • Check if any were superseded by manual commits
    • Note any common feedback themes
  2. Interview/survey maintainers:

    • Ask about documentation PR preferences
    • Understand what makes a good vs. bad documentation PR
    • Clarify unstated quality requirements
    • Understand review process and timing expectations
  3. Analyze successful PRs (previous period):

    • Review the 70% that merged in previous period
    • Identify characteristics of successful PRs
    • Compare to current failed PRs
    • Extract success patterns

Phase 2: Pattern Identification (1 hour)

  1. Categorize closure reasons:

    • Timing (not yet reviewed)
    • Quality (formatting, accuracy, completeness)
    • Alignment (wrong changes, unnecessary)
    • Process (wrong approach)
    • Superseded (manual fix applied)
  2. Quantify each category:

    • Count PRs in each bucket
    • Identify dominant patterns
    • Look for workflow-specific issues

Phase 3: Recommendations (1 hour)

Based on findings, develop specific recommendations for:

  1. Documentation agent prompts (what to change)
  2. PR creation criteria (when to create vs. not create)
  3. PR quality gates (checks before creation)
  4. Alternative approaches (discussions first, issues instead of PRs)
  5. Process changes (timing, review flow)

Expected Outcomes

  1. Root cause identified: Clear understanding of why PRs are being closed
  2. Specific improvements: Actionable changes to agent configuration
  3. Quality gates: Criteria for creating high-quality documentation PRs
  4. Success metrics: Target merge rate and quality indicators
  5. Implementation plan: Step-by-step changes to affected workflows

Success Criteria

  • All 47 closed PRs reviewed and categorized
  • Dominant closure pattern identified with data
  • Maintainer preferences documented
  • Agent prompt improvements drafted
  • PR quality gates defined
  • Target: Increase merge rate from 0% to 40-50% within 2 weeks

Priority: High

Impact: Medium-High - Affects 4 workflows, represents wasted agent effort, degrades ecosystem effectiveness

Effort: 4 hours investigation + 2-4 hours implementation

Timeline: Investigation complete within 48 hours, improvements deployed within 1 week


📊 Related Report: Agent Performance Report - Week of February 7-14, 2026
🔗 Workflow Run: §22021394730

Generated by Agent Performance Analyzer - Meta-Orchestrator

  • expires on Feb 16, 2026, 5:36 PM UTC

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions