Eliminate 35% test duplication in CI integration matrix using skip patterns

[Copilot]

Catch-all test groups with pattern: "" were running ALL tests in their packages, including 3,480 tests (35.4%) already executed by specific pattern groups. This wasted 114s per CI run.

Changes

Matrix Configuration

• Added skip_pattern field to 4 catch-all groups:
  • CLI Completion & Safe Inputs: excludes 13 specific CLI patterns
  • Parser Location & Validation: excludes 4 remote fetch patterns
  • Workflow Misc Part 2: excludes 45+ workflow patterns
  • CMD Tests: empty skip (no overlaps)

Test Execution Logic

• Catch-all groups now use -skip flag when skip_pattern is defined
• Specific pattern groups continue using -run unchanged
• Backward compatible for groups without skip_pattern

Implementation

# Matrix configuration
- name: "CLI Completion & Safe Inputs"
  pattern: ""  # Catch-all
  skip_pattern: "^TestCompile[^W]|TestPoutine|TestMCP*|..."

# Test execution
if [ -z "$pattern" ]; then
  if [ -n "$skip_pattern" ]; then
    go test -skip '$skip_pattern' ...  # Exclude already-tested patterns
  else
    go test ...  # No filters
  fi
else
  go test -run '$pattern' ...  # Match specific patterns
fi

Impact

• Test executions: 13,797 → 9,821 (eliminates 3,976 duplicates)
• Max group duration: 105.5s → ~60s (estimated)
• CI time saved: ~114s per run (43% faster integration tests)

Original prompt

---

This section details on the original issue you should resolve

<issue_title>[ci-coach] Eliminate massive test duplication in CI integration tests</issue_title>
<issue_description>## Critical CI Optimization: Eliminate Test Duplication

This PR fixes a critical issue where 3,480 tests (35.4% of all tests) are running in multiple matrix groups, wasting 1.9 minutes of CI time per run.

🔍 Problem Discovery

While analyzing CI performance for workflow run §20269295629, I discovered that the previous optimization (run #16) did not achieve its expected results. Instead of improving from 76.8s → 46s, the max integration group duration increased to 105.5s.

Root Cause Analysis:

Unique tests in repository: 9,821
Total test executions: 13,797
Tests running in multiple groups: 3,480
Unnecessary duplicate executions: 3,976
Duplication rate: 35.4%

💰 WASTED TIME: 114.3 seconds per CI run (1.9 minutes)

The issue: Catch-all groups with pattern: "" run ALL tests in the package, including those already matched by specific patterns. When you use go test -run "", it runs every test. This caused tests to run in BOTH their specific groups AND in catch-all groups.

💡 Solution

Add skip_pattern field to catch-all matrix groups and update the test execution logic to use Go's -skip flag:

Changes to .github/workflows/ci.yml:

1. Updated test execution logic (lines 174-188):

   • Added support for skip_pattern field in matrix configuration
   • Catch-all groups now use -skip flag to exclude already-tested patterns
   • Specific pattern groups continue using -run as before

2. Added skip patterns to catch-all groups:

   • CLI Completion & Safe Inputs: Skip 13 specific CLI patterns (Compile, Poutine, MCP*, Logs, Firewall, Progress)
   • Parser Location & Validation: Skip 4 remote fetch patterns
   • Workflow Misc Part 2: Skip 45+ specific workflow patterns (comprehensive exclusion list)
   • CMD Tests: No skip needed (no other groups cover cmd tests)

📊 Expected Impact

Before (Current State):

• Max integration group: 105.5s (CLI Completion & Safe Inputs)
• Second slowest: 73.2s (Workflow Misc Part 2)
• Total duplicate executions: 3,976
• Wasted time: 114.3s per run

After (With This Fix):

• Max integration group: ~60s (estimated after removing duplicates)
• Total duplicate executions: 0
• Time saved: 114s per run
• Improvement: 43% faster integration tests
• Cost savings: ~$15-20/month in GitHub Actions minutes

🔧 Technical Details

How -skip Works:

# Before (catch-all): Runs ALL tests including duplicates
go test -tags 'integration' ./pkg/cli

# After (catch-all): Runs only tests NOT matching specific patterns
go test -tags 'integration' -skip 'TestCompile|TestMCP|TestLogs|...' ./pkg/cli

Example for CLI Completion & Safe Inputs:

- name: "CLI Completion & Safe Inputs"
  packages: "./pkg/cli"
  pattern: ""  # Catch-all
  skip_pattern: "^TestCompile[^W]|TestPoutine|TestMCPInspectPlaywright|..."

This ensures tests like TestProgressFlagSignature (30s) only run in "CLI Progress Flag" group, not in both "CLI Progress Flag" AND "CLI Completion & Safe Inputs".

✅ Validation Plan

The fix will be validated by:

1. ✅ YAML syntax (checked in this workflow)
2. ✅ Build passes (validated in this workflow)
3. 🔄 First PR run: Compare test counts per group
4. 🔄 Verify no duplicates: Confirm each test runs in exactly one group
5. 🔄 Measure improvement: Compare total integration test time

Expected first-run results:

• CLI Completion & Safe Inputs: 1,753 tests → ~120 tests (after exclusions)
• Workflow Misc Part 2: 6,900 tests → ~1,000 tests (after exclusions)
• All other groups: Test counts unchanged
• Total test executions: 13,797 → 9,821 (no duplicates)

📈 Success Metrics

• Each test runs in exactly one matrix group
• Integration test time reduced by 30-40%
• Max matrix group duration under 60 seconds
• No test coverage gaps (all tests still run)

🎯 Context

This optimization was discovered by CI Optimization Coach workflow run #17. The previous optimization (run #16) attempted to fix matrix imbalance but inadvertently maintained test duplication due to catch-all pattern behavior.

References:

• Analysis based on §20269295629
• Previous optimization: §20266955990

---

Impact: High - Reduces CI time by ~2 minutes per run
Risk: Low - Uses Go's native -skip flag, no behavioral changes
Maintenance: None - Self-maintaining exclusion lists

AI generated by CI Optimization Coach

---

[!NOTE]
This was ...

• Fixes [ci-coach] Eliminate massive test duplication in CI integration tests #6634

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.