Add error aggregation for validation using errors.Join by Copilot · Pull Request #12661 · github/gh-aw

Copilot · 2026-01-30T02:46:51Z

Validation errors previously stopped at first failure, requiring multiple compile cycles to discover all issues. Now collects and reports all errors in a single pass.

Changes

Core Infrastructure

ErrorCollector utility with fail-fast support via errors.Join
--fail-fast CLI flag to restore legacy behavior (default: aggregate)
FormattedError() method for accurate multi-error formatting

Updated Validations (8 functions)

validateNetworkAllowedDomains - domain pattern errors
validateSafeOutputsAllowedDomains - safe-outputs domain errors
validateStrictMode - aggregates 5 sub-validations (permissions, network, MCP, tools, deprecated fields)

Example

Before:

✗ strict mode: write permission 'contents: write' is not allowed...
# Fix, recompile
✗ strict mode: wildcard '*' is not allowed in network.allowed...
# Fix again, recompile

After:

✗ Found 2 strict mode errors:
  • strict mode: write permission 'contents: write' is not allowed...
  • strict mode: wildcard '*' is not allowed in network.allowed...
# Fix all, done

Backward Compatibility

Use --fail-fast to stop at first error (legacy behavior).

Original prompt

This section details on the original issue you should resolve

<issue_title>[Code Quality] Add error aggregation using errors.Join throughout validation code</issue_title>
<issue_description>### Description

Implement error aggregation throughout the codebase so users see all validation errors, configuration issues, and workflow problems in one report instead of discovering them one at a time through multiple re-runs.

Current Behavior

First error stops validation/compilation

Users fix error, re-run, discover next error

Requires multiple iterations to fix all issues

Poor developer experience for workflows with multiple problems

Desired Behavior

All validation errors collected and reported together

Clear indication of total error count and severity

Errors grouped by category (validation, configuration, permissions)

Users fix all issues in one iteration

Files Affected

pkg/workflow/*_validation.go - Core validation files

pkg/parser/frontmatter_parser.go - Frontmatter validation

pkg/cli/compile_command.go - Compilation error handling

Suggested Changes

Identify validation/compilation code paths for aggregation

Replace early returns with error collection using errors.Join

Update error messages to show "X errors found" with full list

Group errors by category for better readability

Maintain backward compatibility (exit code 1 if any errors)

Add --fail-fast flag to restore old behavior if needed

Update at least 5 high-traffic validation functions

Success Criteria

At least 5 validation functions aggregate errors

Users see all errors in single run

Errors grouped by category (validation, config, permissions)

Total error count displayed prominently

Exit code 1 maintained when errors exist

--fail-fast flag implemented for legacy behavior

Unit tests verify error aggregation

Validation guide updated

All tests pass (make test-unit)

Example Output
❌ Found 3 validation errors in workflow.md:

Validation Errors (2):
  • Line 5: Invalid engine 'copilot-pro' (valid: copilot, claude, codex, custom)
  • Line 12: Required field 'tools.github' missing when using safe-outputs

Configuration Errors (1):
  • MCP server 'playwright' requires network.allowed_domains configuration

Fix all errors above and re-run compilation.
Implementation Strategy

Phase 1: Update pkg/constants/ validation (smallest scope)

Phase 2: Update pkg/cli/ validation

Phase 3: Update pkg/workflow/ validation

Phase 4: Update remaining packages

Run make agent-finish after each phase.

Priority

Medium-High - Significantly improves developer experience, reduces iteration cycles

Source

Extracted from Repository Quality Improvement Report - Workflow Health Monitoring githubnext/gh-aw#12540 - Task 3

AI generated by Discussion Task Miner - Code Quality Improvement Agent

expires on Feb 12, 2026, 5:15 PM UTC

Comments on the Issue (you are @copilot in this section)

Fixes [Code Quality] Add error aggregation using errors.Join throughout validation code #12598

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Changeset

Type: patch
Description: Aggregate validation errors so compilation reports all issues together and add --fail-fast for legacy fail-fast behavior.

Ahoy! This treasure was crafted by 🏴‍☠️ Changeset Generator

- Added ErrorCollector utility for collecting multiple validation errors - Added --fail-fast flag to compile command to control error aggregation - Updated validateNetworkAllowedDomains and validateSafeOutputsAllowedDomains to use error aggregation - Added failFast field to Compiler struct - All error aggregation tests passing Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

- Updated validateStrictMode to collect all validation errors before failing - Added FormatAggregatedError to display error count and grouped messages - All strict mode validation tests passing Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

- Changed assert.NoError to require.NoError for error assertions - Changed assert.Error to require.Error for error assertions - All linting checks now pass Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

- Fixed usage example to properly handle fail-fast mode return values - Added FormattedError() method to ErrorCollector for accurate error counting - Updated strict mode validation to use FormattedError() - Added comprehensive tests for FormattedError() - Fixed staticcheck lint issue (use fmt.Fprintf instead of WriteString+Sprintf) - All linting and tests pass Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

github-actions · 2026-01-30T04:00:14Z

🎉 Yo ho ho! Changeset Generator found the treasure and completed successfully! ⚓💰

github-actions · 2026-01-30T04:00:18Z

🎬 THE END — Smoke Claude MISSION: ACCOMPLISHED! The hero saves the day! ✨

github-actions · 2026-01-30T04:00:19Z

📰 BREAKING: Smoke Copilot is now investigating this pull request. Sources say the story is developing...

github-actions · 2026-01-30T04:00:19Z

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

github-actions · 2026-01-30T04:02:43Z

Agent Container Tool Check

Tool	Status	Version
bash	✅	5.2.21
sh	✅	available
git	✅	2.52.0
jq	✅	1.7
yq	✅	4.50.1
curl	✅	8.5.0
gh	✅	2.86.0
node	✅	24.13.0
python3	✅	3.13.11
go	✅	1.24.12
java	❌	not found
dotnet	❌	not found

Result: 10/12 tools available

Status: ⚠️ PARTIAL - Java and .NET runtimes not installed

AI generated by Agent Container Smoke Test

github-actions · 2026-01-30T04:04:40Z

💥 WHOOSH! 🦸

The Claude smoke test agent just blazed through here at lightspeed! ⚡

KAPOW! Validated all systems - GitHub API, Serena MCP, Playwright, Tavily, file ops - everything checks out!

ZAP! Run ID: §21503889451 - All engines firing on all cylinders! 🚀

Up, up, and away! The agent is done here! 🦸‍♂️✨

AI generated by Smoke Claude

github-actions · 2026-01-30T04:04:43Z

Smoke Test: Claude - ✅ PASS

Last 2 Merged PRs:

Fix Go formatting: remove trailing whitespace in frontmatter_hash_stability_test.go #12665: Fix Go formatting
Verify sandbox.mcp.port validation already implemented #12660: Verify sandbox.mcp.port validation

Test Results:

✅ GitHub MCP
✅ Safe Inputs GH CLI
✅ Serena MCP
✅ Make Build
✅ Playwright
✅ Tavily Web Search
✅ File Ops
✅ Bash Tool
✅ Discussion Interaction

Overall Status: PASS

AI generated by Smoke Claude

github-actions · 2026-01-30T04:04:59Z

AI generated by Smoke Codex

github-actions · 2026-01-30T04:04:59Z

🔍 Smoke Test Results - Copilot Engine

✅ GitHub MCP: Last 2 merged PRs reviewed
✅ Safe Inputs GH CLI: PR query successful
✅ Serena MCP: Project activated
✅ Playwright: GitHub navigation verified
✅ File Writing: Test file created
✅ Bash: File verification successful
✅ Discussion Query: Latest discussion retrieved
✅ Build: make build completed successfully

Overall Status: ✅ PASS

§21503889463

AI generated by Smoke Copilot

github-actions · 2026-01-30T04:05:16Z

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

Initial plan

0f03ffe

Copilot AI assigned Copilot and pelikhan Jan 30, 2026

Copilot started work on behalf of pelikhan January 30, 2026 02:47 View session

Copilot AI and others added 4 commits January 30, 2026 02:57

Fix testifylint issues in error aggregation tests

47fbb91

- Changed assert.NoError to require.NoError for error assertions - Changed assert.Error to require.Error for error assertions - All linting checks now pass Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Add error aggregation for validation code using errors.Join~~ Add error aggregation for validation using errors.Join Jan 30, 2026

Copilot AI requested a review from pelikhan January 30, 2026 03:18

Copilot finished work on behalf of pelikhan January 30, 2026 03:18

pelikhan marked this pull request as ready for review January 30, 2026 03:56

pelikhan added the smoke label Jan 30, 2026

Add changeset [skip-ci]

97a5bd9

github-actions bot mentioned this pull request Jan 30, 2026

Smoke Test: Claude - 21503889451 #12666

Closed

github-actions bot added smoke-claude and removed smoke labels Jan 30, 2026

github-actions bot added smoke-codex smoke-copilot labels Jan 30, 2026

github-actions bot mentioned this pull request Jan 30, 2026

Smoke Test: Copilot - 21503889463 #12668

Closed

pelikhan merged commit 61ae385 into main Jan 30, 2026
84 checks passed

pelikhan deleted the copilot/add-error-aggregation-validation branch January 30, 2026 04:12

github-actions bot mentioned this pull request Jan 30, 2026

[team-status] Daily Team Status - Thursday, January 30, 2026 🌟 #12712

Closed

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add error aggregation for validation using errors.Join#12661

Add error aggregation for validation using errors.Join#12661
pelikhan merged 6 commits intomainfrom
copilot/add-error-aggregation-validation

Copilot AI commented Jan 30, 2026 •

edited by agentic-workflows-dev bot

Loading

Uh oh!

github-actions bot commented Jan 30, 2026 •

edited by agentic-workflows-dev bot

Loading

Uh oh!

github-actions bot commented Jan 30, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Jan 30, 2026

Uh oh!

github-actions bot commented Jan 30, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Jan 30, 2026

Uh oh!

github-actions bot commented Jan 30, 2026

Uh oh!

github-actions bot commented Jan 30, 2026

Uh oh!

github-actions bot commented Jan 30, 2026

Uh oh!

github-actions bot commented Jan 30, 2026

Uh oh!

github-actions bot commented Jan 30, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Copilot AI commented Jan 30, 2026 • edited by agentic-workflows-dev bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Example

Backward Compatibility

Current Behavior

Desired Behavior

Files Affected

Suggested Changes

Success Criteria

Example Output

Implementation Strategy

Priority

Source

Comments on the Issue (you are @copilot in this section)

Changeset

Uh oh!

github-actions bot commented Jan 30, 2026 • edited by agentic-workflows-dev bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jan 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jan 30, 2026

Uh oh!

github-actions bot commented Jan 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jan 30, 2026

Agent Container Tool Check

Uh oh!

github-actions bot commented Jan 30, 2026

Uh oh!

github-actions bot commented Jan 30, 2026

Smoke Test: Claude - ✅ PASS

Uh oh!

github-actions bot commented Jan 30, 2026

Uh oh!

github-actions bot commented Jan 30, 2026

Uh oh!

github-actions bot commented Jan 30, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Copilot AI commented Jan 30, 2026 •

edited by agentic-workflows-dev bot

Loading

github-actions bot commented Jan 30, 2026 •

edited by agentic-workflows-dev bot

Loading

github-actions bot commented Jan 30, 2026 •

edited

Loading

github-actions bot commented Jan 30, 2026 •

edited

Loading