🏥 Safe Output Health Report - February 15, 2026

[github-actions[bot]]

Executive Summary

Audit Period: Last 24 hours (Feb 14, 04:28 UTC → Feb 15, 04:28 UTC)
Runs Analyzed: 41 workflow runs
Workflows Active: 10 unique workflows
Safe Output Jobs Executed: 41
Safe Output Jobs Failed: 5
Overall Success Rate: 87.8%
Error Clusters Identified: 3

---

Safe Output Job Statistics

Job Type	Total Executions	Failures	Success Rate
update_project	28	4	85.7%
create_project_status_update	4	0	100%
assign_copilot	1	1	0%
noop	8	0	100%

---

Error Clusters

Cluster 1: Temporary ID Resolution Failures (High Severity)

Count: 4 occurrences
Affected Workflow: Smoke Project
Affected Job Type: update_project
Affected Runs:

• §22027664549
• §22028274837
• §22028525110
• §22029576944

Sample Error:

##[error]Failed to manage project: draft_issue_id "aw_draft1" not found 
in temporary ID map and no draft_title provided for fallback lookup

GraphQL Error during: update_project
Message: draft_issue_id "aw_draft1" not found in temporary ID map and 
no draft_title provided for fallback lookup
```

**Root Cause:** The `update_project` safe output job is attempting to reference a temporary ID (`aw_draft1`) that was created in an earlier safe output call (`create_project_draft_issue`), but the temporary ID is not present in the temporary ID map when the `update_project` job executes. This indicates either:
1. The draft issue was never created successfully
2. The temporary ID map is not being properly propagated between safe output jobs
3. The order of execution is incorrect

**Impact:** Projects cannot be properly updated with draft issues, breaking the project smoke test workflow's ability to validate draft issue functionality.

---

#### Cluster 2: GitHub Project View Limit Exceeded (Medium Severity)

**Count:** 4 occurrences  
**Affected Workflow:** Smoke Project  
**Affected Job Type:** update_project  
**Affected Runs:** Same as Cluster 1

**Sample Error:**
```
##[error]Failed to manage project: Views are limited to 50 per project
##[error]Failed to create configured view 1: Smoke Test Board
GraphQL Error during: Creating configured view: Smoke Test Board
Message: Views are limited to 50 per project
```

**Root Cause:** The Smoke Project workflow attempts to create a new project view called "Smoke Test Board" every time it runs, but the target project (github/projects/24068) has already reached GitHub's hard limit of 50 views per project.

**Impact:** The smoke test cannot validate project view creation, but this is a test-specific issue rather than a production safe output problem. The smoke test repeatedly creates views without cleaning them up, causing the project to hit the platform limit.

---

#### Cluster 3: Repository Permission Error (Critical Severity)

**Count:** 1 occurrence  
**Affected Workflow:** Duplicate Code Detector  
**Affected Job Type:** assign_copilot  
**Affected Run:** [§22029315368](https://github.com/github/gh-aw/actions/runs/22029315368)

**Sample Error:**
```
##[error]Raw GraphQL error details (for troubleshooting):
##[error]{
##[error]  "errors": [
##[error]    {
##[error]      "type": "FORBIDDEN",
##[error]      "message": "Could not assign agent: target repository is not writable."
##[error]    }
##[error]  ]
##[error]}
##[error]Failed to assign copilot to issue #15861 in github/gh-aw: 
Failed to assign copilot via GraphQL

Root Cause: The assign_copilot safe output job attempted to assign the Copilot agent to issue #15861, but the operation failed because the target repository does not have write permissions for the authenticated user/token. This is a permission mismatch between what the safe output job expects and what the GITHUB_TOKEN actually has access to.

Impact: The workflow cannot automatically assign Copilot agents to issues for automated code fixes, requiring manual intervention. This breaks the automated duplicate code detection → fix workflow.

---

Root Cause Analysis

API-Related Issues

None identified - All GitHub API calls that succeeded worked as expected. Rate limits were not an issue during the audit period.

---

Data Validation Issues

Temporary ID Map Propagation (High Priority)

The temporary ID resolution failures indicate a systematic issue with how temporary IDs are propagated between safe output jobs. The pattern shows:

1. Agent creates safe output with create_project_draft_issue using temporary ID aw_draft1
2. Agent creates safe output with update_project referencing aw_draft1
3. When safe output processor runs, the first job completes successfully
4. When the second job runs, aw_draft1 is not found in the temporary ID map

Potential causes:

• Temporary ID map is not being persisted between job steps
• Race condition in map updates when jobs run in parallel
• Incorrect scoping of temporary ID map (job-level vs. workflow-level)
• Draft issue creation failing silently, never registering the temporary ID

---

Permission Issues

Copilot Agent Assignment Permission Mismatch (Critical)

The assign_copilot job requires write permissions to the repository, but the GITHUB_TOKEN provided to the workflow does not have sufficient permissions. This could be due to:

1. Token scope mismatch: The workflow needs contents: write + issues: write, but may be missing repository-level write access
2. Repository settings: The target repository may have additional protections that prevent agent assignment
3. Organization policy: There may be organization-level policies restricting automated agent assignments

---

Logic Errors

Project View Cleanup Missing (Medium Priority)

The Smoke Project workflow creates a new "Smoke Test Board" view on every run but never cleans up old views. After 50 runs, the project hits GitHub's hard limit and all subsequent runs fail this step. This is test infrastructure cruft, not a safe output bug.

---

Recommendations

Critical Issues (Immediate Action Required)

1. Fix Copilot Agent Assignment Permissions

• Priority: Critical
• Root Cause: GITHUB_TOKEN lacks repository write permissions
• Recommended Action:
  1. Update workflow permissions to include repository-level write access
  2. Verify organization policies don't block automated agent assignments
  3. Add permission check before attempting agent assignment
  4. Implement graceful fallback when permissions are insufficient
• Affected: Duplicate Code Detector workflow and any workflow using assign_copilot
• Estimated Effort: Small (configuration change + validation)

---

High Priority Issues

2. Fix Temporary ID Map Propagation

• Priority: High
• Root Cause: Temporary IDs created in one safe output job are not available in subsequent jobs
• Recommended Action:
  1. Investigate temporary ID map persistence mechanism
  2. Verify map is stored in GitHub Actions outputs/artifacts between jobs
  3. Add debug logging to show when IDs are registered and looked up
  4. Ensure proper ordering of safe output job execution
  5. Add validation that referenced temporary IDs exist before attempting to use them
• Affected: Smoke Project workflow and any workflow using cross-job temporary ID references
• Estimated Effort: Medium (requires code investigation + fix + testing)

---

Medium Priority Issues

3. Implement Project View Cleanup in Smoke Tests

• Priority: Medium
• Root Cause: Smoke test creates views but never deletes them, exhausting the 50-view limit
• Recommended Action:
  1. Add cleanup step to Smoke Project workflow to delete test views after validation
  2. Implement view name with run ID so views can be uniquely identified
  3. Add pre-flight check to detect and warn about approaching view limit
  4. Consider using a dedicated test project that can be reset periodically
• Affected: Smoke Project workflow only (not a production issue)
• Estimated Effort: Small (add cleanup logic to workflow)

---

Bug Fixes Required

Bug 1: Temporary ID Map Not Persisting Between Safe Output Jobs

File/Location: actions/process_safe_outputs.cjs (temporary ID map management)
Problem: Temporary IDs created during safe output processing are not available to subsequent safe output jobs in the same workflow run
Fix: Ensure temporary ID map is:

1. Exported as a workflow output after each safe output job completes
2. Imported at the start of subsequent safe output jobs
3. Merged correctly when multiple jobs create temporary IDs
4. Validated before attempting to resolve references

Affected Jobs: update_project, any job referencing temporary IDs from previous jobs

---

Bug 2: Missing Permission Validation for assign_copilot

File/Location: actions/assign_copilot.cjs
Problem: Job attempts agent assignment without checking if token has required permissions
Fix: Add permission validation:

// Before attempting assignment
const hasWritePermission = await checkRepositoryPermissions();
if (!hasWritePermission) {
  core.warning('Insufficient permissions to assign agent. Requires repository write access.');
  return { success: false, reason: 'insufficient_permissions' };
}

Affected Jobs: assign_copilot

---

Configuration Changes

1. Update Workflow Permissions for Agent Assignment

• Current: contents: write, issues: write
• Recommended: Add explicit repository-level write permission or use a PAT with appropriate scopes
• Reason: Agent assignment requires repository write access beyond issue write access

2. Increase safe_outputs Job Timeout

• Current: Default timeout (likely 360 minutes)
• Recommended: Keep current timeout but add timeout warnings at 80% completion
• Reason: No timeout issues observed, but proactive monitoring would help

---

Process Improvements

1. Add Temporary ID Validation

• Current State: Jobs reference temporary IDs without checking if they exist
• Proposed: Add validation step that:
  • Checks all temporary ID references before job execution
  • Fails fast with clear error message if IDs are missing
  • Provides diagnostic information about available IDs
• Benefits: Easier debugging, faster failure detection, clearer error messages

2. Implement Safe Output Job Health Metrics

• Current State: Errors are logged but not tracked systematically
• Proposed: Add metrics collection for:
  • Job success/failure rates by type
  • Common error patterns
  • Temporary ID resolution success rate
  • API error rate by endpoint
• Benefits: Proactive identification of degrading performance, trend analysis

3. Add Smoke Test Cleanup Phase

• Current State: Smoke tests create resources but don't clean them up
• Proposed: Add cleanup phase that:
  • Deletes project views created during test
  • Removes draft issues after validation
  • Resets project state for next run
• Benefits: Prevents resource exhaustion, cleaner test environment

---

Work Item Plans

Work Item 1: Fix Temporary ID Map Propagation

Type: Bug Fix
Priority: High
Description: Temporary IDs created by safe output jobs are not available to subsequent jobs that reference them, causing update operations to fail.

Acceptance Criteria:

• Temporary IDs created in job N are accessible in job N+1
• Multiple safe output jobs can create and reference IDs in the same workflow
• Clear error messages when temporary ID is not found
• Debug logging shows ID registration and lookup operations
• Smoke Project workflow passes without temporary ID errors

Technical Approach:

1. Review process_safe_outputs.cjs temporary ID map implementation
2. Verify map is exported to workflow outputs after each job
3. Update job initialization to import and merge existing temporary ID map
4. Add validation before attempting to resolve temporary ID references
5. Add comprehensive logging for debugging
6. Update documentation on temporary ID lifecycle

Estimated Effort: Medium (3-5 days)
Dependencies: None

---

Work Item 2: Fix Copilot Agent Assignment Permissions

Type: Bug Fix
Priority: Critical
Description: The assign_copilot safe output job fails with permission errors when attempting to assign agents to issues.

Acceptance Criteria:

• assign_copilot job can successfully assign agents to issues
• Clear error message when permissions are insufficient
• Graceful fallback when assignment fails
• Documentation updated with required permissions
• Duplicate Code Detector workflow completes successfully

Technical Approach:

1. Review current GITHUB_TOKEN permissions in workflow
2. Add permission check before attempting agent assignment
3. Update workflow permissions if needed (or use PAT)
4. Add graceful error handling with actionable error messages
5. Test with various permission scenarios
6. Document required permissions in workflow README

Estimated Effort: Small (1-2 days)
Dependencies: Access to organization admin for permission review

---

Work Item 3: Add Smoke Test Cleanup Phase

Type: Enhancement
Priority: Medium
Description: Smoke Project workflow exhausts GitHub's 50-view-per-project limit by creating views without cleaning them up.

Acceptance Criteria:

• Smoke test creates uniquely-named views with run ID
• Cleanup phase deletes test views after validation
• Pre-flight check warns if project approaching view limit
• Workflow completes successfully even after many runs
• Documentation explains cleanup mechanism

Technical Approach:

1. Update Smoke Project workflow to include run ID in view names
2. Add cleanup job that runs after safe output validation
3. Implement view deletion using GitHub GraphQL API
4. Add pre-flight check to count existing views
5. Consider using dedicated test project that can be reset
6. Document cleanup process and troubleshooting

Estimated Effort: Small (1-2 days)
Dependencies: None

---

Work Item 4: Add Safe Output Health Metrics Dashboard

Type: Process Improvement
Priority: Low
Description: Create a dashboard that tracks safe output job health metrics over time for proactive monitoring.

Acceptance Criteria:

• Metrics collected for all safe output job executions
• Dashboard shows success rates by job type
• Trend analysis identifies degrading performance
• Alerts triggered when success rate drops below threshold
• Historical data retained for 90 days minimum

Technical Approach:

1. Add metrics collection to process_safe_outputs.cjs
2. Export metrics to workflow outputs
3. Create aggregation workflow that collects daily metrics
4. Build visualization dashboard (could be a discussion post)
5. Set up alerts for critical metrics
6. Document metrics meaning and thresholds

Estimated Effort: Large (5-10 days)
Dependencies: None

---

Historical Context

This is the first safe output health audit using the automated gh-aw logs tool. No historical data is available for trend comparison.

Baseline Metrics Established:

• Overall success rate: 87.8%
• Most reliable job: create_project_status_update (100% success)
• Most problematic job: assign_copilot (0% success, 1 attempt)
• Common failure modes: Temporary ID resolution (4 failures), Project limits (4 failures)

Next Steps for Historical Tracking:

• Run daily audits to establish trend baselines
• Compare error patterns across weeks
• Identify seasonal patterns or correlation with deployments
• Track improvement after bug fixes are implemented

---

Metrics and KPIs

Overall Safe Output Success Rate: 87.8%
Most Reliable Job Type: create_project_status_update (100% success)
Most Problematic Job Type: assign_copilot (0% success rate, 1 execution)
Average Errors Per Run: 0.22
Runs With Errors: 5 out of 41 (12.2%)
Critical Errors: 1
High Severity Errors: 4
Medium Severity Errors: 4

---

Next Steps

Immediate (This Week):

• Fix Copilot agent assignment permissions (Work Item 2)
• Begin investigation of temporary ID map propagation (Work Item 1)
• Clean up existing project views manually to unblock Smoke Project tests

Short Term (Next 2 Weeks):

• Complete temporary ID map fix and deploy (Work Item 1)
• Implement smoke test cleanup phase (Work Item 3)
• Verify all fixes with comprehensive smoke tests
• Update documentation with required permissions and troubleshooting

Long Term (Next Month):

• Implement safe output health metrics dashboard (Work Item 4)
• Establish weekly health report reviews
• Create runbook for common safe output failures
• Add automated alerting for critical failures

---

References:

• §22027664549 - Smoke Project with temporary ID errors
• §22029315368 - Duplicate Code Detector with permission errors
• §22029576944 - Most recent Smoke Project failure

AI generated by Safe Output Health Monitor

• expires on Feb 22, 2026, 4:32 AM UTC

[github-actions[bot]]

🤖 Beep boop! The smoke test agent was here checking system health. All systems nominal! 🚀✨

AI generated by Smoke Copilot

[github-actions[bot]]

💥 WHOOSH! The Claude smoke test agent was here! 💨

KABOOM! All systems operational! The agent has successfully completed its validation sweep through the codebase.

🦸 Testing complete! ZOOM! On to the next adventure! 🚀

AI generated by Smoke Claude