Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Jan 21, 2026

Two tests in close_older_issues.test.cjs failed because assertions expected old message formats while the implementation had been updated with emoji prefixes and enhanced descriptions.

Changes

  • Updated warning assertion: "Found 15 older issues...""⚠️ Found 15 older issues..."
  • Updated info assertion: "No older issues found to close""✓ No older issues found to close - operation complete"

Context

The close_older_issues.cjs implementation was enhanced with structured logging output including emoji prefixes and operation status indicators. Test expectations were not synchronized with these changes.

// Before
expect(global.core.warning).toHaveBeenCalledWith(
  `Found 15 older issues, but only closing the first ${MAX_CLOSE_COUNT}`
);

// After
expect(global.core.warning).toHaveBeenCalledWith(
  `⚠️  Found 15 older issues, but only closing the first ${MAX_CLOSE_COUNT}`
);
Original prompt

This section details on the original issue you should resolve

<issue_title>[CI Failure Doctor] JS Test Suite Failure - 44 Tests Failing After PR #11053 (handle_agent_failure changes)</issue_title>
<issue_description># 🏥 CI Failure Investigation - Run githubnext/gh-aw#21223010566

Summary

The CI workflow failed with 44 JavaScript test failures across 4 test files in the js job after merging PR #11053. The failures indicate that changes to handle_agent_failure.cjs and its tests were not properly synchronized with the safe outputs schema validation system.

Failure Details

Root Cause Analysis

🔴 PRIMARY ISSUE: Safe Output Schema Validation Failures (36 tests)

File: collect_ndjson_output.test.cjs
Error Pattern: expected [] to have a length of X but got +0

The commit changed handle_agent_failure.cjs and handle_agent_failure.test.cjs but did NOT update the safe outputs schema definitions. This caused widespread test failures in the validation system.

Failing test categories:

  1. Required field validation (8 tests)
    • add-labels, create-pull-request, create-discussion, create-pull-request-review-comment, update_release, assign_milestone, assign_to_agent
  2. JSON repair functionality (5 tests)
    • Array syntax issues, bracket type mismatches
  3. Code scanning alert validation (7 tests)
    • Valid entries, missing fields, invalid types, severity levels
  4. Sanitization (1 test)
    • Multi-field type handling
  5. Min validation (5 tests)
    • Min requirement enforcement across types
  6. Noop validation (1 test)
    • Multiple noop messages
  7. Link sub-issue temp ID validation (6 tests)
    • Temporary ID format, parent/sub issue validation

🟡 SECONDARY ISSUE: MCP Server Test Timeouts (2 tests)

File: safe_outputs_mcp_server_defaults.test.cjs
Error: Test timed out in 10000ms + AssertionError: expected undefined to deeply equal [...]

Tests:

  • should have optional branch parameter for create_pull_request
  • should have optional branch parameter for push_to_pull_request_branch

Root Cause: Tests expect schema properties to exist but receive undefined, suggesting schema structure mismatch.

🟢 TERTIARY ISSUE: Test Assertion Changes (2 tests)

File: close_older_issues.test.cjs
Error: Unexpected message format changes

Tests:

  • should limit to MAX_CLOSE_COUNT issues - Expected single warning message, received 3 formatted messages
  • should return empty array if no older issues found - Expected single info message, received 24 debug messages

Root Cause: Code now outputs more verbose logging that wasn't reflected in test assertions.

🔵 QUATERNARY ISSUE: Go Handler Failures (4 tests)

File: mcp_handler_go.test.cjs
Error: Various Go script execution failures

Failed Test Summary

  • Test Files: 4 failed | 140 passed (144 total)
  • Tests: 44 failed | 2876 passed | 17 skipped (2937 total)
  • Duration: 80.51s

Changes in PR #11053

Modified files:

  1. actions/setup/js/handle_agent_failure.cjs (+2, -4)
    • Changed parent issue title from "[agentics] Agentic Workflow Issues" to "[agentic-workflows] Failed runs"
    • Removed duplicate title from issue body
  2. actions/setup/js/handle_agent_failure.test.cjs (+3, -3)
    • Updated test assertions to match new title

The problem: These changes updated the templates but didn't synchronize with:

  • Safe output schema definitions in pkg/workflow/js/safe_outputs_tools.json
  • Validation logic expectations in test files
  • MCP server schema parameters

Investigation Findings

Why Did This Pass PR Review?

The PR likely didn't run the full test suite before merging, or tests passed locally but failed in CI due to:

  1. Missing npm test pre-commit check
  2. Schema files cached differently in CI vs local
  3. Environment-specific test behavior

Similar Past Issues

Found similar issue #9965 (closed) from Jan 14, 2026:

  • 46 test failures (similar count to current 44)
  • Same test files affected
  • Different root cause (filesystem permissions)
  • Pattern: JS tests frequently fail in CI after merges

Recommended Actions

🔴 IMMEDIATE FIX (Required to unblock main)

  1. Revert or fix PR Update parent issue template for agentic-workflow failures #11053 changes

    Option A: Quick revert

    git revert 23faa9c7916c23365605ddb707896d88616b2e61

    Option B: Fix schema sync (recommended)

    cd actions/setup/js
    npm ci
    # Update safe_outputs_tools.json schema definitions
    # Update validation expectations in...
    

Custom agent used: ci-cleaner
Tidies up the repository CI state by formatting sources, running linters, fixing issues, running tests, and recompiling workflows


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 2 commits January 21, 2026 20:05
The close_older_issues.cjs implementation was updated to use more
detailed and formatted log messages. This commit updates the test
assertions to match the new message formats:

- Warning message now includes ⚠️ emoji prefix
- Info message now includes ✓ emoji and "operation complete" suffix

Fixes #11053 fallout - resolves 2 JavaScript test failures in
close_older_issues.test.cjs that were blocking CI.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan pelikhan marked this pull request as ready for review January 21, 2026 20:11
Copilot AI changed the title [WIP] Fix JS test suite failures after PR #11053 Fix test assertions for enhanced logging in close_older_issues Jan 21, 2026
Copilot AI requested a review from pelikhan January 21, 2026 20:11
@pelikhan pelikhan merged commit dabc34c into main Jan 21, 2026
48 checks passed
@pelikhan pelikhan deleted the copilot/fix-js-test-suite-failures branch January 21, 2026 20:41
@github-actions
Copy link
Contributor

CI Failure Investigation - Run #30997

This PR fixed the test failure that affected CI run #30997 (21224098253).

Failure Details

Timeline

Error Messages

Error: expected "vi.fn()" to be called with arguments: [ 'No older issues found to close' ]
Expected: "✓ No older issues found to close - operation complete"

Analysis

The test failure was unrelated to PR #11080's changes (which only modified documentation and added Go tests). The failure was caused by enhanced logging format introduced in PR #10882 without updating the test assertions. This PR correctly updated the test expectations to match the new message format.

Prevention

✅ This issue is already resolved. For future prevention:

  • Run full test suite (including JS tests) before merging changes that affect console output
  • When modifying core.info/warning/error messages, always update corresponding test assertions

Investigation stored in /tmp/memory/investigations/2026-01-21-run-21224098253.json

AI generated by CI Failure Doctor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CI Failure Doctor] JS Test Suite Failure - 44 Tests Failing After PR #11053 (handle_agent_failure changes)

2 participants