-
Notifications
You must be signed in to change notification settings - Fork 262
Description
Summary
The CI workflow failed after merging PR #11036 which updated expiration detection regex patterns in ephemerals.cjs. The changes required blockquote prefixes (>) in the regex patterns but failed to update the corresponding test cases, causing 45 test failures across 3 test files.
Failure Details
- Run: 21216811553
- Commit: 702d411
- Trigger: Push to main (merged PR Fix expiration detection for quoted footers and legacy format #11036)
- Failed Job:
js(JavaScript tests) - Duration: ~78 seconds
- Test Results: 45 failed / 2875 passed / 17 skipped (2937 total)
Root Cause Analysis
What Went Wrong
PR #11036 "Fix expiration detection for quoted footers and legacy format" updated regex patterns in ephemerals.cjs:
EXPIRATION_PATTERN - Changed to require blockquote prefix:
// OLD: /^-\s*\[x\]\s+expires\s*<!--.../m
// NEW: /^>\s*-\s*\[x\]\s+expires\s*<!--.../mLEGACY_EXPIRATION_PATTERN - New pattern for old format:
// NEW: /^>\s*-\s*\[x\]\s+expires\s+on\s+(.+?)\s+UTC\s*$/mThe problem: Tests in ephemerals.test.cjs were passing text without blockquote prefixes, causing extractExpirationDate() to return null instead of extracting dates.
Failed Jobs and Errors
Primary Failures: ephemerals.test.cjs (3 tests)
-
Line 47: "should extract date from body with expiration marker"
- Test body missing
>prefix:"Some text\n- [x] expires <!-- gh-aw-expires: 2026-01-25T15:54:08.894Z --> ..." - Error:
expected null to be an instance of Date
- Test body missing
-
Line 111: "should handle standard expiration marker format"
- Same issue - missing blockquote prefix
- Error:
expected null to be an instance of Date
-
Line 103: "should return null for invalid legacy date"
- Test expects null for invalid date but got
9999-01-01T00:00:00.000Z - Date parsing succeeded when it should have failed
- Test expects null for invalid date but got
Secondary Failures: collect_ndjson_output.test.cjs (36 tests)
These failures appear unrelated to ephemerals.cjs changes:
- Validation tests expecting errors but getting empty arrays
- JSON repair tests not properly fixing malformed JSON
- Min validation tests not enforcing minimum requirements
- Code scanning alert validation failing
May indicate:
- Separate pre-existing issue exposed by test run
- Or indirect dependency on ephemerals.cjs that broke validation logic
Tertiary Failures: safe_outputs_mcp_server_defaults.test.cjs (2 timeouts)
- Tests timing out after 10s when checking branch parameters
- Unhandled assertion errors about missing required fields
- May indicate MCP server not responding or configuration issues
Recommended Actions
Immediate Fixes (P0 - Critical)
- Fix ephemerals.test.cjs - Update test bodies to include blockquote prefixes
- Line 47: Add
>to test body - Line 111: Add
>to test body - Line 103: Fix expected behavior for invalid date parsing
- Line 47: Add
Investigation Required (P1 - High)
-
Investigate collect_ndjson_output.test.cjs failures (36 tests)
- Determine if pre-existing or caused by ephemerals.cjs changes
- Check if validation logic depends on expiration pattern matching
-
Investigate safe_outputs_mcp_server_defaults.test.cjs timeouts (2 tests)
- Check MCP server startup/configuration
- Verify branch parameter validation logic
Prevention Strategies
For PR Authors
- Run tests locally before committing:
cd actions/setup/js && npm test - Update tests when changing regex patterns - tests must match production patterns
- Use table-driven tests to cover multiple input formats
- Add regression tests for both old and new formats
For Code Reviewers
- Check that tests are updated when code changes
- Verify regex patterns match expected input formats in tests
- Run tests locally to catch issues before merge
For CI/CD
- Block merges if tests fail - this should have prevented the merge
- Add pre-merge test gates to catch issues earlier
- Consider required status checks for all test suites
AI Team Self-Improvement
Additional prompting instructions for AI coding agents:
When modifying regex patterns (especially in validation, parsing, or detection logic):
- Always update corresponding tests to match the new pattern requirements
- Test both positive and negative cases:
- Valid inputs that should match
- Invalid inputs that should not match
- Edge cases (whitespace variations, optional prefixes)
- Use table-driven tests to cover multiple scenarios:
describe("pattern matching", () => { const testCases = [ { input: "> - [x] expires...", shouldMatch: true }, { input: "- [x] expires...", shouldMatch: false }, { input: "> - [x] expires...", shouldMatch: true }, ]; testCases.forEach(({input, shouldMatch}) => { it(`should ${shouldMatch ? '' : 'not '}match: ${input}`, () => { expect(pattern.test(input)).toBe(shouldMatch); }); }); });
- Run tests locally before committing:
make test-unitor relevant test command - Document pattern requirements in code comments and test descriptions
Historical Context
Test failures after merging PRs are frequent in this repository:
- [CI Failure Doctor] npm ci fails with SSH authentication error for @actions/github-script dependency #11040: npm ci SSH authentication errors
- [CI Failure Doctor] Dependabot PR #11024 broke CI: npm peer dependency conflict between vitest@4.0.17 and @vitest/coverage-v8@4.0.10 #11030: Dependabot peer dependency conflicts
- [CI Failure Doctor] CI Failure Doctor: close_older_issues.test.cjs uses Jest instead of Vitest #10895: Jest/Vitest framework confusion
- [CI Failure Doctor] lint-go failure after merge of PR #10756 (compiler test files) #10772: lint-go failures after test file additions
This suggests pre-merge testing may not be comprehensive enough or CI gates are not strictly enforced.
Additional Notes
Scope of Impact:
- Production code (ephemerals.cjs) is likely correct - the fix was needed for issue [team-status] Daily Team Status - January 19, 2026 🌟 #10667
- Only tests are broken - no immediate runtime impact
- But indicates incomplete PR validation process
Why didn't PR #11036 catch these failures before merge?
- Need to verify if PR had CI checks that ran
- May have been merged despite test failures (manual override?)
- Or tests were skipped/not run in PR CI
AI generated by CI Failure Doctor
To add this workflow in your repository, run
gh aw add githubnext/agentics/workflows/ci-doctor.md@ea350161ad5dcc9624cf510f134c6a9e39a6f94d. See usage guide.