Skip to content

[CI Failure Doctor] Test failures after PR #11036: regex pattern changes without test updates #11055

@github-actions

Description

@github-actions

Summary

The CI workflow failed after merging PR #11036 which updated expiration detection regex patterns in ephemerals.cjs. The changes required blockquote prefixes (>) in the regex patterns but failed to update the corresponding test cases, causing 45 test failures across 3 test files.

Failure Details

Root Cause Analysis

What Went Wrong

PR #11036 "Fix expiration detection for quoted footers and legacy format" updated regex patterns in ephemerals.cjs:

EXPIRATION_PATTERN - Changed to require blockquote prefix:

// OLD: /^-\s*\[x\]\s+expires\s*<!--.../m
// NEW: /^>\s*-\s*\[x\]\s+expires\s*<!--.../m

LEGACY_EXPIRATION_PATTERN - New pattern for old format:

// NEW: /^>\s*-\s*\[x\]\s+expires\s+on\s+(.+?)\s+UTC\s*$/m

The problem: Tests in ephemerals.test.cjs were passing text without blockquote prefixes, causing extractExpirationDate() to return null instead of extracting dates.

Failed Jobs and Errors

Primary Failures: ephemerals.test.cjs (3 tests)

  1. Line 47: "should extract date from body with expiration marker"

    • Test body missing > prefix: "Some text\n- [x] expires <!-- gh-aw-expires: 2026-01-25T15:54:08.894Z --> ..."
    • Error: expected null to be an instance of Date
  2. Line 111: "should handle standard expiration marker format"

    • Same issue - missing blockquote prefix
    • Error: expected null to be an instance of Date
  3. Line 103: "should return null for invalid legacy date"

    • Test expects null for invalid date but got 9999-01-01T00:00:00.000Z
    • Date parsing succeeded when it should have failed

Secondary Failures: collect_ndjson_output.test.cjs (36 tests)

These failures appear unrelated to ephemerals.cjs changes:

  • Validation tests expecting errors but getting empty arrays
  • JSON repair tests not properly fixing malformed JSON
  • Min validation tests not enforcing minimum requirements
  • Code scanning alert validation failing

May indicate:

  • Separate pre-existing issue exposed by test run
  • Or indirect dependency on ephemerals.cjs that broke validation logic

Tertiary Failures: safe_outputs_mcp_server_defaults.test.cjs (2 timeouts)

  • Tests timing out after 10s when checking branch parameters
  • Unhandled assertion errors about missing required fields
  • May indicate MCP server not responding or configuration issues

Recommended Actions

Immediate Fixes (P0 - Critical)

  • Fix ephemerals.test.cjs - Update test bodies to include blockquote prefixes
    • Line 47: Add > to test body
    • Line 111: Add > to test body
    • Line 103: Fix expected behavior for invalid date parsing

Investigation Required (P1 - High)

  • Investigate collect_ndjson_output.test.cjs failures (36 tests)

    • Determine if pre-existing or caused by ephemerals.cjs changes
    • Check if validation logic depends on expiration pattern matching
  • Investigate safe_outputs_mcp_server_defaults.test.cjs timeouts (2 tests)

    • Check MCP server startup/configuration
    • Verify branch parameter validation logic

Prevention Strategies

For PR Authors

  1. Run tests locally before committing: cd actions/setup/js && npm test
  2. Update tests when changing regex patterns - tests must match production patterns
  3. Use table-driven tests to cover multiple input formats
  4. Add regression tests for both old and new formats

For Code Reviewers

  1. Check that tests are updated when code changes
  2. Verify regex patterns match expected input formats in tests
  3. Run tests locally to catch issues before merge

For CI/CD

  1. Block merges if tests fail - this should have prevented the merge
  2. Add pre-merge test gates to catch issues earlier
  3. Consider required status checks for all test suites

AI Team Self-Improvement

Additional prompting instructions for AI coding agents:

When modifying regex patterns (especially in validation, parsing, or detection logic):

  1. Always update corresponding tests to match the new pattern requirements
  2. Test both positive and negative cases:
    • Valid inputs that should match
    • Invalid inputs that should not match
    • Edge cases (whitespace variations, optional prefixes)
  3. Use table-driven tests to cover multiple scenarios:
    describe("pattern matching", () => {
      const testCases = [
        { input: "> - [x] expires...", shouldMatch: true },
        { input: "- [x] expires...", shouldMatch: false },
        { input: ">  -  [x]  expires...", shouldMatch: true },
      ];
      testCases.forEach(({input, shouldMatch}) => {
        it(`should ${shouldMatch ? '' : 'not '}match: ${input}`, () => {
          expect(pattern.test(input)).toBe(shouldMatch);
        });
      });
    });
  4. Run tests locally before committing: make test-unit or relevant test command
  5. Document pattern requirements in code comments and test descriptions

Historical Context

Test failures after merging PRs are frequent in this repository:

This suggests pre-merge testing may not be comprehensive enough or CI gates are not strictly enforced.

Additional Notes

Scope of Impact:

Why didn't PR #11036 catch these failures before merge?

  • Need to verify if PR had CI checks that ran
  • May have been merged despite test failures (manual override?)
  • Or tests were skipped/not run in PR CI

AI generated by CI Failure Doctor

To add this workflow in your repository, run gh aw add githubnext/agentics/workflows/ci-doctor.md@ea350161ad5dcc9624cf510f134c6a9e39a6f94d. See usage guide.

Metadata

Metadata

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions