Skip to content

Comments

Fix ephemerals tests after blockquote prefix requirement in PR #11036#11058

Merged
pelikhan merged 2 commits intomainfrom
copilot/fix-regex-test-failures
Jan 21, 2026
Merged

Fix ephemerals tests after blockquote prefix requirement in PR #11036#11058
pelikhan merged 2 commits intomainfrom
copilot/fix-regex-test-failures

Conversation

Copy link
Contributor

Copilot AI commented Jan 21, 2026

PR #11036 updated EXPIRATION_PATTERN and LEGACY_EXPIRATION_PATTERN to require blockquote prefixes (>) for expiration markers, but test cases still used the old format without prefixes.

Changes

  • Updated 4 test cases to include > prefix in test bodies:
    • should extract date from body with expiration marker
    • should return null for invalid date
    • should return null for invalid legacy date (also fixed to use truly invalid date string)
    • should handle standard expiration marker format
  • Clarified misleading test name: should matchshould NOT match for pattern without blockquote

Before/After

// Before (fails to match)
const body = "- [x] expires <!-- gh-aw-expires: 2026-01-25T15:54:08.894Z -->";

// After (matches regex)
const body = "> - [x] expires <!-- gh-aw-expires: 2026-01-25T15:54:08.894Z -->";

The regex requires ^>\s*-\s*\[x\]\s+expires per the new blockquote-prefixed footer format.

Original prompt

This section details on the original issue you should resolve

<issue_title>[CI Failure Doctor] Test failures after PR #11036: regex pattern changes without test updates</issue_title>
<issue_description>## Summary

The CI workflow failed after merging PR #11036 which updated expiration detection regex patterns in ephemerals.cjs. The changes required blockquote prefixes (>) in the regex patterns but failed to update the corresponding test cases, causing 45 test failures across 3 test files.

Failure Details

Root Cause Analysis

What Went Wrong

PR #11036 "Fix expiration detection for quoted footers and legacy format" updated regex patterns in ephemerals.cjs:

EXPIRATION_PATTERN - Changed to require blockquote prefix:

// OLD: /^-\s*\[x\]\s+expires\s*<!--.../m
// NEW: /^>\s*-\s*\[x\]\s+expires\s*<!--.../m

LEGACY_EXPIRATION_PATTERN - New pattern for old format:

// NEW: /^>\s*-\s*\[x\]\s+expires\s+on\s+(.+?)\s+UTC\s*$/m

The problem: Tests in ephemerals.test.cjs were passing text without blockquote prefixes, causing extractExpirationDate() to return null instead of extracting dates.

Failed Jobs and Errors

Primary Failures: ephemerals.test.cjs (3 tests)

  1. Line 47: "should extract date from body with expiration marker"

    • Test body missing > prefix: "Some text\n- [x] expires <!-- gh-aw-expires: 2026-01-25T15:54:08.894Z --> ..."
    • Error: expected null to be an instance of Date
  2. Line 111: "should handle standard expiration marker format"

    • Same issue - missing blockquote prefix
    • Error: expected null to be an instance of Date
  3. Line 103: "should return null for invalid legacy date"

    • Test expects null for invalid date but got 9999-01-01T00:00:00.000Z
    • Date parsing succeeded when it should have failed

Secondary Failures: collect_ndjson_output.test.cjs (36 tests)

These failures appear unrelated to ephemerals.cjs changes:

  • Validation tests expecting errors but getting empty arrays
  • JSON repair tests not properly fixing malformed JSON
  • Min validation tests not enforcing minimum requirements
  • Code scanning alert validation failing

May indicate:

  • Separate pre-existing issue exposed by test run
  • Or indirect dependency on ephemerals.cjs that broke validation logic

Tertiary Failures: safe_outputs_mcp_server_defaults.test.cjs (2 timeouts)

  • Tests timing out after 10s when checking branch parameters
  • Unhandled assertion errors about missing required fields
  • May indicate MCP server not responding or configuration issues

Recommended Actions

Immediate Fixes (P0 - Critical)

  • Fix ephemerals.test.cjs - Update test bodies to include blockquote prefixes
    • Line 47: Add > to test body
    • Line 111: Add > to test body
    • Line 103: Fix expected behavior for invalid date parsing

Investigation Required (P1 - High)

  • Investigate collect_ndjson_output.test.cjs failures (36 tests)

    • Determine if pre-existing or caused by ephemerals.cjs changes
    • Check if validation logic depends on expiration pattern matching
  • Investigate safe_outputs_mcp_server_defaults.test.cjs timeouts (2 tests)

    • Check MCP server startup/configuration
    • Verify branch parameter validation logic

Prevention Strategies

For PR Authors

  1. Run tests locally before committing: cd actions/setup/js && npm test
  2. Update tests when changing regex patterns - tests must match production patterns
  3. Use table-driven tests to cover multiple input formats
  4. Add regression tests for both old and new formats

For Code Reviewers

  1. Check that tests are updated when code changes
  2. Verify regex patterns match expected input formats in tests
  3. Run tests locally to catch issues before merge

For CI/CD

  1. Block merges if tests fail - this should have prevented the merge
  2. Add pre-merge test gates to catch issues earlier
  3. Consider required status checks for all test suites

AI Team Self-Improvement

Additional prompting instructions for AI coding agents:

When modifying regex patterns (especially in validation, parsing, or detection logic):

  1. Always update corresponding tests to match the new pattern requirements
  2. Test both positive and negative cases:
    • Valid inputs that should match
    • Invalid inputs that should not match
    • Edge cases (whitespace variations, optional prefixes)
  3. Use table-driven tests to cover multiple scenarios:
    describe("pattern match...

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

- Update test bodies to include '>' blockquote prefix required by updated regex patterns
- Fix invalid date test to use a truly invalid date string
- Clarify test name for EXPIRATION_PATTERN without blockquote prefix
- All 35 ephemerals tests now pass

Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix test failures due to regex pattern changes Fix ephemerals tests after blockquote prefix requirement in PR #11036 Jan 21, 2026
Copilot AI requested a review from mnkiefer January 21, 2026 17:27
@pelikhan pelikhan marked this pull request as ready for review January 21, 2026 17:41
@pelikhan pelikhan merged commit c4a0b67 into main Jan 21, 2026
@pelikhan pelikhan deleted the copilot/fix-regex-test-failures branch January 21, 2026 17:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CI Failure Doctor] Test failures after PR #11036: regex pattern changes without test updates

3 participants