[CI Failure Doctor] JS tests failed after PR #10421 merged - MCP Gateway format change broke tests

[github-actions]

🏥 CI Failure Investigation - Run #21089437610

⚠️ Critical Issue: PR Merged with Failing Tests

PR #10421 was merged to main despite the JS test job failing. This indicates a CI configuration issue where required checks are not properly blocking merges.

Summary

The CI workflow run for commit bb65d5921db32c7f27be4c37e47e8ed54b7359b4 (PR #10421) failed due to JavaScript test failures in the js job. The PR introduced changes to MCP Gateway schema validation, changing from Docker command format to the new container-based format specified in the MCP Gateway Specification v1.0.0.

Failure Details

• Run: 21089437610
• Commit: bb65d5921db32c7f27be4c37e47e8ed54b7359b4
• PR: Fix MCP Gateway schema validation for stdio servers with container field #10421 - "Fix MCP Gateway schema validation for stdio servers with container field"
• Trigger: push to main
• Merged At: 2026-01-17T05:47:33Z
• Failed Job: js (Job ID: 60658521863)
• Failed Step: "Run tests" (Step Weekly Research Report: AI Workflow Automation Landscape and Market Opportunities - August 2025 #7)
• Test Command: cd actions/setup/js && npm test

Root Cause Analysis

Changes Made in PR #10421

The PR changed MCP Gateway config format from legacy Docker commands to the new MCP Gateway container-based format:

Before:

{
  "type": "stdio",
  "command": "docker",
  "args": ["run", "--rm", "-i", "mcp/ast-grep:latest"]
}

After:

{
  "type": "stdio",
  "container": "mcp/ast-grep:latest"
}

Files Modified

1. pkg/workflow/mcp-config.go:

   • Removed transformation of container field to docker run commands
   • Now combines container + version into single image string
   • Updated property rendering to include container, entrypoint, entrypointArgs, mounts

2. pkg/workflow/mcp_container_args_test.go:

   • Updated Go tests to expect MCP Gateway format
   • Tests now check result.Container instead of result.Command == "docker"

3. Lock files regenerated (5 workflows):

   • .github/workflows/brave.lock.yml
   • .github/workflows/go-pattern-detector.lock.yml
   • .github/workflows/mcp-inspector.lock.yml
   • .github/workflows/notion-issue-summary.lock.yml
   • .github/workflows/scout.lock.yml

Failure Hypothesis

The JavaScript test job (actions/setup/js && npm test) failed, suggesting:

1. Schema Validation Tests: JS tests may validate MCP config schema and expect the old Docker command format
2. Runtime Validation: The compiled workflow lock files use the new format, but JS runtime validation may not support it yet
3. Test Fixtures: Test fixtures or mocks may hardcode the old Docker command structure

Investigation Findings

What We Know

✅ Go tests passed - The mcp_container_args_test.go was updated correctly
✅ Workflow compilation succeeded - Lock files were regenerated successfully
❌ JS tests failed - The js job failed at "Run tests" step
⚠️ Merged despite failure - PR was merged even though required tests failed

What Needs Investigation

1. Which specific JS test(s) failed?
2. Are JS tests checking MCP config format?
3. Does the JS codebase have schema validation that needs updating?
4. Why did the CI allow the merge despite the failed js job?

Recommended Actions

Immediate (Priority 1)

• Run tests locally: cd actions/setup/js && npm ci && npm test to reproduce failure
• Check main branch CI: Verify if subsequent commits are also failing
• Identify failing test: Review test output to find which test(s) failed
• Consider revert: If tests cannot be quickly fixed, revert PR Fix MCP Gateway schema validation for stdio servers with container field #10421

Short-term (Priority 2)

• Fix JS tests: Update tests to work with new MCP Gateway format
• Update schema validation: Ensure JS runtime validation supports new format
• Verify lock files: Confirm regenerated lock files work correctly
• Add regression tests: Ensure both formats are tested going forward

Long-term (Priority 3)

• Fix CI configuration: Make js job a required check that blocks merges
• Add pre-merge validation: Ensure all tests pass before allowing merge
• Improve test coverage: Add integration tests for MCP config format changes
• Document format migration: Add migration guide for MCP config format changes

Prevention Strategies

CI Configuration

1. Required Checks: Ensure the js job is marked as required in branch protection rules
2. Merge Blocking: Configure GitHub to block merges when required checks fail
3. Status Checks: Add branch protection rule requiring js job to pass

Testing Strategy

1. Format Validation Tests: Add explicit tests for MCP config format compatibility
2. Schema Version Tests: Test both legacy and new MCP Gateway formats
3. Integration Tests: Add end-to-end tests that compile and validate workflows
4. Pre-commit Hooks: Run JS tests before allowing commits

Code Review

1. Format Changes: Require extra review for changes to MCP config format
2. Lock File Changes: Verify regenerated lock files are valid
3. Test Updates: Ensure both Go and JS tests are updated together

AI Team Self-Improvement

Add the following to AGENTS.md or developer instructions:

### MCP Config Format Changes

**CRITICAL**: When modifying MCP Gateway config format in Go code:

1. **Update ALL tests**: Both Go (`pkg/workflow/*_test.go`) AND JS (`actions/setup/js/*.test.cjs`)
2. **Run full test suite**: Execute `make test` (Go) AND `cd actions/setup/js && npm test` (JS)
3. **Verify lock files**: After regenerating lock files, spot-check that MCP configs use correct format
4. **Check CI status**: Ensure ALL jobs pass before requesting merge
5. **Never merge with failures**: If any job fails, fix it before merging

**MCP Config Format Evolution**:
- Legacy format: `{"command": "docker", "args": ["run", "--rm", "-i", "container"]}`
- MCP Gateway v1.0.0: `{"container": "container:version"}`
- JS tests must support BOTH formats during migration

Historical Context

This is a new failure pattern. Previous investigations in the cache show no similar issues with JS tests failing due to MCP config format changes.

Related Issues: None found

Run URL

https://github.com/githubnext/gh-aw/actions/runs/21089437610

---

Investigation completed: 2026-01-17T05:50:41Z
Investigator: CI Failure Doctor (Automated)

AI generated by CI Failure Doctor

To add this workflow in your repository, run gh aw add githubnext/agentics/workflows/ci-doctor.md@ea350161ad5dcc9624cf510f134c6a9e39a6f94d. See usage guide.