[smoke-detector] 🔍 Smoke Test Investigation - Smoke Claude: Invalid MCP Configuration Argument

# 🔍 Smoke Test Investigation - Run #18769543228

## Summary
The Smoke Claude workflow failed immediately when Claude Code CLI attempted to parse the `--mcp-config` argument. The CLI received **inline JSON** but interpreted it as a **file path**, causing it to exit with "MCP config file not found" error before any agent execution could begin.

## Failure Details
- **Run**: [#18769543228](https://github.com/githubnext/gh-aw/actions/runs/18769543228)
- **Commit**: 24bb96c15a83f0c0fac4c0726a7486cdd826cf88
- **Branch**: copilot/update-engines-cli-arguments
- **Trigger**: workflow_dispatch
- **Duration**: 1.8 minutes
- **Failed Job**: agent (21s duration)

## Root Cause Analysis

### Primary Error
```
Error: Invalid MCP configuration:
MCP config file not found: /home/runner/work/gh-aw/gh-aw/{mcpServers:{github:{command:docker,args:[run,-i,--rm,-e,GITHUB_PERSONAL_ACCESS_TOKEN,-e,GITHUB_READ_ONLY=1,-e,GITHUB_TOOLSETS=default,ghcr.io/github/github-mcp-server:v0.19.1],env:{GITHUB_PERSONAL_ACCESS_TOKEN:ghs***}},safe_outputs:{...}}}
```

The error shows Claude Code CLI attempted to use the **entire JSON string as a file path** at pkg/workflow/claude_engine.go:238:17 (cli.js:3744).

### Investigation Findings

**The Problem**: CLI Argument Interpretation Mismatch

The commit 24bb96c15a83f0c0fac4c0726a7486cdd826cf88 made changes to JSON compaction:
- **Modified** `validateAndCompactJSON` to remove backslash escaping from `${VAR}` patterns
- **Intent**: Correct for single-quoted CLI arguments (no shell expansion needed)
- **Effect on Claude**: Claude Code's `--mcp-config` expects a **file path**, not inline JSON

| Component | Expected Format | Actual Format | Status |
|-----------|----------------|---------------|--------|
| Claude Code --mcp-config | File path to JSON config | Inline JSON string | ❌ Mismatch |
| Copilot --additional-mcp-config | Raw JSON string | Raw JSON string | ✅ Works |

### CLI Command That Failed
```bash
claude --print \
  --mcp-config "{"mcpServers":{"github":{...},"safe_outputs":{...}}}" \
  --allowed-tools "..." \
  --debug --verbose \
  --permission-mode bypassPermissions \
  "$(cat /tmp/gh-aw/aw-prompts/prompt.txt)"
```

Claude Code tried to open `/home/runner/work/gh-aw/gh-aw/{mcpServers:{...}}` as a file, which obviously doesn't exist.

## Failed Jobs and Errors

### Job Sequence
1. ✅ **activation** - succeeded (9s)
2. ❌ **agent** - failed (21s) - Claude Code CLI couldn't parse MCP config argument
3. ❌ **create_issue** - failed (5s) - Dependent on agent job
4. ⏭️ **missing_tool** - skipped
5. ⏭️ **detection** - skipped

### Error Details

**Location**: /tmp/gh-aw/agent-stdio.log:5-8

```
Error: Invalid MCP configuration:
MCP config file not found: /home/runner/work/gh-aw/gh-aw/{mcpServers:...}
MCP config file not found: /home/runner/work/gh-aw/gh-aw/{mcpServers:...}  
MCP config file not found: /home/runner/work/gh-aw/gh-aw/{mcpServers:...}
```

**Context**: Claude Code CLI validates the `--mcp-config` argument during initialization and expects it to be a file path, not raw JSON content.

## Commit Analysis

**Commit**: 24bb96c15a83f0c0fac4c0726a7486cdd826cf88  
**Author**: copilot-swe-agent[bot]  
**Message**: "Fix JSON escaping issue for environment variables"

**Files Changed** (12 lines total):
- `pkg/workflow/engine_shared_helpers.go` (+8 lines)
- `pkg/workflow/mcp-config.go` (+4 lines)

**Changes in validateAndCompactJSON** (engine_shared_helpers.go:387-411):
```go
// Look ahead - if next char is $, skip the backslash (for env var refs)
if i+1 < len(jsonStr) && jsonStr[i+1] == '$' && inString {
    // Skip the backslash - we're in a single-quoted context so no escaping needed
    continue
}
```

**Intent**: The change was correct for Copilot (which uses `--additional-mcp-config` and accepts raw JSON), but broke Claude Code which interprets `--mcp-config` as a **file path** parameter.

## Comparison with Similar Issues

This is similar to **issue #2262** (Copilot Base64 vs JSON), but with a different root cause:

| Issue | Engine | Problem | Root Cause |
|-------|--------|---------|------------|
| #2262 | Copilot | Expected JSON, got base64 | Workflow not recompiled after code changes |
| **This Issue** | **Claude** | **Expected file path, got JSON** | **Different CLI argument semantics between engines** |

## Recommended Actions

### Critical Priority
- [ ] **Switch Claude to file-based MCP config**
  ```yaml
  # Instead of:
  claude --mcp-config "$MCP_JSON" ...
  
  # Use:
  echo "$MCP_JSON" > /tmp/mcp-config.json
  claude --mcp-config /tmp/mcp-config.json ...
  ```
  **Rationale**: Match Claude Code's expected behavior

- [ ] **Verify Claude Code CLI --mcp-config documentation**
  - Confirm it only accepts file paths
  - Check if there's an alternative flag for inline JSON
  - **Rationale**: Ensure we're using the correct CLI interface

### High Priority
- [ ] **Review BuildMCPConfigJSON usage across engines**
  ```bash
  grep -r "BuildMCPConfigJSON" pkg/workflow/
  ```
  - Identify which engines use inline JSON vs file paths
  - Document engine-specific requirements
  - **Rationale**: Prevent similar issues with other engines

- [ ] **Add engine-specific integration tests**
  - Test Claude with file-based MCP config
  - Test Copilot with inline JSON MCP config
  - **Rationale**: Catch CLI argument incompatibilities early

### Medium Priority
- [ ] **Document MCP config approach for each engine**
  ```markdown
  - Claude: File path via --mcp-config
  - Copilot: Inline JSON via --additional-mcp-config  
  - Codex: TOML config file
  - GenAIScript: File path via --mcp-config
  ```

- [ ] **Consider unified approach**
  - Evaluate using file-based config for ALL engines
  - Would eliminate escaping/quoting issues
  - **Rationale**: Simplify and standardize

## Prevention Strategies

1. **Engine-Specific CLI Tests**: Add tests that invoke actual CLI tools with MCP config
   ```go
   func TestClaudeMCPConfig(t *testing.T) {
       // Write config to file
       // Invoke claude CLI with --mcp-config /path/to/file
       // Verify it starts without errors
   }
   ```

2. **Documentation**: Create engine comparison matrix documenting CLI differences

3. **Code Review Checklist**: Add item: "Does this affect MCP config for any engine?"

4. **Smoke Tests in PR CI**: Run smoke tests automatically when engine code changes

## Technical Details

### Expected Behavior (After Fix)
```bash
# Write MCP config to file
cat > /tmp/mcp-config.json << 'EOF'
{
  "mcpServers": {
    "github": { ... },
    "safe_outputs": { ... }
  }
}
EOF

# Pass file path to Claude
claude --mcp-config /tmp/mcp-config.json ...
```

### Actual Behavior (Current)
```bash
# Attempt to pass inline JSON
claude --mcp-config '{"mcpServers":{...}}' ...

# Claude interprets entire string as file path
# Error: MCP config file not found: /home/runner/work/gh-aw/gh-aw/{mcpServers:...}
```

### Engine MCP Config Comparison

| Engine | CLI Flag | Accepts | Format |
|--------|----------|---------|--------|
| Claude Code | `--mcp-config` | File path | JSON file |
| Copilot | `--additional-mcp-config` | Inline JSON | JSON string |
| Codex | Config file | File path | TOML file |
| GenAIScript | `--mcp-config` | File path | JSON file |

## Related Information

- **Workflow Source**: `.github/workflows/smoke-claude.md`
- **Engine Code**: `pkg/workflow/claude_engine.go`
- **Shared Helper**: `pkg/workflow/engine_shared_helpers.go:387-411`
- **Related Issue**: #2262 (Copilot base64 vs JSON config)

## Pattern Storage

**Investigation saved to**: `/tmp/gh-aw/cache-memory/investigations/2025-10-24-18769543228.json`  
**Pattern ID**: CLAUDE_INVALID_MCP_CONFIG_ARG

---

**Investigation Metadata:**
- **Investigator**: Smoke Detector
- **Investigation Run**: [#18769573968](https://github.com/githubnext/gh-aw/actions/runs/18769573968)
- **Pattern ID**: CLAUDE_INVALID_MCP_CONFIG_ARG
- **Severity**: Critical
- **Is Flaky**: No
- **Related Patterns**: COPILOT_BASE64_MCP_CONFIG

**Labels**: smoke-test, investigation, claude, configuration, mcp, critical




> AI generated by [Smoke Detector - Smoke Test Failure Investigator](https://github.com/githubnext/gh-aw/actions/runs/18769573968)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[smoke-detector] 🔍 Smoke Test Investigation - Smoke Claude: Invalid MCP Configuration Argument #2267

🔍 Smoke Test Investigation - Run #18769543228

Summary

Failure Details

Root Cause Analysis

Primary Error

Investigation Findings

CLI Command That Failed

Failed Jobs and Errors

Job Sequence

Error Details

Commit Analysis

Comparison with Similar Issues

Recommended Actions

Critical Priority

High Priority

Medium Priority

Prevention Strategies

Technical Details

Expected Behavior (After Fix)

Actual Behavior (Current)

Engine MCP Config Comparison

Related Information

Pattern Storage

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Component	Expected Format	Actual Format	Status
Claude Code --mcp-config	File path to JSON config	Inline JSON string	❌ Mismatch
Copilot --additional-mcp-config	Raw JSON string	Raw JSON string	✅ Works

Issue	Engine	Problem	Root Cause
#2262	Copilot	Expected JSON, got base64	Workflow not recompiled after code changes
This Issue	Claude	Expected file path, got JSON	Different CLI argument semantics between engines

Engine	CLI Flag	Accepts	Format
Claude Code	`--mcp-config`	File path	JSON file
Copilot	`--additional-mcp-config`	Inline JSON	JSON string
Codex	Config file	File path	TOML file
GenAIScript	`--mcp-config`	File path	JSON file

[smoke-detector] 🔍 Smoke Test Investigation - Smoke Claude: Invalid MCP Configuration Argument #2267

Description

🔍 Smoke Test Investigation - Run #18769543228

Summary

Failure Details

Root Cause Analysis

Primary Error

Investigation Findings

CLI Command That Failed

Failed Jobs and Errors

Job Sequence

Error Details

Commit Analysis

Comparison with Similar Issues

Recommended Actions

Critical Priority

High Priority

Medium Priority

Prevention Strategies

Technical Details

Expected Behavior (After Fix)

Actual Behavior (Current)

Engine MCP Config Comparison

Related Information

Pattern Storage

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions