[Schema Consistency] Type Inconsistency: version Fields Reject Numeric Values Despite Schema

[github-actions[bot]]

Overview

Analysis of runtime type behavior versus schema type definitions revealed 2 critical inconsistencies where numeric version values are silently ignored despite schema advertising string | number support. Additionally, confirmed 1 previously-reported issue has been fixed (expires hours support).

Key Finding: Schema advertises that version fields accept numbers with examples like 20 and 3.11, but implementations only handle strings, causing user configurations to fail silently without error messages.

---

Critical Issues

🔴 Issue 1: engine.version Rejects Numeric Types

Location: pkg/workflow/engine.go:66-69

Problem: Schema defines engine.version as "type": ["string", "number"] with examples [20, 3.11], but code only accepts strings:

// Current implementation - ONLY checks string
if version, hasVersion := engineObj["version"]; hasVersion {
    if versionStr, ok := version.(string); ok {
        config.Version = versionStr
    }
}

Schema Evidence:

{
  "type": ["string", "number"],
  "examples": ["beta", "stable", 20, 3.11]
}

Impact: Users following schema examples with engine.version: 20 have configuration silently ignored with no error/warning. Falls back to default version, causing confusion when version-specific features don't work.

Recommended Fix: Apply the pattern already used for max-turns field (same file, lines 81-87):

switch v := version.(type) {
case string:
    config.Version = v
case int, int64, uint64:
    config.Version = fmt.Sprintf("%d", v)
case float64:
    config.Version = fmt.Sprintf("%g", v)
}

---

🔴 Issue 2: MCP Version Fields Reject Numeric Types (2 Locations)

Locations:

• pkg/parser/mcp.go:372 (GitHub MCP server)
• pkg/parser/mcp.go:447 (Playwright container)

Problem: Multiple MCP version fields advertise string | number but only handle strings.

Example (GitHub MCP, line 372):

if version, exists := toolConfig["version"]; exists {
    if versionStr, ok := version.(string); ok {  // ❌ Only string!
        dockerImage := "ghcr.io/github/github-mcp-server:" + versionStr
    }
}

Schema Examples:

// GitHub MCP
{"type": ["string", "number"], "examples": ["v1.0.0", "latest", 20, 3.11]}

// Playwright  
{"type": ["string", "number"], "examples": ["v1.41.0", 1.41, 20]}

Impact: Configurations like tools.playwright.version: 1.41 silently ignored. Particularly problematic for version pinning in production workflows.

Recommended Fix: Create reusable helper:

func parseVersionValue(version any) string {
    switch v := version.(type) {
    case string: return v
    case int, int64, uint64: return fmt.Sprintf("%d", v)
    case float64: return fmt.Sprintf("%g", v)
    default: return ""
    }
}

---

Positive Findings ✅

Previously Reported Issue Fixed

The expires field hours (h/H) support, reported as missing in 2025-12-21 analysis, is now properly implemented in pkg/workflow/config_helpers.go:229-240. This demonstrates the consistency checking workflow is effective - issues get identified and fixed.

Excellent Type Coercion Quality

The codebase demonstrates high-quality type handling in most areas:

• Consistent type switch patterns across 15+ parsing functions
• Overflow protection for uint64 → int conversions
• Float truncation warnings
• Zero-value fallbacks for invalid types

Well-Implemented Examples:

• max-turns field (engine.go:81-87) ✅
• expires field (config_helpers.go:177-202) ✅
• ParseIntFromConfig helper (config_helpers.go:260-298) ✅

The issue is localized gaps, not systemic poor design.

---

Recommendations

Immediate Actions (Priority: HIGH)

1. Fix engine.version (engine.go:66-69) - 10 min
2. Fix GitHub MCP version (mcp.go:372) - 10 min
3. Fix Playwright version (mcp.go:447) - 10 min
4. Add regression tests for numeric version values - 30 min

Total Critical Fixes: ~1 hour

Follow-Up Actions

• Audit other version fields (serena, awf, etc.)
• Create parseVersionValue() helper for DRY
• Add comprehensive type validation test suite
• Document version field conventions

---

Risk Assessment

Overall Risk: MEDIUM-HIGH

Why This Matters:

• Silent failures without error messages
• Schema examples actively encourage numeric format
• Version pinning fails unexpectedly
• Low detectability - users may not realize configuration ignored

Mitigation:

• Localized to 3 specific code locations
• Straightforward fixes using existing patterns
• No architectural changes needed
• Low risk of introducing regressions

---

Analysis Details

Strategy: Strategy-017 (Runtime Type Behavior vs Schema Type Definitions)
Date: 2025-12-30
Findings: 2 critical issues (3 code locations), 1 confirmed fix
Effectiveness: Very High

Methodology:

1. Extracted all type definitions from schema using jq
2. Identified fields with "type": ["string", "number"]
3. Mapped to parser/compiler implementations
4. Compared runtime type handling vs schema promises
5. Cross-referenced with previous run findings

Files Analyzed:

• pkg/parser/schemas/main_workflow_schema.json
• pkg/workflow/engine.go
• pkg/parser/mcp.go
• pkg/workflow/config_helpers.go

---

Action Items Summary

Priority	Action	Files	Effort
🔴 HIGH	Fix engine.version type handling	engine.go:66-69	10 min
🔴 HIGH	Fix GitHub MCP version handling	mcp.go:372	10 min
🔴 HIGH	Fix Playwright version handling	mcp.go:447	10 min
🟡 MEDIUM	Add regression test cases	test files	30 min
🟡 MEDIUM	Audit other version fields	various	1 hour

Next Analysis: Scheduled in ~7-10 days to validate fixes

AI generated by Schema Consistency Checker

[pelikhan]

/plan

[github-actions[bot]]

🎉 Yo ho ho! Plan Command found the treasure and completed successfully! ⚓💰