Cross-Schema Consistency Analysis - 8 Issues Found (Dec 27)

[github-actions[bot]]

Executed Strategy-004 (Cross-Schema Consistency & Type Analysis) focusing on comparing the three schema files (main_workflow_schema.json, included_file_schema.json, mcp_config_schema.json) to identify structural inconsistencies, definition duplication, and feature parity gaps.

Key Results:

• Inconsistencies Found: 8 (5 critical, 3 moderate)
• New Issues Discovered: 3 (firewall missing in included, enum vs const inconsistency, undocumented tool restrictions)
• Confirmed Previous Issues: 2 (engine field duplication, permissions shortcuts missing)
• Positive Validations: 6 areas of excellent consistency
• Overall Assessment: Significant inter-schema maintenance burden and feature parity gaps identified

This analysis reveals that while individual schemas are well-structured, cross-schema consistency needs improvement to reduce maintenance burden and ensure feature parity between main and included workflows.

Schema Comparison Metrics

• Main Schema: 39 top-level properties, 4 $defs
• Included Schema: 15 top-level properties, 3 $defs
• MCP Schema: 12 top-level properties, 0 $defs
• Shared Fields (Main ↔ Included): 13 fields

Critical Issues

1. Engine Field: $ref vs Inline Duplication Risk

Priority: CRITICAL | Category: Schema Structure / Maintenance Burden

Problem:

• Main Schema: Uses $ref: "#/$defs/engine_config" to reference centralized definition
• Included Schema: Uses inline definition with identical oneOf structure
• Complete duplication of engine_config (env, id, max-turns, model, steps, version)

Impact:

• Every engine schema change must be manually duplicated across two files
• Risk of schema drift if one is updated but not the other
• Violates DRY principle
• Maintenance burden scales with schema complexity

Recommendation:
Change included schema to use $ref: "#/$defs/engine_config" like main schema.

---

2. Permissions: Missing String Shortcuts in Included Schema

Priority: HIGH | Category: UX Consistency / Feature Parity

Problem:

• Main Schema: Supports 2 forms via oneOf:
  • String shortcuts: "read-all", "write-all", "read", "write"
  • Object form with 15 granular permissions
• Included Schema: Only supports object form (no shortcuts)

Impact:

• Users cannot write permissions: read-all in included files
• Must use verbose object syntax even for simple cases
• Inconsistent UX between main workflows and included files

Example:

# Main workflow - WORKS
permissions: read-all

# Included file - FAILS validation
permissions: read-all  # Error: must be object

Recommendation:
Add string enum shortcuts to included schema permissions.oneOf to match main schema's UX.

---

3. Network: Missing Firewall Configuration in Included Schema

Priority: HIGH | Category: Security / Feature Parity

Problem:

• Main Schema: network.oneOf[1] has properties: allowed, firewall
• Included Schema: network.oneOf[1] has property: allowed only
• Firewall configuration completely unavailable in included files

Impact:

• Included files cannot configure advanced network security rules
• Modular workflows lose critical security feature
• Prevents defense-in-depth architecture with included files

Recommendation:
Add firewall property to included schema network field to restore feature parity.

---

4. Tools Field: 9 of 13 Tools Missing from Included Schema

Priority: MODERATE | Category: Feature Parity / Documentation

Main Schema: 13 tool configuration properties
Included Schema: 4 tool configuration properties (bash, cache-memory, github, repo-memory)

Missing Tools from Included Schema:
agentic-workflows, edit, playwright, safety-prompt, serena, startup-timeout, timeout, web-fetch, web-search

Impact:

• Included files cannot configure 9 tools
• Unclear if restriction is intentional design or oversight
• No documentation explaining why tools are limited

Recommendation:

1. If intentional: Document in schema description why included files have restricted tools
2. If not intentional: Add missing 9 tools to included schema

---

5. Safe-Outputs: 37 Properties vs 1 Property

Priority: MODERATE | Category: Feature Parity / Intentional Design

Problem:

• Main Schema: 37 safe-output properties (issues, pull-requests, discussions, labels, comments, etc.)
• Included Schema: 1 safe-output property (jobs only)

Impact:

• Included files can only configure job-level safe-outputs
• Cannot configure safe-outputs for GitHub operations
• May be intentional design but undocumented

Recommendation:
Add schema description explaining included files are job-scoped only, OR expand safe-output operations.

---

Moderate Issues

6. Schema Style: enum vs const Inconsistency

Priority: LOW | Category: Schema Quality

• Main Schema: Uses "enum": ["defaults"] for single-value enums
• Included Schema: Uses "const": "defaults" for single-value strings

Functionally equivalent but stylistically inconsistent. Recommendation: Standardize on enum.

---

7. $defs Organization

Shared $defs: http_mcp_tool, stdio_mcp_tool ✓ (identical across schemas)

Main-Only $defs: engine_config (should be shared), github_token

Included-Only $defs: safe_job

Observation: MCP tool definitions show excellent DRY discipline, while engine_config is duplicated.

---

8. Included-Specific Fields

Included schema has 2 unique fields not in main schema:

• applyTo - Filters which jobs use the included file
• inputs - Parameterization for included files

Analysis: These are legitimately included-file-specific features (expected difference).

---

Positive Findings

✓ MCP Tool Definitions: Byte-for-byte identical across schemas - exemplary DRY discipline

✓ Field Naming: Consistent kebab-case naming convention across all three schemas

✓ No Broken References: All $ref references resolve correctly

✓ 13 Shared Fields: Properly shared between main and included

✓ Engine Structure: oneOf structure identical (despite $ref vs inline)

✓ Network Patterns: Domain allowlist patterns consistent where they exist

---

Recommendations

Immediate Actions (Priority 1)

1. Fix Engine Duplication - Change included schema to use $ref: "#/$defs/engine_config"
2. Add Permissions Shortcuts - Add string enum oneOf option to included schema
3. Add Network Firewall - Add firewall property to included schema network field

Documentation Actions (Priority 2)

4. Document Tool Restrictions - Explain why only 4 tools in included schema
5. Document Safe-Outputs Scope - Explain job-scoped limitation for included files

Schema Quality (Priority 3)

6. Standardize enum vs const - Use enum consistently
7. Add Cross-Schema Comments - Document main vs included feature differences

---

Strategy Performance

• Effectiveness: VERY HIGH
• Unique Value: Discovers inter-schema issues invisible to single-schema validation
• Recommendation: Continue using Strategy-004 every 4-5 analyses

Key Insights

1. Maintenance Burden: Engine field duplication creates highest ongoing risk
2. Feature Parity Gaps: Multiple features restricted in included schema without documentation
3. DRY Discipline Inconsistent: MCP definitions perfect, engine definition duplicated
4. Intentionality Unclear: Cannot determine if restrictions are design choices or oversights

---

Files Analyzed:

• pkg/parser/schemas/main_workflow_schema.json (39 properties, 4 $defs)
• pkg/parser/schemas/included_file_schema.json (15 properties, 3 $defs)
• pkg/parser/schemas/mcp_config_schema.json (12 properties, 0 $defs)

AI generated by Schema Consistency Checker

[pelikhan]

/plan

[github-actions[bot]]

🎉 Yo ho ho! Plan Command found the treasure and completed successfully! ⚓💰

[github-actions[bot]]

Plan Created

I've created a tracking issue with 5 actionable sub-issues to address the cross-schema consistency problems identified in the analysis.

Summary

Parent Issue: Fix cross-schema consistency issues in workflow schemas

Sub-Issues Created:

1. Fix engine field duplication (CRITICAL) - Eliminate maintenance burden from $ref vs inline duplication
2. Add permissions string shortcuts (HIGH) - Restore UX consistency for included files
3. Add network firewall configuration (HIGH) - Restore security feature parity
4. Document tool restrictions (MODERATE) - Clarify intentional design limitations
5. Standardize enum vs const usage (LOW) - Improve schema quality consistency

These issues target the highest priority problems identified in the analysis, focusing on:

• Maintenance burden reduction (engine field duplication)
• Feature parity restoration (permissions, network firewall)
• Documentation clarity (tool restrictions)
• Code quality (enum/const standardization)

The remaining issues (safe-outputs scope, $defs organization) can be addressed in future iterations as they represent more complex architectural decisions.

---

AI generated by Planning Assistant

AI generated by Plan Command for discussion #7935