[Schema Consistency] Schema Consistency Check - Workflow Compiler Pipeline Trace

[github-actions[bot]]

This analysis used a new detection strategy: Workflow Compiler Pipeline Trace - following how frontmatter flows through parsing → validation → code generation. Found 8 inconsistencies including critical timeout field duplication and missing validation.

Summary

• Inconsistencies Found: 8
• Categories Analyzed: Schema, Parser/Compiler, Documentation, Real Workflows
• Strategy Used: Workflow Compiler Pipeline Trace (NEW)
• Effectiveness: HIGH - 8 findings across all severity levels
• Workflow Run: §21854686060

Breakdown by Severity:

• 🔴 Critical: 2 issues (timeout field duplication, missing labels validation)
• 🟡 Medium: 3 issues (documentation gaps, error message consistency)
• 🟢 Low: 3 issues (architecture differences, workflow typo)

---

Critical Issues

1. 🔴 timeout_minutes vs timeout-minutes Field Duplication

Problem: Schema defines BOTH timeout-minutes (preferred) and timeout_minutes (deprecated), causing user confusion.

Evidence:

• Schema location: pkg/parser/schemas/main_workflow_schema.json
• Code usage: timeout-minutes (28 references), timeout_minutes (3 references)
• Real workflows: 142 workflows use timeout-minutes
• Documentation: docs/src/content/docs/reference/frontmatter.md:382 mentions deprecation

Impact: Users may not know which field to use. Schema validation accepts both, creating ambiguity about the "correct" approach.

Recommendation:

1. Remove timeout_minutes from schema
2. Add runtime warning if timeout_minutes detected in frontmatter
3. Update any remaining code references to use timeout-minutes

---

2. 🔴 Labels Field Missing Validation

Problem: The labels field is defined in schema with constraints but not validated in the compiler pipeline.

Evidence:

• Schema: Defines labels as array of strings with minLength: 1 constraint
• Struct: FrontmatterConfig.Labels field exists (pkg/workflow/frontmatter_types.go:91)
• Usage: 72 real workflows use labels
• Missing: No validation in validateWorkflowData() pipeline

Impact: Invalid label values (empty strings, non-strings, etc.) could pass through compilation without error, causing runtime issues.

Recommendation: Add labels validation in pkg/workflow/compiler.go validateWorkflowData() function:

// Validate labels if present
if len(workflowData.Labels) > 0 {
    for i, label := range workflowData.Labels {
        if len(strings.TrimSpace(label)) == 0 {
            return formatCompilerError(markdownPath, "error", 
                fmt.Sprintf("labels[%d]: label cannot be empty", i), nil)
        }
    }
}

---

Documentation Gaps

3. 🟡 command Field Purpose Unclear

Problem: Top-level command field exists in schema but is undocumented and barely used.

Evidence:

• Schema: Defines command as string with description "Command name for the workflow"
• Code: 2 legacy references found:
  • pkg/workflow/frontmatter_extraction_security.go: accesses agentObj["command"]
  • pkg/workflow/frontmatter_extraction_yaml.go: accesses onMap["command"] (marked legacy)
• Workflows: 0 occurrences in real workflows
• Documentation: Not mentioned in docs/src/content/docs/reference/frontmatter.md

Impact: Schema pollution. Unclear whether this is deprecated, reserved for future use, or should be removed.

Recommendation: Choose one:

• Option A: Document the field if it's still supported
• Option B: Mark as deprecated in schema with migration notice
• Option C: Remove from schema entirely if truly unused

---

4. 🟡 post-steps Uses Indirect Access Pattern

Problem: post-steps field is well-documented and widely used, but accessed indirectly in compiler (not via frontmatter["post-steps"]).

Evidence:

• Schema: Defines post-steps with oneOf (object | array) structure
• Documentation: Fully documented in frontmatter.md
• Usage: 34 real workflows use post-steps
• Code: 5 references but accessed through YAML template generation, not direct map lookup

Impact: LOW - Works correctly, but confusing for developers maintaining the compiler. They might expect direct access pattern.

Recommendation: Add code comment in compiler explaining why post-steps uses indirect access:

// Note: post-steps field is processed during YAML template generation
// rather than via direct frontmatter["post-steps"] access

---

5. 🟡 Error Message Inconsistency

Problem: Compiler error messages use inconsistent phrasing across different validation functions.

Evidence:
Different error patterns found:

• "X is required" vs "X must have Y" vs "X not found"
• "invalid X" vs "not valid"
• "configuration is required" vs "must be an object"

Examples:

"safe-outputs configuration is required"
"artifact upload must have a name"  
"script not found in registry"
"setup action reference is required but could not be resolved"
"invalid action-mode feature flag '%s'. Must be 'dev', 'release', or 'script'"

Impact: Inconsistent user experience when reading error messages. Makes errors harder to pattern-match in documentation.

Recommendation:

1. Establish error message style guide (e.g., always use "X is required" for missing fields)
2. Refactor error messages for consistency across compiler
3. Document error message patterns in contributor guide

---

Schema/Code Alignment Issues

6. 🟢 Schema Fields Use Multiple Access Patterns

Finding: Many schema fields are not accessed via frontmatter["field"] pattern but through other mechanisms.

Fields using indirect access:

• cache, concurrency, container, env, environment
• imports, jobs, secrets, services, steps, run-name, runs-on

Access mechanisms:

• Helper functions (extractString(), extractMap())
• YAML template generation
• Struct unmarshaling (FrontmatterConfig)

Impact: None - this is an architectural pattern, not a bug. Just different access styles.

Recommendation: Add architecture documentation comment explaining the two access patterns and when each is used.

---

7. 🟢 Invalid Field in Real Workflow

Finding: Workflow .github/workflows/copilot-pr-nlp-analysis.md contains invalid try: field (empty value).

Evidence:

• Field: try: (not in schema)
• Appears to be typo or leftover from editing
• Workflow still works (field is ignored)

Impact: Pollutes frontmatter but doesn't break workflow execution.

Recommendation: Remove try: field from workflow file.

---

8. 🟢 Validation Pipeline Order Not Documented

Finding: The validateWorkflowData() function executes 11 validation steps in specific order, but this is not documented.

Validation order (pkg/workflow/compiler.go:116-199):

1. Expression safety
2. Runtime-import files
3. Feature flags
4. Action mode override
5. Dangerous permissions
6. Agent file
7. Sandbox config
8. Safe-outputs target
9. Safe-outputs allowed-domains
10. Network allowed domains
11. Network firewall config

Impact: Developers may not understand validation dependencies or why order matters.

Recommendation: Add documentation comment in compiler.go explaining validation order and any dependencies between steps.

---

Recommendations Summary

Immediate Actions (High Priority)

1. ✅ Remove timeout_minutes from schema - Eliminate duplication confusion
2. ✅ Add labels validation - Prevent invalid label values
3. ✅ Document or remove command field - Clarify schema intent

Medium-Term Improvements

4. 📝 Standardize error messages - Improve user experience
5. 📝 Add compiler architecture docs - Help maintainers understand access patterns
6. 📝 Document validation pipeline order - Clarify dependencies

Low Priority Cleanup

7. 🧹 Remove try: from workflow - Clean up invalid field
8. 🧹 Add code comments for indirect access - Clarify post-steps pattern

---

Strategy Performance

Strategy Used: Workflow Compiler Pipeline Trace (NEW)

Approach: Traced frontmatter data flow through actual compilation pipeline:

• Parsing → Validation → Code Generation
• Direct vs indirect field access patterns
• Validation ordering and dependencies
• Error message consistency
• Real workflow field usage statistics

Findings: 8 inconsistencies (2 critical, 3 medium, 3 low)

Effectiveness: HIGH - This strategy successfully identified practical implementation gaps that impact users (timeout duplication) and missing validations (labels) that earlier strategies missed.

Should Reuse: YES - Excellent for finding validation gaps and data flow inconsistencies. Complements field enumeration strategies well.

---

Next Steps

• Update schema to remove timeout_minutes field
• Add labels validation in compiler pipeline
• Resolve command field status (document or deprecate)
• Establish error message style guide
• Add compiler architecture documentation
• Document validation pipeline order
• Clean up invalid try: field in workflow
• Add code comments for indirect access patterns

Analysis Date: 2026-02-10
Strategy: Workflow Compiler Pipeline Trace (NEW)
Total Strategies in Cache: 4

AI generated by Schema Consistency Checker

• expires on Feb 17, 2026, 6:53 AM UTC