Skip to content

conformance: deconflict expected.error_code field semantics #285

@jithinraj

Description

@jithinraj

Problem

The expected.error_code field in workflow conformance fixtures (and other conformance suites) carries two distinct meanings:

  1. Validation error code (normative) - The code a validator MUST emit when rejecting invalid input (e.g., E_WORKFLOW_SELF_PARENT)
  2. Runtime error code (informational) - The code that might appear at runtime for edge cases that pass structural validation

This overload makes it ambiguous whether a test harness should assert on error_code for edge-case fixtures where expected.valid is true but an error_code is present for documentation.

Currently the JSON Schema enforces error_code is required when valid: false (via allOf/if/then), which is correct. But the field name itself suggests it's always a validation output, which could confuse cross-language implementors.

Proposal

Split into two distinct fields with a deprecation window:

Phase 1: Add new fields (next conformance version bump)

  • validation_error_code - Normative. Required when valid: false. Test harnesses MUST assert this.
  • meta.runtime_error_code - Informational. Optional. Documents what code might appear at runtime. Test harnesses MUST NOT assert this.

Phase 2: Deprecation (1 release cycle)

  • Accept both error_code (legacy) and validation_error_code (new) for 1 release
  • Fixture schema validates either field is present when valid: false

Phase 3: Remove legacy field

  • Remove error_code from fixture schema
  • All conformance suites use validation_error_code

Affected Fixture Suites

  • specs/conformance/fixtures/workflow/ (valid.json, invalid.json, edge-cases.json)
  • specs/conformance/fixtures/agent-identity/
  • specs/conformance/fixtures/attribution/
  • specs/conformance/fixtures/dispute/
  • specs/conformance/fixtures/bundle/
  • specs/conformance/fixtures/purpose/

Acceptance Criteria

  • New field names documented in fixture README
  • JSON Schema updated with oneOf for transition period
  • All test harnesses (TS, Go) updated to read new field
  • Legacy error_code removed after 1 release cycle
  • Cross-language test harnesses updated

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions