Bug: Archive validation fails despite requirement headers containing SHALL/MUST

[PeakMNA]

Bug Report: Archive Validation Fails Despite Requirement Headers Containing SHALL/MUST

Description

The openspec archive command validation incorrectly rejects requirement headers that contain "SHALL" or "MUST" keywords, even when these keywords are explicitly present in the requirement header text.

Environment

• OpenSpec Version: 0.9.2
• Node.js Version: 22 LTS
• OS: macOS (Darwin 25.0.0)
• Project: ideator

Expected Behavior

When a requirement header contains "SHALL" or "MUST" (e.g., ### Requirement: Circuit Breaker State Management SHALL be implemented), the validation should pass and allow archiving.

Actual Behavior

The validation rejects the header with the error:

✗ ADDED "Circuit Breaker State Management SHALL be implemented" must contain SHALL or MUST

The error message literally shows the header contains "SHALL", yet claims it doesn't contain SHALL or MUST.

Steps to Reproduce

1. Create an OpenSpec change with delta specs
2. Add requirement headers with SHALL/MUST keywords:

   ## ADDED Requirements
   
   ### Requirement: Circuit Breaker State Management SHALL be implemented
   **ID**: REQ-CB-001
   **Priority**: P1 (High)
   
   The system MUST implement a circuit breaker with three states.
   
   #### Scenario: Normal operation
   **Given** the circuit breaker is in CLOSED state
   **When** a request is made
   **Then** the request is executed normally

3. Run openspec validate [change-id] --strict
4. Observe validation error despite SHALL being present
5. Try to archive: openspec archive [change-id] --yes
6. Same validation error blocks archiving

Attempted Workarounds

❌ Using --no-validate flag

openspec archive prevent-application-crashes --yes --no-validate

Result: Still enforces validation despite the flag, same errors appear

❌ Using --skip-specs flag

openspec archive prevent-application-crashes --yes --skip-specs

Result: Still enforces validation, same errors appear

✅ Manual archive (workaround used)

mkdir -p openspec/changes/archive/YYYY-MM-DD-[change-id]
cp -r openspec/changes/[change-id]/* openspec/changes/archive/YYYY-MM-DD-[change-id]/
rm -rf openspec/changes/[change-id]

Validation Examples

All 27 requirement headers in our change contain SHALL or MUST:

Circuit Breaker Spec (5 requirements):

• ✅ ### Requirement: Circuit Breaker State Management SHALL be implemented
• ✅ ### Requirement: AI Service MUST have Circuit Breaker
• ✅ ### Requirement: Redis MUST have Circuit Breaker
• ✅ ### Requirement: Circuit Breaker Configuration MUST be provided
• ✅ ### Requirement: Circuit Breaker Monitoring MUST be implemented

Frontend Error Handlers Spec (5 requirements):

• ✅ ### Requirement: Window Error Handler MUST be implemented
• ✅ ### Requirement: Unhandled Promise Rejection Handler MUST be implemented
• ✅ ### Requirement: Integration with React Error Boundary MUST be provided
• ✅ ### Requirement: Error Reporting Service Integration MUST be implemented
• ✅ ### Requirement: User Feedback for Errors MUST be provided

Health Monitoring Spec (8 requirements):

• ✅ ### Requirement: Liveness Probe MUST be implemented
• ✅ ### Requirement: Readiness Probe MUST be implemented
• ✅ ### Requirement: Detailed Health Check MUST be provided
• ✅ ### Requirement: Database Health Check MUST be implemented
• ✅ ### Requirement: Redis Health Check MUST be implemented
• ✅ ### Requirement: AI Service Health Check MUST be implemented
• ✅ ### Requirement: Memory Health Check MUST be implemented
• ✅ ### Requirement: Health Check Caching MUST be implemented

Process Error Handlers Spec (4 requirements):

• ✅ ### Requirement: Uncaught Exception Handler MUST be implemented
• ✅ ### Requirement: Unhandled Promise Rejection Handler MUST be implemented
• ✅ ### Requirement: Error Logging Integration MUST be provided
• ✅ ### Requirement: Non-Terminating Behavior MUST be ensured

Service Error Boundaries Spec (5 requirements):

• ✅ ### Requirement: Service Error Boundary Wrapper MUST be implemented
• ✅ ### Requirement: Error Normalization in Services MUST be implemented
• ✅ ### Requirement: Critical Service Operations MUST have fallback
• ✅ ### Requirement: Service Operation Context Tracking MUST be implemented
• ✅ ### Requirement: Retry Logic for Transient Failures MUST be implemented

All 27 headers contain SHALL or MUST, yet validation rejects all of them.

Full Validation Output

$ openspec archive prevent-application-crashes --yes

Proposal warnings in proposal.md (non-blocking):
  ⚠ Change must have a Why section. Missing required sections. Expected headers: "## Why" and "## What Changes". Ensure deltas are documented in specs/ using delta headers.

Validation errors in change delta specs:
  ✗ ADDED "Circuit Breaker State Management SHALL be implemented" must contain SHALL or MUST
  ✗ ADDED "AI Service MUST have Circuit Breaker" must contain SHALL or MUST
  ✗ ADDED "Redis MUST have Circuit Breaker" must contain SHALL or MUST
  ✗ ADDED "Circuit Breaker Configuration MUST be provided" must contain SHALL or MUST
  ✗ ADDED "Circuit Breaker Monitoring MUST be implemented" must contain SHALL or MUST
  ✗ ADDED "Window Error Handler MUST be implemented" must contain SHALL or MUST
  ✗ ADDED "Unhandled Promise Rejection Handler MUST be implemented" must contain SHALL or MUST
  ✗ ADDED "Integration with React Error Boundary MUST be provided" must contain SHALL or MUST
  ✗ ADDED "Error Reporting Service Integration MUST be implemented" must contain SHALL or MUST
  ✗ ADDED "User Feedback for Errors MUST be provided" must contain SHALL or MUST
  ✗ ADDED "Liveness Probe MUST be implemented" must contain SHALL or MUST
  ✗ ADDED "Readiness Probe MUST be implemented" must contain SHALL or MUST
  ✗ ADDED "Detailed Health Check MUST be provided" must contain SHALL or MUST
  ✗ ADDED "Database Health Check MUST be implemented" must contain SHALL or MUST
  ✗ ADDED "Redis Health Check MUST be implemented" must contain SHALL or MUST
  ✗ ADDED "AI Service Health Check MUST be implemented" must contain SHALL or MUST
  ✗ ADDED "Memory Health Check MUST be implemented" must contain SHALL or MUST
  ✗ ADDED "Health Check Caching MUST be implemented" must contain SHALL or MUST
  ✗ ADDED "Uncaught Exception Handler MUST be implemented" must contain SHALL or MUST
  ✗ ADDED "Unhandled Promise Rejection Handler MUST be implemented" must contain SHALL or MUST
  ✗ ADDED "Error Logging Integration MUST be provided" must contain SHALL or MUST
  ✗ ADDED "Non-Terminating Behavior MUST be ensured" must contain SHALL or MUST
  ✗ ADDED "Service Error Boundary Wrapper MUST be implemented" must contain SHALL or MUST
  ✗ ADDED "Error Normalization in Services MUST be implemented" must contain SHALL or MUST
  ✗ ADDED "Critical Service Operations MUST have fallback" must contain SHALL or MUST
  ✗ ADDED "Service Operation Context Tracking MUST be implemented" must contain SHALL or MUST
  ✗ ADDED "Retry Logic for Transient Failures MUST be implemented" must contain SHALL or MUST

Validation failed. Please fix the errors before archiving.
To skip validation (not recommended), use --no-validate flag.

Suspected Root Cause

The validation regex or parsing logic appears to be checking the wrong part of the requirement. Possible issues:

1. Checking first line of requirement body instead of header: The validator might be looking at the first sentence after the header (e.g., "The system MUST...") instead of the header itself.

2. Regex not matching the full header: The regex might only match ### Requirement: [Text] without the SHALL/MUST suffix, or might be checking a substring that doesn't include the keywords.

3. Case sensitivity: Though unlikely since we're using uppercase SHALL/MUST as recommended.

4. Parsing error: The header parsing might be stopping before the SHALL/MUST keywords.

Suggested Fix

Check the validation code that tests for SHALL/MUST in requirement headers. The regex should match patterns like:

// Correct pattern should match:
/###\s+Requirement:\s+.*?\b(SHALL|MUST)\b/i

Ensure the validation is checking the entire header line (from ### to end of line), not just the first few words.

Documentation Reference

According to openspec/AGENTS.md line 257:

Use SHALL/MUST for normative requirements (avoid should/may unless intentionally non-normative)

This guidance seems to suggest SHALL/MUST should appear in the requirement text (first sentence), not necessarily the header. However, the validation error suggests the header must contain these keywords.

Recommendation: Either:

1. Fix the validation to check requirement text instead of header, OR
2. Update documentation to clarify SHALL/MUST must be in header, OR
3. Make the validation configurable/optional for this rule

Impact

• Severity: High - Blocks legitimate changes from being archived
• Workaround: Manual archive required (breaking automation workflows)
• User Experience: Confusing error message showing the keywords exist but claiming they don't
• Documentation: Creates confusion about where SHALL/MUST should appear

Additional Context

This bug prevented us from archiving a production-ready change with:

• 9 files created (2,328 lines of code)
• 5 files modified
• 68 comprehensive tests
• 5 capability specifications
• Full implementation of critical error handling features

The implementation was complete and tested, but the tooling bug forced us to manually archive.

Proposed Solution

1. Short-term: Fix regex to correctly detect SHALL/MUST in headers
2. Medium-term: Make --no-validate flag actually skip validation as documented
3. Long-term: Improve error messages to show exactly where validation is checking and what it found

Test Case

Create a minimal test case:

# openspec/changes/test-validation/specs/test/spec.md

## ADDED Requirements

### Requirement: Test Feature MUST be implemented
**ID**: REQ-TEST-001
**Priority**: P1

The system MUST implement a test feature.

#### Scenario: Test scenario
**Given** a condition
**When** an action occurs
**Then** a result happens

Run validation:

openspec validate test-validation --strict

Expected: Validation passes
Actual: Validation fails with "must contain SHALL or MUST" despite SHALL/MUST being present

---

Related

• OpenSpec version: 0.9.2
• Project: ideator
• Change ID: prevent-application-crashes (successfully implemented but blocked from archiving)

[TabishB]

@PeakMNA This is super helpful thanks. I'm looking to re-work some of the validation logic and also introducing scaffolding tools to reduce the risk of invalid formats.