feat: cross-file reference validation for BMB source files and generated output #7
Description
Describe your idea
Add cross-file reference validation to BMad Builder in two layers:
- CI validation of BMB's source files — catch broken references in BMB's 165 source files before they ship to users
- Post-generation validation of BMB output — when BMB creates an agent, workflow, or module, validate that all file references in the generated output resolve to real files
This builds on the cross-file reference validator proposed for BMAD-METHOD (bmad-code-org/BMAD-METHOD#1494), extending it with an --installed mode that validates _bmad/ directories directly.
Why is this needed?
We ran the BMAD-METHOD validator against BMB — here's what it found
| Metric | BMAD-METHOD | BMB |
|---|---|---|
| Source files scanned | 217 | 159 |
| References checked | 483 | 442 |
| Refs per file | 2.2 | 2.8 |
| Broken references found | 5 | 154 |
| Files with issues | 4 | 69 |
The 154 broken references break down as:
| Category | Count | Nature |
|---|---|---|
| Cross-module refs (core/, bmm/, cis/) | 95 | Not bugs — resolve at install time. Validator needs --skip-external. |
| Wrong relative path depth | 20 | Genuine bugs — ../../data/ should be ../data/ in module builder steps. Same class as BMAD-METHOD #1495/#1496. |
| Self-module path mapping | 15 | Validator mapping needs adjustment for BMB's source layout. |
| Template/doc placeholders | 11 | Not bugs — step-[N]-[name].md etc. in templates. |
| Example agent references | 5 | Illustrative, not resolvable. |
| Misnamed step reference | 1 | step-01-validate.md referenced but actual file is step-01-load-target.md. |
| Missing KB CSV files | 3 | Referenced in agent YAML but don't exist in source. |
| Absolute path leaks | 3 | /Users/ paths in documentation examples. |
The 20 wrong-relative-path bugs affect every step file in the module builder's steps-b/, steps-c/, steps-e/, and steps-v/ directories — ../../data/ resolves one level too high; the correct path is ../data/.
In BMAD-METHOD's closed issue history (422 issues, 236 bugs), broken file references and path handling account for 59 bugs (25%) and 129+ comments. The same reference patterns and bug classes appear in BMB.
Generated output is not validated for reference correctness
BMB generates agent YAML, workflow markdown, and module configs containing file references (exec: paths, conversational_knowledge KB paths, nextStepFile chains, cross-module refs). BMB's Zod schema validation checks YAML structure but not whether referenced files exist. Users discover broken references at runtime when an LLM tries to load a missing file.
How should it work?
Roadmap
flowchart LR
subgraph "Step 0 — Prerequisite"
Z["Reopen & merge<br/>BMAD-METHOD #1493 + PR #1494"]
end
subgraph "Step 1 — BMAD-METHOD CI"
A[validate-file-refs.js] -->|source mode| B["src/ files<br/>~480 refs"]
B --> C{"Broken?"}
C -->|Yes| D["⚠️ Warning in build log"]
C -->|No| E["✓ OK"]
end
subgraph "Step 2 — PR Surfacing"
D --> F["::warning:: annotations<br/>on Files Changed tab"]
D --> G["PR review comment<br/>(like CodeRabbit)"]
end
subgraph "Step 3 — BMB CI"
A -->|source mode| H["BMB src/ files<br/>~442 refs"]
H --> I{"Broken?"}
I -->|Yes| F
I -->|Yes| G
end
subgraph "Step 4 — BMB Output"
A -->|installed mode| J["User's _bmad/<br/>generated content"]
J --> K{"Broken?"}
K -->|Yes| L["🔍 Immediate feedback<br/>to user at creation time"]
end
Z --> A
Each step is a separate PR. Steps 2–4 build on Step 1 but don't depend on each other.
Sprint outline
| Step | Work items | Est. |
|---|---|---|
| 0. Prerequisite | Reopen BMAD-METHOD #1493 (closed due to misunderstanding — clarifying comment posted). Merge validator PR #1494 and fix PRs #1497, #1498, #1499. | — |
| 1. BMAD-METHOD CI | Already complete. Validator runs in validate CI job, warning-only (exit 0). |
Done |
| 2. PR surfacing | Add ::warning:: annotations (1 pt), $GITHUB_STEP_SUMMARY (1 pt), PR review comment via workflow_run (3 pts). |
5 pts |
| 3. BMB CI integration | Add --installed mode to validator (1 pt). Add --skip-external for cross-module refs (2 pts). Add validate:refs CI step to BMB (2 pts). Surface findings in BMB PRs (2 pts). |
7 pts |
| 4. BMB output validation | Post-generation validation step in agent builder workflow (2 pts), workflow builder (2 pts), module builder (2 pts). | 6 pts |
| 5. Strict enforcement | Promote to --strict (exit 1) in both repos once existing broken refs are fixed. |
1 pt |
| Total | 19 pts |
Unified tool approach
Both CI and output validation use the same validator with a mode flag:
--source(default): maps{project-root}/_bmad/X→src/Xfor CI validation--installed: checks_bmad/Xdirectly for installed/generated content--skip-external: skip cross-module refs that resolve at install time
The reference extraction patterns (regex for {project-root}/_bmad/, relative paths, exec, nextStepFile, etc.) are shared. Only the path resolution differs.
What this does NOT do
- Does not validate runtime variable interpolation (
{{mustache}},{installed_path}, etc.) - Does not modify any generated files — read-only validation
- Does not change existing CI checks or test behavior
- Does not require new dependencies (uses
node:fs,node:path, andyaml)
PR
We're contributing this work. The BMAD-METHOD validator (PR bmad-code-org/BMAD-METHOD#1494) is ready for review. Once merged, we'll submit PRs for steps 2–4 following the one-fix-per-PR convention.
Additional context
- BMAD-METHOD feature request: feat: cross-file reference validator for BMAD source files BMAD-METHOD#1493 (closed due to misunderstanding — clarifying comment posted, awaiting response)
- BMAD-METHOD validator PR: feat: cross-file reference validator for BMAD source files BMAD-METHOD#1494 (open, passing CI)
- Broken ref fix PRs found by the validator: fix: correct relative path to prd-purpose.md in step-11-polish BMAD-METHOD#1497, #1498, #1499
- BMAD-METHOD historical analysis: 422 closed issues classified, 289 unique broken refs tracked across 26 release tags (alpha.0 → Beta.2)