feat: add retrospective command for spec drift analysis #1533
Conversation
Add /speckit.retrospective command template that performs post-implementation analysis measuring spec drift - the deviation between specification and actual implementation. Features: - Calculates Spec Drift Score (% of requirements implemented as specified) - Classifies deviations: CRITICAL, SIGNIFICANT, MINOR, POSITIVE - Requirement coverage matrix (FR/NFR tracking) - Success criteria assessment against actual values - Architecture drift analysis (data model, tech stack, APIs) - Root cause analysis (when/why deviations occurred) - Structured lessons learned by category - Actionable recommendations for future features - Read-only analysis mode (non-destructive)
There was a problem hiding this comment.
Pull request overview
This pull request adds a new /speckit.retrospective command that performs post-implementation analysis to measure spec drift - the deviation between what was specified and what was actually built in a feature. This creates a feedback loop in the Spec-Driven Development workflow by quantifying implementation deviations and generating actionable lessons learned.
Changes:
- Adds a comprehensive retrospective analysis command with read-only operation
- Implements spec drift scoring based on requirement adherence
- Provides structured root cause analysis for deviations with severity classification (CRITICAL/SIGNIFICANT/MINOR/POSITIVE)
- Generates detailed retrospective reports with traceability and actionable recommendations
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…nts and rename spec drift score to spec adherence score
…n updates and feature creation
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated 11 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| $ARGUMENTS | ||
| ``` | ||
|
|
||
| Consider user input before proceeding (if not empty). |
There was a problem hiding this comment.
The user input consideration statement should use the stronger "You MUST consider" wording to be consistent with all other commands in the codebase. Every other command file (analyze.md, checklist.md, clarify.md, constitution.md, implement.md, plan.md, specify.md, tasks.md, taskstoissues.md) uses "You MUST consider the user input before proceeding (if not empty)." instead of just "Consider user input before proceeding (if not empty)."
| ### 2. Validate Completeness | ||
|
|
||
| ```bash | ||
| total_tasks=$(grep -c '^\- \[[ Xx]\]' "$TASKS") |
There was a problem hiding this comment.
The bash regex pattern '^\- \[[ Xx]\]' will match tasks in any state (unchecked [ ], checked [X], or [x]), which is correct for counting total tasks. However, the pattern should be '^- \[[ Xx]\]' (without backslash before the hyphen) as the hyphen doesn't need to be escaped at the beginning of the pattern. This is more consistent with standard grep patterns and avoids potential confusion.
| total_tasks=$(grep -c '^\- \[[ Xx]\]' "$TASKS") | |
| total_tasks=$(grep -c '^- \[[ Xx]\]' "$TASKS") |
| total_tasks=$(grep -c '^\- \[[ Xx]\]' "$TASKS") | ||
| completed_tasks=$(grep -c '^\- \[[Xx]\]' "$TASKS") |
There was a problem hiding this comment.
Similarly, the pattern '^\- \[[Xx]\]' for completed tasks should be '^- \[[Xx]\]' without escaping the hyphen.
| total_tasks=$(grep -c '^\- \[[ Xx]\]' "$TASKS") | |
| completed_tasks=$(grep -c '^\- \[[Xx]\]' "$TASKS") | |
| total_tasks=$(grep -c '^- \[[ Xx]\]' "$TASKS") | |
| completed_tasks=$(grep -c '^- \[[Xx]\]' "$TASKS") |
| ```bash | ||
| total_tasks=$(grep -c '^\- \[[ Xx]\]' "$TASKS") | ||
| completed_tasks=$(grep -c '^\- \[[Xx]\]' "$TASKS") | ||
| completion_rate=$((completed_tasks * 100 / total_tasks)) |
There was a problem hiding this comment.
The division operation $((completed_tasks * 100 / total_tasks)) will fail with a "division by zero" error if there are no tasks in the file. While this is unlikely in normal usage (since tasks.md should have tasks), it's a potential edge case that should be handled. Consider adding a check or using a safer calculation approach.
| completion_rate=$((completed_tasks * 100 / total_tasks)) | |
| if [ "$total_tasks" -eq 0 ]; then | |
| completion_rate=0 | |
| else | |
| completion_rate=$((completed_tasks * 100 / total_tasks)) | |
| fi |
| ### 13. Save Report | ||
|
|
||
| 1. Write to `FEATURE_DIR/retrospective.md` with YAML frontmatter (feature, branch, date, rates, counts) | ||
| 2. Commit: `Add retrospective analysis - Spec adherence: X% | Completion: X%` |
There was a problem hiding this comment.
The commit message format uses a dash separator "Spec adherence: X% | Completion: X%" but most of the codebase and conventional commit messages would use a colon after "Add retrospective analysis". Consider using "Add retrospective analysis: Spec adherence X%, Completion X%" or following conventional commit format like "feat(retrospective): add analysis report (adherence X%, completion X%)" for better git log readability and consistency.
| 2. Commit: `Add retrospective analysis - Spec adherence: X% | Completion: X%` | |
| 2. Commit: `Add retrospective analysis: Spec adherence X%, Completion X%` |
| @@ -0,0 +1,220 @@ | |||
| --- | |||
| description: Perform a post-implementation retrospective analysis measuring spec drift, implementation deviations, and generating lessons learned for future features. | |||
There was a problem hiding this comment.
The CHANGELOG.md should be updated to document the addition of the new /speckit.retrospective command. According to the AGENTS.md guidelines (line 15): "Any changes to init.py for the Specify CLI require a version rev in pyproject.toml and addition of entries to CHANGELOG.md." While this change doesn't modify init.py, it does add a significant new command to the toolkit that should be documented for users. The changelog should include a description of the new retrospective command and its purpose (spec drift analysis, lessons learned, etc.).
| | Metric | Value | | ||
| |--------|-------| | ||
| | Total/Implemented/Modified/Not Implemented/Unspecified | X | |
There was a problem hiding this comment.
The table format appears incorrect. Line 127 shows "Total/Implemented/Modified/Not Implemented/Unspecified | X" which suggests multiple metrics crammed into a single row. This should likely be a multi-row table where each metric (Total, Implemented, Modified, Not Implemented, Unspecified) has its own row with a corresponding value, or it should be a multi-column table with separate columns for each metric type. The current format is unclear and won't render properly as a meaningful table.
| | Metric | Value | | |
| |--------|-------| | |
| | Total/Implemented/Modified/Not Implemented/Unspecified | X | | |
| | Metric | Total | Implemented | Modified | Not Implemented | Unspecified | | |
| |--------|-------|-------------|----------|------------------|-------------| | |
| | Count | X | X | X | X | X | |
| - **Post-Implementation**: Run after implementation complete; warn if <80% tasks done, abort if <50% | ||
|
|
||
| ## Execution Steps | ||
|
|
||
| ### 1. Initialize Context | ||
|
|
||
| Run `{SCRIPT}` from repo root. Parse JSON for FEATURE_DIR. Derive paths: SPEC, PLAN, TASKS = FEATURE_DIR/{spec,plan,tasks}.md. Abort if missing. | ||
|
|
||
| For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot"). | ||
|
|
||
| ### 2. Validate Completeness | ||
|
|
||
| ```bash | ||
| total_tasks=$(grep -c '^\- \[[ Xx]\]' "$TASKS") | ||
| completed_tasks=$(grep -c '^\- \[[Xx]\]' "$TASKS") | ||
| completion_rate=$((completed_tasks * 100 / total_tasks)) | ||
| ``` | ||
|
|
||
| - **≥80%**: Full retrospective | **50-79%**: Warn, partial analysis | **<50%**: Confirm before proceeding |
There was a problem hiding this comment.
The command states "abort if <50%" in line 32, but the detailed instruction in line 50 says "Confirm before proceeding" for <50% completion. These instructions are inconsistent. Based on the context and safer design (allowing users to proceed if they explicitly confirm), the "Confirm before proceeding" approach is more appropriate than an absolute abort, but the instructions should be consistent.
|
|
||
| ### 5. Discover Implementation | ||
|
|
||
| - Extract file paths from completed tasks + `git log --name-only HEAD~50..HEAD` |
There was a problem hiding this comment.
The git log command git log --name-only HEAD~50..HEAD assumes there are at least 50 commits in the repository history. In a new or small repository, this will fail with an error like "fatal: ambiguous argument 'HEAD~50': unknown revision or path not in the working tree." Consider using a safer approach like git log --name-only --since="1 month ago" or wrapping the command with error handling.
| - Extract file paths from completed tasks + `git log --name-only HEAD~50..HEAD` | |
| - Extract file paths from completed tasks + `git log --name-only --since="1 month ago"` (or another safe range that works even in small repositories) |
| | Req ID | Description | Status | Notes | | ||
|
|
||
| ### Success Criteria Assessment | ||
| | Criterion | Target | Actual | Met? | | ||
|
|
||
| ### Architecture Drift | ||
| | Entity/Component | Specified | Actual | Change Type/Reason | | ||
|
|
||
| ### Significant Deviations | ||
| For each: Severity, Requirements Affected, Specified vs Implemented, Discovery, Root Cause, Impact, Lesson | ||
|
|
||
| **Example:** | ||
| > #### Auth Method Changed | ||
| > - **Severity**: SIGNIFICANT | **Affected**: FR-003, SC-002 | ||
| > - **Specified**: Email/password with bcrypt | **Implemented**: OAuth2 SSO | ||
| > - **Discovery**: Phase 1 | **Cause**: Tech Constraint (email service unavailable) | ||
| > - **Impact**: Better UX, added third-party dependency | ||
| > - **Lesson**: Specify auth dependencies in planning phase | ||
|
|
||
| ### Innovations & Best Practices | ||
| | Innovation | Description | Benefit | Reusability | Constitution Candidate? | | ||
|
|
||
| **Example:** | ||
| | Response caching | Redis cache for frequent queries | 85% less DB load, 60% faster | All read-heavy endpoints | Yes - "Cache-First" principle | | ||
|
|
||
| ### Constitution Compliance | ||
| | Article | Title | Status | Notes | | ||
| **Violations**: [None / List with justifications] | ||
|
|
||
| ### Analysis Effectiveness (if applicable) | ||
| | Category | Predicted & Found | Predicted & Not Found | Found & Not Predicted | | ||
|
|
||
| ### Unspecified Implementations | ||
| | Addition | Description | Justification | Should Have Been Specified? | | ||
|
|
||
| ### Task Execution Analysis | ||
| | Phase | Planned | Completed | Added | Dropped | | ||
|
|
||
| ### Lessons Learned | ||
| - Specification/Planning/Process/Technical improvements | ||
|
|
||
| ### Recommendations | ||
| - For this feature / For future features / Constitution updates | ||
|
|
||
| ### Appendix: File Traceability | ||
| | File | Created/Modified | Requirements | |
There was a problem hiding this comment.
Multiple tables in the report template are missing the required separator row between the header and content rows. According to Markdown table syntax, each table must have a separator row with dashes. The following tables are incomplete:
- Line 130: "Requirement Coverage Matrix" (missing separator like
|--------|-------------|--------|-------|) - Line 133: "Success Criteria Assessment" (missing separator)
- Line 136: "Architecture Drift" (missing separator)
- Line 150: "Innovations & Best Practices" (missing separator)
- Line 156: "Constitution Compliance" (missing separator)
- Line 160: "Analysis Effectiveness" (missing separator)
- Line 163: "Unspecified Implementations" (missing separator)
- Line 166: "Task Execution Analysis" (missing separator)
- Line 175: "Appendix: File Traceability" (missing separator)
Without these separator rows, the tables won't render correctly in most Markdown parsers.
1 participant
Summary
Adds a new
/speckit.retrospectivecommand for post-implementation analysis that measures spec drift - the deviation between what was specified and what was actually built.Motivation
The Spec-Driven Development workflow currently covers the full lifecycle from specification through implementation, but lacks a critical feedback loop: learning from completed features. Without retrospective analysis, teams lose valuable insights that could improve future specifications and reduce rework.
The Problem
The Solution
The
/speckit.retrospectivecommand provides:Expected Impact
Design Decisions
check-prerequisites.shfor consistency with other commandsUsage
Run after implementation is complete (or at a significant milestone):