Skip to content

Comments

feat(pr-review): Add missing documentation and changeset checks#1766

Closed
claude[bot] wants to merge 1 commit intomainfrom
closed-pr-review-auto-improver/pr-1699-learnings
Closed

feat(pr-review): Add missing documentation and changeset checks#1766
claude[bot] wants to merge 1 commit intomainfrom
closed-pr-review-auto-improver/pr-1699-learnings

Conversation

@claude
Copy link
Contributor

@claude claude bot commented Feb 6, 2026

Summary

This PR improves the automated PR review system based on patterns identified from human reviewer feedback in PR #1699.

Changes

1. Documentation Check (HIGH priority - CRITICAL severity)

File: .claude/agents/pr-review-docs.md

What it does:

  • Automatically checks if PR modifies user-facing code without documentation updates
  • Triggers on specific file patterns:
    • New UI components: *.tsx in agents-manage-ui/src/components/
    • New API routes: new files in agents-api/src/domains/*/routes/
    • Schema changes: *-schema.ts or packages/agents-core/src/validation/
    • New package exports: package.json exports field changes
    • New SDK patterns: agents-sdk/src/ changes
  • Creates CRITICAL finding when documentation is missing

Why this matters:

Example finding:

CRITICAL: PR modifies UI components but lacks documentation updates.
Per AGENTS.md: 'ALL new work MUST include... Documentation - Create or update
documentation in /agents-docs/content/docs/'. Users will not know how to use
this new feature/change.

2. Changeset Check (HIGH priority - CRITICAL severity)

File: .claude/agents/pr-review-standards.md

What it does:

  • Automatically checks if PR modifies published packages without changeset files
  • Triggers when PR modifies: agents-core, agents-api, agents-sdk, agents-manage-ui, agents-cli, create-agents, ai-sdk-provider
  • Excludes internal-only changes: test files, docs, comment-only changes
  • Creates CRITICAL finding when changeset is missing

Why this matters:

Example finding:

CRITICAL: PR modifies published packages [agents-core, agents-manage-ui] but
lacks a changeset. Per AGENTS.md: 'Create a changeset for any user-facing
change to a published package.' Version bumps won't be tracked. Package
changes won't appear in changelog.

Fix: Run: pnpm bump <patch|minor|major> --pkg <package-name> "<description>"

3. React Compiler Annotation Fix (MEDIUM priority - Bug fix)

File: .claude/agents/pr-review-frontend.md

What it does:

Why this matters:


Methodology

These improvements were identified through systematic analysis of PR #1699:

  1. Compared human reviewer comments vs automated bot comments
  2. Identified patterns humans caught that bots missed
  3. Applied 4 generalizability criteria (all must pass):
    • Actionable: Can be turned into a specific check/rule
    • Frequent: Likely to recur in future PRs
    • Objectively verifiable: Can be checked programmatically
    • Not already covered: Not duplicating existing bot checks

Result: 2 HIGH-generalizability patterns + 1 bug fix identified


Impact

Before this PR:

  • Human reviewers had to manually remind about documentation
  • Human reviewers had to manually request changesets
  • Bot created false positives for React Compiler annotations

After this PR:

  • Bot automatically checks for missing documentation (CRITICAL finding)
  • Bot automatically checks for missing changesets (CRITICAL finding)
  • Bot correctly recognizes React Compiler annotations (no false positives)

Testing Recommendation

To validate these changes work correctly:

  1. Create a test PR that modifies agents-manage-ui/src/components/ without docs → should trigger CRITICAL finding
  2. Create a test PR that modifies packages/agents-core/ without changeset → should trigger CRITICAL finding
  3. Create a test PR with 'use memo' directive → should NOT flag as error

Related

Learned from PR #1699 human reviewer feedback:

1. **Documentation Check (HIGH priority)**
   - pr-review-docs now checks if PR modifies user-facing code without docs
   - Triggers on: new UI components, API routes, schemas, package exports, SDK changes
   - Creates CRITICAL finding per AGENTS.md requirement
   - Human reviewer had to manually remind: "can you check for docs changes"

2. **Changeset Check (HIGH priority)**
   - pr-review-standards now checks if PR modifies published packages without changeset
   - Triggers on: agents-core, agents-api, agents-sdk, agents-manage-ui, agents-cli, etc.
   - Creates CRITICAL finding per AGENTS.md requirement
   - Human reviewer had to manually request: "can you write a changeset"

3. **React Compiler Annotation Fix (MEDIUM priority)**
   - pr-review-frontend now recognizes 'use memo' as valid React Compiler annotation
   - Bot incorrectly flagged this as invalid directive in PR #1699
   - Human provided documentation link showing it's valid

These patterns were identified through systematic analysis of human vs bot
comments, meeting all 4 generalizability criteria: actionable, frequent,
objectively verifiable, and not already covered.
@vercel
Copy link

vercel bot commented Feb 6, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
agents-api Ready Ready Preview, Comment Feb 6, 2026 7:00am
agents-docs Ready Ready Preview, Comment Feb 6, 2026 7:00am
agents-manage-ui Ready Ready Preview, Comment Feb 6, 2026 7:00am

Request Review

@changeset-bot
Copy link

changeset-bot bot commented Feb 6, 2026

⚠️ No Changeset found

Latest commit: 7d9621e

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@nick-inkeep
Copy link
Collaborator

Closing - PR body still too verbose (had extra sections). Updated agent with stricter template.

@nick-inkeep nick-inkeep closed this Feb 6, 2026
@nick-inkeep nick-inkeep deleted the closed-pr-review-auto-improver/pr-1699-learnings branch February 6, 2026 07:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant