feat(pr-review): Add learnings from PR #1699 - custom libraries and framework features

[claude]

Summary

Improvements to PR review agents based on human reviewer insights from PR #1699 that automated bots completely missed.

Changes

1. Custom Implementations vs Standard Libraries (pr-review-standards)

What humans caught:

• Custom jsonSchemaToZod function was missing critical functionality (handling required field)
• Sarah recognized validation should distinguish required vs optional fields
• Dima identified the root cause: custom implementation violated JSON Schema spec

Bot behavior:

• ✅ No syntax errors detected
• ✅ No type errors detected
• ❌ Missed semantic spec violation

Improvement added:

• Detection heuristics for custom implementations of standard functionality
• Review questions to evaluate if libraries would be better
• Example finding template for suggesting well-maintained alternatives
• Reduces maintenance burden and catches incomplete implementations

Applies to:

• Schema conversion (JSON Schema, OpenAPI, GraphQL)
• Standard algorithms (JWT parsing, date formatting, URL parsing)
• Protocol implementations (OAuth, SAML, OpenID Connect)

2. Framework Compiler Feature Validation (pr-review-frontend)

What humans caught:

• Bot incorrectly flagged `'use memo'` as invalid directive
• Dima provided React Compiler documentation showing it's valid with `babel-plugin-react-compiler`

Bot behavior:

• ❌ Flagged valid React Compiler directive as invalid
• Didn't check for compiler plugins in package.json
• Training data didn't include recent framework features

Improvement added:

• Validation process for unusual directives before flagging as invalid
• Checks package.json for compiler plugins
• Framework feature database (React, Next.js, Vue)
• Reduces false positives on emerging framework features

Applies to:

• React Compiler directives (`'use memo'`)
• Next.js directives (`'use cache'` in Next.js 15+)
• Vue macros (`defineModel`, `defineProps`)
• Any framework-specific pragma or annotation

Supporting Documentation

Created comprehensive learning document:

• `.claude/agents/pr-review-learnings/pr-1699-custom-libraries-and-framework-features.md`
• Documents what humans caught, why bots missed it, and generalizability analysis
• Includes patterns evaluated but not implemented (too complex for automation)
• Provides success metrics and future directions

Key Insight

Bots excel at syntactic correctness but miss semantic correctness and implementation choices.

Human reviewers reason about:

• ✅ Specification compliance (does this follow JSON Schema spec?)
• ✅ Library alternatives (should we use a standard library?)
• ✅ Emerging framework features (is this a valid compiler directive?)

Bots focus on:

• ✅ Syntax errors, type errors, undefined variables
• ✅ Common patterns (null checks, error handling)
• ❌ Domain specification semantics
• ❌ Implementation necessity vs library alternatives
• ❌ Recent framework features not in training data

Testing Recommendations

1. Verify reduced false positives:

   • Test on PRs with React Compiler directives
   • Test on PRs with Next.js 15+ features
   • Ensure valid framework features aren't flagged

2. Verify increased valuable findings:

   • Test on PRs with custom schema conversion
   • Test on PRs with custom JWT/date/URL parsing
   • Ensure suggestions match available libraries

3. Cross-validation:

   • Review next 5-10 merged PRs to see if patterns would have caught real issues
   • Compare bot findings to human reviewer insights

Source

PR #1699: #1699

• Title: "Notify users to enter custom headers if required in chat"
• Merged: 2026-02-05
• Human reviewers: Sarah, Miles, Dima

Analysis: Applied 4-criteria generalizability test. Patterns P3 (custom vs library) and P4 (framework features) passed all criteria. Patterns P1 (schema semantics) and P2 (behavioral correctness) were too complex or context-dependent for automation.

Related

• Follows guidance from `.claude/agents/closed-pr-review-auto-improver.md`
• Implements feedback loop from human reviewers to automated reviews
• Part of continuous improvement for PR review quality

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
agents-api	Ready	Preview, Comment	Feb 6, 2026 6:32am
agents-docs	Ready	Preview, Comment	Feb 6, 2026 6:32am
agents-manage-ui	Ready	Preview, Comment	Feb 6, 2026 6:32am

[changeset-bot]

⚠️ No Changeset found

Latest commit: 273e941

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[nick-inkeep]

Closing - output was too verbose. Re-running with updated agent that enforces compact format.