fix: resolve Cypress E2E test flakiness (tooltip overlay, seed race, bypass secret)#2032
Conversation
The Chrome renderer process was crashing during agent.cy.ts due to memory pressure on GitHub Actions runners. Split the 9-test spec into 3 smaller files (4+2+3 tests) so each gets a fresh browser context. Also added --js-flags=--max-old-space-size=4096 Chrome flag to increase V8 heap limit for headless CI. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
PR Review Summary
(0) Total Issues | Risk: Low
This PR is a clean CI stability fix that reorganizes Cypress E2E tests to reduce Chrome renderer memory pressure. All reviewers confirmed no issues.
Verification Summary
✅ Test Parity Verified: All 9 original tests preserved across 3 files:
agent.cy.ts(4 tests) — Unsaved changes dialogagent-prompt.cy.ts(2 tests) — Prompt autocomplete and error highlightingagent-tools.cy.ts(3 tests) — Sub-agent ID editing + JSON/JS formatting
✅ Logical Grouping: Tests appropriately organized by functional area
✅ Helper Functions: dragNode() and connectEdge() correctly co-located with agent-tools.cy.ts where they're used
✅ Config Change: The --js-flags=--max-old-space-size=4096 flag is a reasonable V8 heap limit increase for headless CI stability
✅ Existing Patterns: File naming follows conventions in cypress/e2e/ (e.g., projects.cy.ts, skills.cy.ts)
✅ APPROVE
Summary: Clean infrastructure change with no behavioral modifications. The test split strategy is sound — smaller spec files reduce per-spec memory footprint, and the V8 heap flag provides additional headroom. This should address the intermittent CI crashes. Ship it! 🚀
Reviewers (2)
| Reviewer | Returned | Main Findings | Consider | While You're Here | Inline Comments | Pending Recs | Discarded |
|---|---|---|---|---|---|---|---|
pr-review-devops |
0 | 0 | 0 | 0 | 0 | 0 | 0 |
pr-review-tests |
0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Total | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
1. Tooltip overlay blocking drag operations: Add `force: true` to all cy.trigger() calls in dragNode/connectEdge helpers so React Flow handle interactions bypass tooltip overlay elements (~40% of failures) 2. Seed race condition: Add retry logic (3 attempts with backoff) to the "Push Weather Example Project" CI step for when the API server isn't fully warmed up after health check passes (~10-15% of failures) 3. Bypass secret env var mismatch: Align api-config.ts to use INKEEP_AGENTS_MANAGE_API_BYPASS_SECRET (matching what CI sets and what the API server checks) instead of INKEEP_AGENTS_API_BYPASS_SECRET Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
nick-inkeep
changed the title
There was a problem hiding this comment.
PR Review Summary
(0) Total Issues | Risk: Low
This re-review covers the delta since the last automated review (1 commit: a9844f4e — "fix: resolve 3 root causes of Cypress E2E flakiness").
Delta Changes Reviewed
1. Retry Logic for inkeep push (.github/composite-actions/cypress-e2e/action.yml)
- Added 3-attempt retry with exponential backoff (5s, 10s delays)
- Correctly handles directory navigation between retries via
$GITHUB_WORKSPACE - Appropriate pattern for transient CI failures
2. force: true on Drag/Drop Triggers (agents-manage-ui/cypress/e2e/agent-tools.cy.ts)
- Added to
dragstart,dragover,drop,mousedown,mousemove,mouseuptriggers - Validated as appropriate — React Flow canvas elements have complex z-index/transform positioning that can cause false actionability failures in Cypress
- Tests still verify actual behavior (node counts, save confirmation) so this doesn't mask real bugs
3. Env Var Rename (agents-manage-ui/src/lib/api/api-config.ts)
- Renamed from
INKEEP_AGENTS_API_BYPASS_SECRET→INKEEP_AGENTS_MANAGE_API_BYPASS_SECRET - Correctly aligns with domain-specific bypass secret naming convention already used elsewhere in the codebase
✅ APPROVE
Summary: The delta changes are sound CI stability improvements. The retry logic is correctly implemented with proper directory management, the force: true additions are appropriate for React Flow canvas interactions (precedent exists in monaco-editor.cy.ts), and the env var rename maintains consistency with the codebase's domain-specific naming pattern. Ship it! 🚀
Discarded (2)
| Location | Issue | Reason Discarded |
|---|---|---|
action.yml:72 |
GITHUB_WORKSPACE might be unset |
Invalid — GITHUB_WORKSPACE is a built-in GitHub Actions env var guaranteed to be set by the runner in all contexts including composite actions |
action.yml:67-84 |
Could capture error output for better debugging | INFO severity — improvement suggestion but doesn't meet Main threshold; command output already flows to step logs |
Reviewers (2)
| Reviewer | Returned | Main Findings | Consider | While You're Here | Inline Comments | Pending Recs | Discarded |
|---|---|---|---|---|---|---|---|
pr-review-devops |
2 | 0 | 0 | 0 | 0 | 0 | 2 |
pr-review-tests |
0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Total | 2 | 0 | 0 | 0 | 0 | 0 | 2 |
1 participant
Summary
Problem
Cypress E2E tests were failing intermittently in CI with ~13-26% failure rate. Investigation of 500+ CI runs over 14 days identified 3 distinct root causes:
1. Tooltip overlay blocking drag operations (~40% of failures)
cy.trigger() calls in dragNode() and connectEdge() helpers were blocked by a tooltip div[data-state=closed] covering React Flow handle nodes. The tooltip element intercepts Cypress actionability checks even though it is visually invisible.
2. Seed/auth race condition (~10-15% of failures)
The Push Weather Example Project CI step sometimes failed with 401 errors when the API server was not fully warmed up after the health check passed. When this failed, no project data was seeded, causing ALL subsequent tests to fail (0 passing).
3. Bypass secret env var name mismatch
api-config.ts checked process.env.INKEEP_AGENTS_API_BYPASS_SECRET but CI sets INKEEP_AGENTS_MANAGE_API_BYPASS_SECRET. This prevented the Next.js server-side API calls from using the bypass secret as a fallback when cookie forwarding failed.
Changes
Test plan