test: fix hook integration test flakiness on Windows CI#18665
Merged
NTaylorMullen merged 1 commit intomainfrom Feb 15, 2026
Merged
test: fix hook integration test flakiness on Windows CI#18665NTaylorMullen merged 1 commit intomainfrom
NTaylorMullen merged 1 commit intomainfrom
Conversation
Contributor
|
Warning Gemini encountered an error creating the summary. You can try again by commenting |
|
Size Change: -73 B (0%) Total Size: 24.4 MB ℹ️ View Unchanged
|
adamfweidman
approved these changes
Feb 9, 2026
3e6cdea to
33ef0ea
Compare
33ef0ea to
f764fb4
Compare
NTaylorMullen
added a commit
that referenced
this pull request
Feb 10, 2026
- Refactored remaining hook tests in hooks-system.test.ts to use 'rig.createScript' and forward slashes for cross-platform path compatibility. - Replaced 'node -e' usages with script files to avoid brittle quoting and escaping issues on Windows shells. Part of #18665
NTaylorMullen
added a commit
that referenced
this pull request
Feb 10, 2026
- Enforce 'sequential: true' for all hook tests to prevent telemetry leaks and race conditions. - Normalize all path assertions in hooks-system.test.ts using a new 'normalizePath' helper to handle Windows backslashes consistently. - Update 'createScript' in test-rig to return normalized paths. - Ensure 'PATH' is explicitly passed to node-pty spawn options to prevent 'posix_spawnp' errors in some environments. - Clean up manual path replacements in tests in favor of the centralized helper. Part of #18665
7 tasks
NTaylorMullen
added a commit
that referenced
this pull request
Feb 10, 2026
- Ensure 'SystemRoot', 'COMSPEC', 'windir', and 'PATHEXT' are passed to node-pty on Windows to prevent 'posix_spawnp' failures. - Clean up test directories in 'TestRig.setup' to ensure a fresh state for retries and prevent telemetry log accumulation (fixing the 1, 2, 3 failure pattern). - Fix path normalization in 'Hook Disabling' test to ensure disabled hooks are correctly matched on Windows. Part of #18665
NTaylorMullen
added a commit
that referenced
this pull request
Feb 10, 2026
- Refactored remaining hook tests in hooks-system.test.ts to use 'rig.createScript' and forward slashes for cross-platform path compatibility. - Replaced 'node -e' usages with script files to avoid brittle quoting and escaping issues on Windows shells. Part of #18665
NTaylorMullen
added a commit
that referenced
this pull request
Feb 10, 2026
- Enforce 'sequential: true' for all hook tests to prevent telemetry leaks and race conditions. - Normalize all path assertions in hooks-system.test.ts using a new 'normalizePath' helper to handle Windows backslashes consistently. - Update 'createScript' in test-rig to return normalized paths. - Ensure 'PATH' is explicitly passed to node-pty spawn options to prevent 'posix_spawnp' errors in some environments. - Clean up manual path replacements in tests in favor of the centralized helper. Part of #18665
NTaylorMullen
added a commit
that referenced
this pull request
Feb 10, 2026
- Ensure 'SystemRoot', 'COMSPEC', 'windir', and 'PATHEXT' are passed to node-pty on Windows to prevent 'posix_spawnp' failures. - Clean up test directories in 'TestRig.setup' to ensure a fresh state for retries and prevent telemetry log accumulation (fixing the 1, 2, 3 failure pattern). - Fix path normalization in 'Hook Disabling' test to ensure disabled hooks are correctly matched on Windows. Part of #18665
672f57b to
3db7732
Compare
NTaylorMullen
added a commit
that referenced
this pull request
Feb 14, 2026
- Refactored remaining hook tests in hooks-system.test.ts to use 'rig.createScript' and forward slashes for cross-platform path compatibility. - Replaced 'node -e' usages with script files to avoid brittle quoting and escaping issues on Windows shells. Part of #18665
NTaylorMullen
added a commit
that referenced
this pull request
Feb 14, 2026
- Enforce 'sequential: true' for all hook tests to prevent telemetry leaks and race conditions. - Normalize all path assertions in hooks-system.test.ts using a new 'normalizePath' helper to handle Windows backslashes consistently. - Update 'createScript' in test-rig to return normalized paths. - Ensure 'PATH' is explicitly passed to node-pty spawn options to prevent 'posix_spawnp' errors in some environments. - Clean up manual path replacements in tests in favor of the centralized helper. Part of #18665
NTaylorMullen
added a commit
that referenced
this pull request
Feb 14, 2026
- Ensure 'SystemRoot', 'COMSPEC', 'windir', and 'PATHEXT' are passed to node-pty on Windows to prevent 'posix_spawnp' failures. - Clean up test directories in 'TestRig.setup' to ensure a fresh state for retries and prevent telemetry log accumulation (fixing the 1, 2, 3 failure pattern). - Fix path normalization in 'Hook Disabling' test to ensure disabled hooks are correctly matched on Windows. Part of #18665
51a061b to
f28fa76
Compare
f28fa76 to
8a62402
Compare
- Stabilized TestRig cleanup with synchronous exponential backoff using Atomics.wait - Optimized BeforeToolSelection test by truncating mock response loop - Improved hook output parsing to handle both stdout and stderr regardless of exit code - Standardized hook settings structure in integration tests - Enforced child_process PTY on Windows for CI stability
8a62402 to
a780ad2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
test: robust fixes for windows hook flakiness
This PR addresses multiple sources of flakiness and failure in the hook system integration tests on Windows.
Fixes
Telemetry Leakage / Accumulating Failures:
globalSetuptimestamp), causingreadToolLogsto pick up telemetry from previous failed attempts. This led to assertions seeing 1, then 2, then 3 tool calls instead of 0.TestRig.setupto aggressively clean up the test directory before starting a test run.Process Spawning Failures (
posix_spawnp):node-ptyfailed withposix_spawnp failedon Windows. This is often due to the PTY process missing critical environment variables likeSystemRootorCOMSPEC.TestRigto explicitly includeSystemRoot,COMSPEC,windir, andPATHEXTin the PTY environment options on Windows.Disabled Hook Matching:
normalizePathfor the disabled list configuration, ensuring consistent path matching.General Flakiness & Timeouts:
sequential: truefor all hook definitions in tests to prevent race conditions and ensure clean telemetry isolation.normalizePathhelper for all path assertions to avoid fragile manual string replacement.Validation
ntm/repro-hook-flakinesswith a trimmed-down CI workflow to verify these fixes specifically on Windows.