Better reactive test cancellation

[thomhurst]

No description provided.

[claude]

CRITICAL ISSUE - Resource Leak in TimeoutHelper.cs:60-78

When a timeout occurs, the code awaits executionTask without any timeout or cancellation. If the task does not respond to cancellation and hangs indefinitely, this will wait forever, defeating the purpose of the timeout.

RECOMMENDATION: Add a grace period (e.g., 1 second) using Task.WaitAsync() before throwing TimeoutException.

PERFORMANCE ISSUE - Unnecessary allocations in hot path (TimeoutHelper.cs:46-49)

When timeout is null, we still create Task.Delay(Timeout.Infinite), a CTS, and use Task.WhenAny with 3 tasks. This adds overhead for every test without a timeout.

RECOMMENDATION: Add early return when timeout is null to minimize allocations in the hot path.

Per CLAUDE.md Rule 4, TUnit processes millions of tests daily and performance is critical in test execution paths.

POSITIVE NOTES:

• Simplified logic is much cleaner
• Better reactive cancellation with TaskCompletionSource
• Good test coverage in both Engine.Tests and TestProject
• Improved token chaining in TestSessionCoordinator
• Clean RunOptions fluent API

ASSESSMENT: Good refactoring but needs fixes before merge. The resource leak is critical and the performance optimization is highly recommended for a test framework.

[Copilot]

Pull request overview

This pull request enhances TUnit's test cancellation mechanism to be more reactive by wiring up the engine's cancellation token throughout the execution pipeline and refactoring timeout handling. The main improvement allows graceful cancellation requests (e.g., Ctrl+C) to immediately propagate to running tests rather than waiting for them to complete.

Key Changes

• Added _serviceProvider.CancellationToken.Token to the linked cancellation token source in TestSessionCoordinator, enabling engine cancellation to flow through test execution
• Refactored TimeoutHelper to monitor cancellation token alongside timeout, allowing immediate response to cancellation requests
• Added test infrastructure and test cases to verify graceful cancellation works correctly

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
TUnit.Engine/TestSessionCoordinator.cs	Links the engine's cancellation token to test execution, enabling reactive cancellation
TUnit.Engine/Helpers/TimeoutHelper.cs	Refactored to use TaskCompletionSource pattern for monitoring cancellation alongside timeout
TUnit.TestProject/CanCancelTests.cs	Added integration test with long-running task to verify cancellation behavior
TUnit.Engine.Tests/CanCancelTests.cs	Added test that verifies graceful cancellation terminates tests within expected time
TUnit.Engine.Tests/InvokableTestBase.cs	Enhanced test infrastructure to support graceful/forceful cancellation tokens and execution callbacks

[Copilot]

The TaskCompletionSource and its cancellation token registration could cause a memory leak if the executionTask completes before cancellation. The registration is disposed when the using statement exits, but if executionTask completes first, the registration remains active until the method returns. Consider disposing the registration as soon as the executionTask completes to free resources immediately, or use a try-finally pattern to ensure proper cleanup.

[Copilot]

This is a breaking behavioral change from the previous implementation. The old code passed a linked cancellation token (timeoutCts.Token) to the taskFactory, which would be automatically cancelled when the timeout elapsed. The new code passes the original cancellationToken, which means the task won't be automatically cancelled on timeout. This could cause tasks to continue running in the background after timeout, leading to resource leaks and unexpected behavior. Restore the previous behavior by creating a linked CancellationTokenSource and passing its token to taskFactory.

[Copilot]

Important XML documentation for parameters and return values has been removed. According to TUnit development guidelines, code documentation should be maintained. The removed documentation described the parameters (taskFactory, timeout, cancellationToken, timeoutMessage) and return value, as well as the TimeoutException that can be thrown. Restore this documentation.

[Copilot]

Important XML documentation for the generic type parameter and method parameters has been removed. According to TUnit development guidelines, code documentation should be maintained. The removed documentation described the type parameter T and the parameters (taskFactory, timeout, cancellationToken, timeoutMessage), return value, and TimeoutException. Restore this documentation.

[Copilot]

The test method name "Test" is ambiguous and doesn't describe what it's testing. According to TUnit development guidelines and best practices, test methods should have descriptive names that explain the scenario and expected outcome. Consider renaming to something like "CanCancelTests_WhenGracefullyCancelled_FailsWithinTimeout" or "GracefulCancellation_ShouldTerminateTestBeforeTimeout".

Suggested change

	public async Task Test(CancellationToken ct)
	public async Task CanCancelTests_WhenGracefullyCancelled_FailsWithinTimeout(CancellationToken ct)

[Copilot]

The CancellationToken parameter 'ct' is declared but never used in the test method. Either remove the unused parameter or use it if cancellation support was intended (e.g., passing it to RunTestsWithFilter if that method supports it).

Suggested change

	using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(5));
	using var cts = CancellationTokenSource.CreateLinkedTokenSource(ct);
	cts.CancelAfter(TimeSpan.FromSeconds(5));

[claude]

Code Review: Better Reactive Test Cancellation

I've reviewed PR #4158 and have the following feedback:

✅ Strengths

1. Improved Cancellation Responsiveness: The changes make cancellation more reactive by using TaskCompletionSource to detect when cancellationToken is triggered. This is a solid improvement for user experience.

2. Good Test Coverage: Adding dedicated CanCancelTests in both TUnit.Engine.Tests and TUnit.TestProject demonstrates proper dual-mode testing.

3. Resource Cleanup: The code properly cancels the timeout task when no longer needed.

4. Grace Period Handling: The 1-second grace period for cancelled tasks is thoughtful.

🐛 Critical Issues

1. Allocation Concern in Hot Path (TimeoutHelper.cs:54-71)

The fast path now allocates a TaskCompletionSource and registers a cancellation callback on EVERY call, even when cancellation is unlikely. This is a performance regression.

According to CLAUDE.md Rule 4, TUnit processes millions of tests daily. Test execution is a hot path where allocations must be minimized.

Suggested Fix: Add a check for !cancellationToken.CanBeCanceled first to avoid allocations when the token cannot be cancelled.

2. Unused Variable Name (TimeoutHelper.cs:87)

winningTask2 suggests leftover refactoring. Should be completedTask or similar.

3. Resource Leak on Early Return (TimeoutHelper.cs:90-93)

When cancellation wins, executionTask is left running without waiting. The task could hold resources (connections, files).

Suggested Fix: Add the same grace period handling as the timeout path before throwing OperationCanceledException.

⚠️ Medium Priority

4. Missing timeoutCts.Cancel() call when external cancellation occurs.

5. Test timeout (30s) matches MaxExpectedDuration - consider using 60s to distinguish assertion failure from infrastructure timeout.

🧪 Testing Concerns

6. Windows Exclusion - ExcludeOn(OS.Windows) means cancellation is untested on Windows. Consider adding unit tests that don't rely on subprocess control.

📋 Checklist

• ⚠️ Performance-critical code changed - should profile before/after per Rule 4
• ⚠️ Did you run snapshot tests (TUnit.Core.SourceGenerator.Tests and TUnit.PublicAPI)?

🎯 Summary

Recommendation: Address the critical performance issue (#1) and resource leak (#3) before merging.

Great work on improving cancellation reactivity! The core idea is sound, just needs polish on hot path performance. 🚀

[claude]

PR Review: Better Reactive Test Cancellation

This PR significantly improves test cancellation handling in TUnit with better responsiveness and resource management. Overall, the implementation is solid and follows TUnit's development guidelines. Below are my findings:

✅ Strengths

1. Performance Optimizations: The fast path optimizations in TimeoutHelper.cs are excellent:

   • Avoids allocations when !timeout.HasValue
   • Skips unnecessary work when !cancellationToken.CanBeCanceled
   • These align perfectly with TUnit's performance-first principle

2. Reactive Cancellation: The use of TaskCompletionSource for immediate cancellation detection (lines 66-74, 86-99) is the right approach for responsive cancellation

3. Resource Cleanup: Properly cancels timeout tasks to prevent resource leaks (line 98, 133)

4. Grace Period: The 1-second grace period (lines 105-126) gives tests a chance to clean up gracefully before termination

5. Multi-Framework Support: Conditional compilation for NET8+ vs older frameworks shows good compatibility considerations

6. Test Coverage: Added comprehensive test in CanCancelTests.cs that verifies the feature works end-to-end

7. Code Style: Follows modern C# patterns - collection expressions, file-scoped namespaces, proper async/await

⚠️ Issues Found

1. Critical: Missing Cancellation Signal to Execution Task

Location: TUnit.Engine/Helpers/TimeoutHelper.cs:103-129

When timeout occurs, the code never calls timeoutCts.Cancel() to signal the execution task that it should stop. This means:

• The task continues running in the background without knowing it timed out
• Resources may not be released properly
• The grace period waits for a task that doesn't know it should cancel

Fix needed:

// Timeout occurred
if (winner == timeoutTask)
{
    // Signal the execution task to cancel
    timeoutCts.Cancel();  // <-- ADD THIS LINE
    
    // Give the execution task a brief grace period to handle cancellation
    try
    {
        // ... rest of code
    }
}

2. Potential Issue: Duplicate Token Registration

Location: TUnit.Engine/Helpers/TimeoutHelper.cs:80-87

You're creating both:

• A linked token source (timeoutCts) that includes the original cancellationToken
• A separate TaskCompletionSource that also observes cancellationToken

This creates two parallel mechanisms observing the same cancellation token. Consider:

• If you only need reactive behavior, you might not need the linked token source
• If the linked token is needed for the task to observe cancellation, the separate TCS might be redundant

Recommendation: Clarify the intent with a comment explaining why both are needed, or simplify if one mechanism suffices.

3. Performance: Allocation in No-Timeout Path

Location: TUnit.Engine/Helpers/TimeoutHelper.cs:57-76

Even in the fast path (no timeout), when cancellationToken.CanBeCanceled is true, you allocate a TaskCompletionSource and registration. For a hot path (which this likely is), consider:

• Is this allocation justified?
• Could you just await the task and catch OperationCanceledException?

The current implementation races to throw immediately on cancellation, but at the cost of allocations. Profile whether this trade-off is worth it for TUnit's use case.

4. Test Clarity: Expected Outcome

Location: TUnit.Engine.Tests/CanCancelTests.cs:26

The test expects Outcome.ShouldBe("Failed") - is this the correct expected outcome for a cancelled test? Typically:

• Cancelled or Skipped might be more appropriate than Failed
• This could be confusing for users - a cancelled test isn't really a "failed" test

Verify: Is this the intended behavior? If so, add a comment explaining why cancelled tests are marked as "Failed".

5. Minor: Magic Number

Location: TUnit.Engine/Helpers/TimeoutHelper.cs:109, 111

The 1-second grace period is a magic number. Consider extracting to a named constant:

private const int GracePeriodSeconds = 1;

// Then use:
await executionTask.WaitAsync(TimeSpan.FromSeconds(GracePeriodSeconds), ...);

This improves readability and makes it easier to tune if needed.

📋 Additional Considerations

1. Dual-Mode Implementation: ✅ This appears to be engine-level infrastructure that works for both source-generated and reflection modes

2. Snapshot Tests: No source generator or public API changes detected, so no snapshot updates needed ✅

3. Breaking Changes: No public API changes detected ✅

4. AOT Compatibility: No reflection usage in changed code ✅

🎯 Recommendations

Priority Order:

1. HIGH: Add timeoutCts.Cancel() before the grace period in the timeout path
2. MEDIUM: Clarify or simplify the dual cancellation observation mechanism
3. MEDIUM: Verify the "Failed" outcome for cancelled tests is intentional
4. LOW: Consider profiling the no-timeout fast path allocations
5. LOW: Extract grace period to named constant

📝 Summary

This is a well-implemented improvement to TUnit's cancellation handling. The main issue is the missing timeoutCts.Cancel() call, which should be added before merging. The performance optimizations and reactive cancellation design are excellent and align with TUnit's principles.

Recommendation: Request changes for the critical issue, then approve after fix.

[claude]

Pull Request Review: Better Reactive Test Cancellation

This PR improves test cancellation handling by making cancellation more reactive and adding proper grace period handling. Overall well-implemented with good test coverage.

Strengths

1. Excellent test coverage with new CanCancelTests.cs
2. Performance-conscious optimizations in fast path (lines 60-76)
3. Graceful degradation with 1-second grace period
4. Platform-aware testing with ExcludeOn(OS.Windows)
5. Good documentation of behavior

Critical Issue: Resource Leak (TimeoutHelper.cs:71-75)

The fast path has a timing issue where registration.Dispose() happens too late:

Current code awaits the winner task while still holding the registration, which delays cleanup. Fix by disposing immediately after WhenAny determines the winner.

Medium Priority Issues

1. Duplicate Cancellation Registration (Lines 84-92): Both timeoutCts (line 79) and cancellationTcs (line 86) register the same cancellationToken. This doubles allocation cost. Remove cancellationTcs - the linked CTS already handles external cancellation.

2. Unobserved Exception (Lines 103-114): The .NET < 8 grace period branch doesn't await executionTask if it completes within the grace period, potentially leaving exceptions unobserved.

3. Missing Cleanup: Line 95 should also cancel timeoutCts when execution completes normally for faster resource release.

Minor Issues

• Test documentation could explain why we test this (not just what we expect)
• Magic numbers could use clarifying comments
• Unused return value in InvokableTestBase.cs:195

TUnit Compliance

• Performance-first principle mostly followed, but resource leak contradicts Rule 4
• Modern C# syntax used correctly
• Good test coverage
• Since this changes hot-path code, please profile before/after per CLAUDE.md

Overall: 7.5/10

Excellent architectural approach with strong testing. Fix the resource leak and duplicate registration before merge. With these changes, this will be a solid improvement.

Great work on the reactive cancellation improvement!