Add File.OpenNullHandle() for efficient process I/O redirection

[Copilot]

Description

Adds File.OpenNullHandle() API to obtain a handle to the system's null device (NUL on Windows, /dev/null on Unix). This enables efficient discarding of process output/error streams or providing empty input without the overhead of reading and ignoring data.

Current inefficient pattern:

process.StartInfo.RedirectStandardOutput = true;
process.OutputDataReceived += (sender, e) => { }; // Wasteful
process.BeginOutputReadLine();

New API:

using SafeFileHandle nullHandle = File.OpenNullHandle();
// Pass to process creation APIs to discard output or provide empty input

Changes

• API Surface: Added File.OpenNullHandle() to System.IO.File
• Platform Implementation:
  • Introduced platform-specific NullDevicePath constants in File.Windows.cs ("NUL") and File.Unix.cs ("/dev/null")
  • Main implementation in File.cs calls SafeFileHandle.Open() with the platform-specific path constant
  • Opens with FileOptions.None (enforces O_CLOEXEC on Unix for non-inheritability)
• Handle Capabilities:
  • ReadWrite access (reads return EOF, writes succeed)
  • Works with both sync and async I/O operations
  • Non-inheritable by child processes on Windows (default behavior)
  • Non-inheritable by child processes on Unix (via O_CLOEXEC)
• Tests: 13 test cases covering sync/async read/write operations with FileStream, including:
  • RemoteExecutor test to verify concurrent access doesn't lock the null device
  • Tests with simplified FileStream(handle, FileAccess.*) constructors
  • Note: Theory test with explicit isAsync parameter was attempted but reverted due to FileStream validation preventing use of sync handles with isAsync: true (tracked separately in issue Remove isAsync flag validation from FileStream ctors #123760)

Testing

All tests passing. Verified:

• Sync and async I/O operations work correctly with simplified FileStream(handle, FileAccess.*) constructors
• Null device is not locked - concurrent access from parent and child processes works correctly
• Reads always return 0 (EOF)
• Writes always succeed
• Code refactored to use platform-specific constants for better maintainability

Original prompt

---

This section details on the original issue you should resolve

<issue_title>File: low-level APIs for opening null device</issue_title>
<issue_description>### Background and motivation

As of today, BCL does not provie any API for discarding standard input, output, or error of the started process in optimal way. Because of that, users often read the STD OUT/ERR just to ignore it, which is very inefficient. Sample from our code base:

process.StartInfo.RedirectStandardOutput = true;
process.StartInfo.RedirectStandardError = true;

// Send the output and error streams to empty handlers because the text is also written to the log files
process.OutputDataReceived += (sender, e) => { };
process.ErrorDataReceived += (sender, e) => { };

process.Start();

process.BeginOutputReadLine();
process.BeginErrorReadLine();

This could be avoided if we give the users the ability to open handles to the null device (NUL on Windows, /dev/null on Unix). Those handles can then be passed to the process creation APIs to discard unwanted output or provide empty input.

API Proposal

namespace System.IO;

public static class File
{
+    public static SafeFileHandle OpenNullFileHandle();
     public static SafeFileHandle OpenHandle(string path, FileMode mode = FileMode.Open, FileAccess access = FileAccess.Read,
            FileShare share = FileShare.Read, FileOptions options = FileOptions.None, long preallocationSize = 0);

}

API Usage

using SafeFileHandle nullHandle = File.OpenNullFileHandle();

using var procHandle = SafeChildProcessHandle.Start(options, input: nullHandle, output: nullHandle, error: nullHandle);
procHandle.WaitForExit();

Risks

Not in the API design, but the implementations need to enforce CLOEXEC (Unix) / bInheritHandle: false (Windows) semantics on the opened handle to avoid leaking it to child processes unintentionally.</issue_description>

<agent_instructions>1. Add new public method to the ref file.
2. Introduce the actual method in the source file.
3. Open "NUL" file on Windows and "/dev/null" on Unix. If you can (file sharing could be an issue), just call "SafeFileHandle.Open".
4. Add XML doc comments for the new method. Recommend using Stream.Null in the remarks for scenarios that don't require raw file handles/descriptors.
5. The opened handle should always allow for both reading and writing.
6. Add tests
7. The tests need to ensure that every (sync and async) read always returns 0. You can use FileStream for that (created from given SafeFileHandle).
8. The tests need to ensure that every (sync and async) write always succeeds.
</agent_instructions>

Comments on the Issue (you are @copilot in this section)

@adamsitnik As pointed by @stephentoub in the API review, the tests should ensure it works with `FileStream(handle, useAsync: true)` @bartonjs [Video](https://www.youtube.com/watch?v=KpqIje2wzZM&t=1h4m53s)

• Since File is already in the type name, and the return type name, we can simplify the method name to just OpenNullHandle()

namespace System.IO;

public static partial class File
{
    public static SafeFileHandle OpenNullHandle();
}
```</body></comment_new>
</comments>

• Fixes File: low-level APIs for opening null device #122803

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[adamsitnik]

@copilot please address my feedback

[Copilot]

@copilot please address my feedback

All feedback addressed in 09dadab

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-io
See info in area-owners.md if you want to be subscribed.

[adamsitnik]

LGTM!

[Copilot]

Pull request overview

This PR adds a new File.OpenNullHandle() API to efficiently obtain a handle to the system's null device (NUL on Windows, /dev/null on Unix). This enables efficient process I/O redirection by discarding output or providing empty input without the overhead of reading and ignoring data.

Changes:

• Added new public API File.OpenNullHandle() to the System.IO.File class
• Implemented platform-specific core methods that open the null device with read/write access and non-inheritable handles
• Added comprehensive test coverage with 14 test cases covering sync/async read/write operations

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
src/libraries/System.Runtime/ref/System.Runtime.cs	Added public API signature for OpenNullHandle() method
src/libraries/System.Private.CoreLib/src/System/IO/File.cs	Added public OpenNullHandle() method with comprehensive XML documentation
src/libraries/System.Private.CoreLib/src/System/IO/File.Windows.cs	Implemented Windows-specific core that opens "NUL" device with non-inheritable handle
src/libraries/System.Private.CoreLib/src/System/IO/File.Unix.cs	Implemented Unix-specific core that opens "/dev/null" device with O_CLOEXEC (non-inheritable)
src/libraries/System.Runtime/tests/System.IO.FileSystem.Tests/System.IO.FileSystem.Tests.csproj	Added test file reference to project
src/libraries/System.Runtime/tests/System.IO.FileSystem.Tests/File/OpenNullHandle.cs	Added comprehensive tests for sync/async read/write operations and concurrent access

[adamsitnik]

LGTM!

[adamsitnik]

/ba-g infrastructure timeouts