-
Notifications
You must be signed in to change notification settings - Fork 4.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Set of offset-based APIs for thread-safe file IO #53669
Conversation
…Helpers to SafeFileHandle itself
…le.OpenHandle to FileStreamHelpers
…omFile.ReadAtOffset are going to need it)
…s it's a Windows-specific concept
…Seek for regular files
Co-authored-by: Stephen Toub <stoub@microsoft.com>
* with NtCreateFile there is no need for Validation (NtCreateFile returns error and we never create a safe file handle) * unify status codes * complete error mapping
src/libraries/System.Private.CoreLib/src/Microsoft/Win32/SafeHandles/SafeFileHandle.Windows.cs
Outdated
Show resolved
Hide resolved
src/libraries/System.Private.CoreLib/src/System/IO/File.netcoreapp.cs
Outdated
Show resolved
Hide resolved
src/libraries/System.Private.CoreLib/src/System/IO/RandomAccess.Unix.cs
Outdated
Show resolved
Hide resolved
src/libraries/System.Private.CoreLib/src/System/IO/RandomAccess.Windows.cs
Outdated
Show resolved
Hide resolved
src/libraries/System.Private.CoreLib/src/System/IO/RandomAccess.Windows.cs
Outdated
Show resolved
Hide resolved
Co-authored-by: Stephen Toub <stoub@microsoft.com>
{ | ||
public byte* Base; | ||
public UIntPtr Count; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: nuint
src/libraries/Common/src/Interop/Windows/Kernel32/Interop.FileScatterGather.cs
Outdated
Show resolved
Hide resolved
|
||
int64_t count = 0; | ||
int fileDescriptor = ToFileDescriptor(fd); | ||
#if HAVE_PREADV && !defined(TARGET_WASM) // preadv is buggy on WASM |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's open an issue in runtime for now and include the link here.
@@ -41,6 +41,14 @@ typedef struct | |||
// add more fields when needed. | |||
} ProcessStatus; | |||
|
|||
// NOTE: the layout of this type is intended to exactly match the layout of a `struct iovec`. There are | |||
// assertions in pal_networking.c that validate this. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we move them to here?
runtime/src/libraries/Native/Unix/System.Native/pal_networking.c
Lines 163 to 168 in c87354c
// We require that IOVector have the same layout as iovec. | |
c_static_assert(sizeof(IOVector) == sizeof(iovec)); | |
c_static_assert(sizeof_member(IOVector, Base) == sizeof_member(iovec, iov_base)); | |
c_static_assert(offsetof(IOVector, Base) == offsetof(iovec, iov_base)); | |
c_static_assert(sizeof_member(IOVector, Count) == sizeof_member(iovec, iov_len)); | |
c_static_assert(offsetof(IOVector, Count) == offsetof(iovec, iov_len)); |
src/libraries/System.Private.CoreLib/src/System/IO/RandomAccess.Unix.cs
Outdated
Show resolved
Hide resolved
src/libraries/System.Private.CoreLib/src/System/IO/RandomAccess.Windows.cs
Show resolved
Hide resolved
src/libraries/System.Private.CoreLib/src/System/IO/RandomAccess.Windows.cs
Outdated
Show resolved
Hide resolved
src/libraries/System.Private.CoreLib/src/System/IO/Strategies/SyncWindowsFileStreamStrategy.cs
Outdated
Show resolved
Hide resolved
src/libraries/System.Private.CoreLib/src/System/IO/Strategies/SyncWindowsFileStreamStrategy.cs
Outdated
Show resolved
Hide resolved
Co-authored-by: Stephen Toub <stoub@microsoft.com>
@adamsitnik this change has broken my local build on Apple Silicon. It fails with:
However the checks for HAVE_PREADV and HAVE_PWRITEV in the configuration have succeeded. I suspect that the functions have moved to a different header file in recent SDK and the pal_io.c doesn't include it. But I am not sure yet. |
Btw, this was a clean build after git clean -xdf. |
I can see that the preadv / pwritew in my SDK are available only if: |
Nevermind, I see you already checked the headers meanwhile. Also I realized that the Apple Silicon version may target macOS 11 as minimum (unlike the x64 one). |
Adding |
I've moved all the logic responsible for opening and initializing
SafeFileHandle
toSafeFileHandle
. A lot of duplicated code (mostly for initializingThreadPoolBinding
) got removed. On Unix, it's also now much more clear how many syscalls we perform to just open a file.Other changes:
NtCreateFile
for cases whenpreallocationSize
was specified andCreateFileW
when not, I've followed @JeremyKuhne suggestion and switched to usingNtCreateFile
only. This has simplified the code and provided some minor perf gain. I've discovered a bug in the previous implementation (related toDesiredAccess.DELETE
) fixed it, and added a testfstat()
to ensure that a given file is not a directory we can also determine whether a given file is seekable or not. This has removed one syscall for the initialization ofCanSeek
.Fixes #24847
@carlossanlop @jozkee @stephentoub PTAL. Once this PR is merged, I would like to finally publish the .NET Blog post.
cc @alexbudmsft