Add public Nt/Rtl header (winternl)

[JeremyKuhne]

Important advanced APIs here such as NtCreateFile, NtQuery*, and NT string APIs.

[tannergooding]

Could you clarify where these APIs are intended to be used and how?

Many of the Nt/Rtl functions are intended for writing drivers, not user mode applications, and can have considerable overhead compared to the user-mode equivalents as each represents a system call and therefore a context switch into the kernel.

[tannergooding]

I also think that these are ones that possibly shouldn't be exposed due to: https://docs.microsoft.com/en-us/windows/win32/devnotes/calling-internal-apis.

That is winternl is effectively considered "internal implementation detail" to user-mode code.

[JeremyKuhne]

The likelihood of them changing is about zero. They've not changed since they were introduced (which in many cases is 30 years ago). The .NET Framework itself uses some of these, notably NtCreateFile()- it's the only way to open/create a file relative to an existing handle, which is a huge performance advantage.

They are all there because there is no way to accomplish some of the things they provide through the Win32 surface area. Either at all, or as efficiently.

As to more expensive, they aren't any more expensive than other Win32 APIs. The Nt* APIs are user mode NT APIs, Zw* equivalents are the kernel mode ones. They only thunk through where they absolutely have to, no different than normal Win32 APIs, outside of there is yet another level on top of the actual NT (native) API. Win32 is just a subsystem (projection) as OS/2 and POSIX were originally. Having to fit the original Win16 model, the Win32 projections sometimes have to add significant overhead to translate between NT data types and Win32 data types.

"Using Nt and Zw Versions of the Native System Services Routines" talks a bit more about this. Anything that is documented in Zw* form is fair game to call in its Nt* format. If there is no documentation on either end, then there more of a plausible risk to consuming them.

[tannergooding]

They only thunk through where they absolutely have to, no different than normal Win32 APIs

That doesn't look to be the case in the headers or source. They are explicitly documented as __kernel_entry and from the handful I looked at the source for are effectively just trapping kernel mode and dispatching to the relevant kernel-side functionality.

it's the only way to open/create a file relative to an existing handle, which is a huge performance advantage.

Could you provide some more details and an example of what you mean here? I don't see any non-deprecated functions that take in an existing file handle and you can DuplicateHandle if you need a secondary version of an existing handle.

---

What about the Rtl functions. These are generally extensions of the CRT and we generally have equivalent or better alternatives in managed directly (or exposed as non Rtl variants of the API). I've explicitly avoid exposing most of the RTL stuff so far because of this.

[tannergooding]

The .NET Framework itself uses some of these, notably NtCreateFile()- it's the only way to open/create a file relative to an existing handle, which is a huge performance advantage.

It looks like it basically just uses this for directory enumeration which would likely be better handled by FindFirstFile and FindNextFile. Most of the relevant information, particularly the name, doesn't need you to open and close a full handle; the Directory Enumeration APIs will do it for you and do it efficiently.

[JeremyKuhne]

That doesn't look to be the case in the headers or source. They are explicitly documented as __kernel_entry

For the ones here, yes. They (like any IO api) have to go to kernel so there is no advantage to having a user mode equivalent. The Rtl ones are user mode.

Could you provide some more details and an example of what you mean here?

The OBJECT_ATTRIBUTES takes a RootDirectory for the path name. That allows skipping a good chunk of code to normalize the path name and walk through the alias to the right root and walk down the directory chain in the driver. Waaaay faster.

It looks like it basically just uses this for directory enumeration which would likely be better handled by FindFirstFile and FindNextFile. Most of the relevant information, particularly the name, doesn't need you to open and close a full handle; the Directory Enumeration APIs will do it for you and do it efficiently.

It is significantly faster for a number of reasons.

• The skipping of navigating as described above.
• FindFirstFile creates a new buffer every time you use it, this has significant performance implications when recursively navigating.
• The data structures don't match what the NT API returns, which requires translating (copying) the data.

I spent months redoing the code here to avoid all of this overhead. :) (Lots and lots of perf testing and walking through the Windows sources.)

What about the Rtl functions.

If there are CRT equivalents it's fair to exclude them. I don't think there are any for these. UNICODE_STRING is a Windows thing. All of these have some valid use. RtlNtStatusToDosError is a more "correct" choice to convert NTSTATUS to Win32 system error codes (ERROR_*, etc.) than the other more obscure LsaNtStatusToWinError, at least from a documentation perspective.

I'll also point out that all of the Zw* documentation (ZwCreateFile for example) has a block that says:

"Note If the call to this function occurs in user mode, you should use the name "Nt*" instead of "Zw*"."

While it is primarily documented from the kernel mode usage perspective (there is, after all, no Win32 in kernel), there is no reason you can't use the actual NT APIs from user. If you need functionality that isn't projected into Win32 you have no choice, and in some cases you save a ton of overhead (all of the Win32 APIs call Nt*, Rtl* and equivalent APIs under the covers).

I'm not suggesting that everything be done directly through Nt APIs, but using them is completely legit and sometimes the right, or even the only, choice.

[JeremyKuhne]

FWIW I've been putting up PRs for headers I'm actually using in my WInterop project.

[tannergooding]

Sorry for the delay. I had set this to auto-merge but forgot to actually approve it so the merge could happen.