Why tokio uses blocking file io under the hood?

[AngelicosPhosphoros]

General
As I see, tokio uses blocking IO on filesystem.
Calls here a sys::run.
Which is actually a spawn::blocking.
Which runs blocking code on special thread pool.

Is it intentional or just temporary solution? If first, why we need to use blocking fs API when there is async one?

Describe the solution you'd like
I am new at async programming so would describe only Windows version of solution. However, I think there is a Linux version too.

I suggest to use either OVERLAPPING IO or Input-Output Completion Ports.

AFAIK, you are already using IOCP for network in mio but I don't know how complex to integrate file IO with this.

For the overlapped IO I made little example. It is quite ugly but enough to show basic schema.
The overlapped IO requires using separate thread for handling FS IO but I think it is still better than few threads from pool blocked on FS.

Additional context

According to this table latency of disk read can be 12 times slower than network request so why use async on net and threads on disk?

Maybe I can help?

If you like to start using this file async APIs, I can offer my help. I can start from implementing Windows APIs then do Linux thing, however, I can't do anything for MacOS or other OSes.

However, I will need some guidance:

1. What exactly better to use: IOCP or Overlapping IO?
2. If use IOCP, how can I match it with mio queue?
3. How implement it? In tokio or in some new crate like mio-fs? As I see, there are not much platform specific code in tokio.
4. How handle from_std method? As I see, it would mostly provide synchonous files: https://github.com/rust-lang/rust/blob/ccea570488694e502c1a7ca8f3f0866c9b763ce4/library/std/src/sys/windows/fs.rs#L171 Maybe run sync files using blocking executer and use async only if they created by tokio? Or deprecate or remove ability to use std::fs::File and let users explicitely use tokio blocking tasks?

[Darksonn]

I'm not super familiar with the situation on Windows, but there is no suitable async file IO support in Linux besides the very new io_uring API, which is only supported on the newest Linux kernels.

[AngelicosPhosphoros]

What do you mean by no suitable?

As for Windows, .Net developers use Overlapped IO:
https://github.com/dotnet/runtime/blob/master/src/libraries/System.Private.CoreLib/src/System/IO/FileStream.Windows.cs
https://github.com/dotnet/runtime/blob/master/src/libraries/System.Private.CoreLib/src/System/IO/FileStreamCompletionSource.Win32.cs

Their code quite complicated however. I believe, it is because their FileStream class can be open in async and sync modes and must support both sync and async interfaces in both modes. I think, we don't need to support sync interface at all.

[Darksonn]

You can find some details on io_uring in Tokio here #2411. However note that we are currently focusing on getting 0.3 and 1.0 out in 2020, so io_uring has been postponed until we have finished that transition as described in #2692 (comment).

[ghost]

Windows Overlapped I/O (including using it with IOCP) is not a perfect solution:
https://docs.microsoft.com/en-us/troubleshoot/windows/win32/asynchronous-disk-io-synchronous#asynchronous-io-still-appears-to-be-synchronous
It's also not used in libuv, but it's used in Boost.Asio and .NET.

[AngelicosPhosphoros]

There are also another caveat of Overlapped IO
https://stackoverflow.com/a/44665663/8195987
TL;DR: It limited by 64 requests by thread.

So if tokio will use Windows async file I/O, it is better to use IOCP for it.

[carllerche]

Closing due to inactivity.

If someone wants to make this happen, I would suggest opening a proposal issue w/ implementation details and a discussion on how to manage IOCP file ops still being synchronous at times.