Officially support stdin

[Chuongv]

Opening an issue as per your request: It would be nice if mio could support stdin

[dpc]

It is possible to handle stdin in mio, just in case that's a problem. I have a project that does just that: https://github.com/dpc/colerr/blob/master/src/iomuxer.rs#L26

But I guess some wrappers to support Windows etc. would be nice, yes. Obviously stderr and stdout should be supported too.

[carllerche]

This is going to require more research into how STDIN & co works w/ IOCP. I will tentatively assign this to the 1.0 milestone, but will potentially have to punt if it is tricky.

[alexchandel]

It would be nice if mio added both stdin and file operations. It's justifiable since mio is already a good interface to kqueue/epoll/iocp. Pipes and files are treatable by kqueue and epoll similar to network operations. And files (and named pipes) work with iocp just fine.

Unfortunately standard handles are the exception, and require extra treatment. For some horrible Windows reason, "console" handles and anonymous pipes don't work with IOCP. For console input, there is GetNumberOfConsoleInputEvents, PeekConsoleInput, or WaitForMultipleObjects, and for anonymous pipe input, there is PeekNamedPipe or WaitForMultipleObjects.

[raphlinus]

I'll put in a vote for this. I'm currently using a thread-heavy design in xi-editor, which is the source of more overhead than I'd like. Now that futures-rs is out, basing my RPC mechanism on that is very appealing, but I also very much don't want to have separate code paths on Windows.

A small amount of discussion in this reddit thread.

[alexchandel]

stdin probably has to be treated separately, because the Windows implementation must be different for stdin. But it is possible to read stdin non-blocking on Windows. I believe it must be polled with WaitForMultipleObjects.

[timClicks]

Decided to look into this and take a look at how a few other projects have proceeded. Ended up with a 1,700 word blog post on IOCP and stdio.

@alexchandel I didn't encounter much use of WaitForMultipleObjects, but then again I am not very familiar with the Win32 API. If you could can anything, feel free.

[alexchandel]

@timClicks

Your tldr was about my conclusion as well.

The main problem is that keys don't arrive to the "console stdin" of a process as a byte stream, like they do on normal OSs. Instead, when you type into a "console window" on Windows, they arrive in some Win32 input event queue of the process, along with mouse events, menu events, context-change events, and other garbage. This queue receives them in the form of key-up and key-down events, and even receives them for backspace and the arrow-keys.

When you call getc, fread, std::getline, ReadFile, ReadConsole, or anything else that should read characters from stdin, those call into a Win32 function that processes the input queue, which includes handling arrow-keys, backspaces, command history, and line discipline, and which eventually returns a character or line (depending on what you asked for) or blocks until it can.

Note that cmd.exe itself doesn't handle any key input. Note that there is no Win32 function that will perform this input event processing but NOT block if no character/line is available. Note that not even conhost.exe, the process that draws the "console" window in the first place (cmd.exe is just a batchfile language interpreter really), can be told to parse keyboard events and send them as a byte stream.

This horrible, atrocious design is why cmd.exe cannot ever have command history, why lines don't appear when you type them unless the process is blocking reading, and why you cannot do IOCP on "console stdin". You need a thread to process the events & do line discipline.

What you can do with WaitForMultipleObjects is poll the input queue, and when events are available, filter out the keyboard events and use those as character inputs in lieu of getc. Since you're just taking the key-ups (or key-downs) directly, that means character-echoing, arrow-keys, and backspace won't work, unless you explicitly implement your own input buffer and echo it back to the console. So it's a nasty hack, and probably useful only for console/terminal games like nethack or dwarf fortress, which obviously don't need line discipline, line history, arrow-keys, key-echoing, or anything else.

In cases where you can't poll, or where filtering key events is a pain or too costly to do on your thread, creating another thread that reads stdin blockingly and posts it in on a named pipe/tcp port/IOCP-compatible thing seems like the only way.

The Twisted solution in your blog post doesn't work in the general case, since you need to change your parent/input source as well, which is generally impossible (e.g. for conhost.exe, the most important case).

[alexchandel]

With regard to mio/tokio, the best approach is probably for mio to create a "stdin thread" on Windows if the input is a console input handle or an anonymous pipe, and feed that input into an IOCP, since mio already uses IOCP internally on Windows.

The issue is that environments with line discipline (e.g. the default state of a terminal emulator, and I believe of Win32 when reading a console handle) correspond to calling readline or ReadConsole on a Win32 console handle, whereas environments without line discipline (e.g. terminal emulators that have been configured to provide immediate input, an anonymous pipes, a named pipe, etc) correspond to calling ReadFile or maybe getc on a Win32 console handle (theoretically one could poll WaitForMultipleObjects here, but mio doesn't poll and polling is ugly). I believe the "console mode" of ReadFile is to read until a carriage return on Windows.

[carllerche]

Thanks all! The answer is going to be "out of scope for mio".

Mio provides all the necessary hooks to add stdin support out of crate. This lets others make decisions that work for them and Mio doesn't have to take a stance.

[AlanFoster]

Decided to look into this and take a look at how a few other projects have proceeded. Ended up with a 1,700 word blog post on IOCP and stdio.

Looks like this post is now found here: https://tim.mcnamara.nz/post/176613307022/iocp-and-stdio