CSI not sent to child process, other bizarre ConPTY behavior.

[amoldeshpande]

Environment

Windows build number:18362
Windows Terminal version (if applicable):

Any other software?

Steps to reproduce

I am attaching a simple program that write "\033[" and "0c" to stdout. When I use the EchoCon sample to launch this under ConPTY, I only see "0c" in the output. Breaking at the WriteFile at the bottom of the EchoCon sample, I see the following things:

1. An initial XTERM SetTitle OSC, which is unexpected but manageable.
2. A sequence of "[escape code for cursor off]" "0c" "[escape code cursor on]" (notice \033[ is missing)

code attached:

#include <Windows.h>
#include <stdio.h>

int main(int argc, char**argv) {

  char *csi = "\033[";
  char *da = "0c";
  DWORD bwrote =0;

	/*
    HANDLE hConsole = { GetStdHandle(STD_OUTPUT_HANDLE) };

 DWORD consoleMode = 0;
  GetConsoleMode(hConsole, &consoleMode);
   SetConsoleMode(hConsole, consoleMode | ENABLE_VIRTUAL_TERMINAL_PROCESSING);
   */
  WriteFile(GetStdHandle(STD_OUTPUT_HANDLE),csi,2,&bwrote,NULL);
  WriteFile(GetStdHandle(STD_OUTPUT_HANDLE),da,2,&bwrote,NULL);

}

If I uncomment the call to SetConsoleMode in the child process, it shows a bizzarre sequence of "\r\n" after the Xterm sequence.

Expected behavior

Just my \033[0c

Actual behavior

As described above. I am probably just overlooking something, but I'd love to find an explanation for this.

[zadjii-msft]

I could've sworn that I had a longer post on this somewhere in the repo, but I can't find it at the moment.

Conpty isn't just a dumb pipe connecting the commandline application and the terminal application, unlike unix ptys. While a unix pty can just be a dumb pipe, conpty needs to maintain compatibility for the entire console API, so actually has a whole console buffer that it's hosting. Then, when the buffer changes at all, we "render" the changes to the buffer as VT sequences. So, when an application starts a conpty instance, conpty starts by "rendering" its initial state - the title string, the cursor visibility, and the contents of the buffer (just a bunch of blank lines).

So you'll never get "just the \033[0c" from conpty unfortunately (unless we implement a passthrough mode for conpty). This is so that we can maintain compatibility for Console applications, but also act in a way compatible with *nix terminals.

The 0c is showing up in your example as it is because the conpty you're attached to hasn't enabled VT processing, as you've correctly determined. This is something we're thinking of changing by default, see #1965 for more on that.

[amoldeshpande]

ahh, I see. Is there a plan to add a passthrough mode ? Seems like it would be a lot simpler to write cross-platform applications that way (once someone writes a termcap/terminfo-ish layer :) )

I was trying to make a tmux version, but it looks like that's not really possible without rewriting large chunks of the original.

Edit: Shouldn't legacy mode (not enabling VT processing) just pass through all characters received on input, instead of hijacking my CSI ?

[zadjii-msft]

There are "plans" to do passthrough mode, though it's fairly low on the backlog. #1985 is the issue I've just made to track it. It's more been something I've played with in my head, but nothing set in stone.

Fortunately, conhost/conpty is a pretty good terminal itself, so anything that you emit from a client app will be handled well by conhost itself. What'll be rendered to the terminal should be equivalent to what the commandline app emitted, though it may not be as optimal.

Most *nix apps already work pretty well with conhost - take wsl for example. Tmux works great in conhost already, so what conpty will "render" to the terminal will still look right. Here's tmux running in wsl attached to a conpty that's using Windows Terminal as a head:

[amoldeshpande]

I get that Microsoft doesn't do Windows any more, but WSL is not my use case. I guess I'll try digging through the Windows Terminal source to understand how it works.

If the host process that spawned the conpty/conhost+child has no access to the raw escape sequences, I don't see how Windows Terminal is able to host various processes in a DX-accelerated GUI without frightful hacks. I have some weekend reading of source code ahead of me.

A couple of side questions (not trying derail the issue, but these will help me understand/debug conpty better ):

1. Why does Windows Terminal only run as a packaged app ? It would be really nice to be able to run it out of Visual Studio for debugging. Is that something that can be easily changed ?

2. Is there a way to spawn one's own Conhost instead of the system installed one ? I see in the source it has the ability, but it's not clear to me if that's easily done for debugging.

I do appreciate the time you guys take to quickly respond to issues and provide detailed answers. Thank you.

[zadjii-msft]

Woah now, I don't want to give anyone the impression that we don't "do Windows". The entire point of the Windows Terminal is to do better with Windows. I was merely using WSL as an example of an application that emits a large breadth of VT sequences that conhost is capable of processing. And if conhost can process them, conpty can render the effects of those VT sequences to a terminal application.

Here's a diagram I posted when ConPTY was in it's earliest phases, back in #57:

      +------------------+    +----------------------+
      | Terminal         |    |                      |
      | Application      |    | Client Application   |
      |                  |    | (cmd.exe)            |
      +-----------+--^---+    +-----------^----------+
Communication via |  |                    ||Communication over
pipes (using VT)  |  |                    ||Win32 API
                  |  |                    ||
      +-----------v--+---------------------v---------+
      |                   Conhost                    |
      |Render console to VT <------+Traditional Win32|
      |Synthesize input from VT |-->functionality    |
      +----------------------------------------------+

So a client application that only knows about how the Win32 world works would still communicate with a conhost via the API, but a terminal application could be written to read conhost state via VT and write input to the host via VT.

In your case, the client application is communicating with conhost not by the Console API, but instead with VT. Theoretically, we could pass that VT straight through to the Terminal, however, because a client application could also be attached to conhost at the same time, conhost needs to process the VT emitted by the client app first. Conhost will process that VT, so that any subsequent API calls will be able to read the state of the console correctly.

As far as your follow up questions:

1. Windows Terminal needs to be packaged because we're using UWP XAML for our UI. UWP XAML has a pretty hard requirement about running in a packaged environment. You can launch it directly from VS, with debugging, however. YOu just need to launch CascadiaPackage instead of WindowsTerminal.
2. Yep, that's what ConhostConnection is doing currently. We're shipping a conhost.exe that's built from the repo alongside the terminal app, and using that one as the conpty host instead of the system one.

[DHowett-MSFT]

(I think this is answered, so I'm going to close it. Feel free to speak up if you disagree!)

[amoldeshpande]

I guess so. \033[0c is not handled in VT mode (just echoes to the screen), but if you want to handle that separately (or at all) that's fine.