Support manually clearing the conpty buffer

[zadjii-msft]

From @Tyriar in microsoft/vscode#75141

This is happening because Terminal: Clear tells the frontend (the thing rendering everything) to clear, but the way that the conpty backend works is it will sometimes reprint what it thinks should be visible. Since it doesn't know about our clear it reprints the unwanted output, this doesn't happen on a regular linux/mac backend that conpty is trying to emulate.

Is there anything that can be done here to tell conpty to not reprint the screen? I suspect python.exe does not enter alt mode so I'm not sure why a reprint is necessary.

[DHowett-MSFT]

Thoughts: new thing to put on the signal pipe? We don't want to introduce too many non-standard VT things to the in/out pipes, so the signal pipe is our "get out of jail free" card.

[Tyriar]

I don't think this is a good idea for that reason.

[DHowett-MSFT]

Sorry, I think you'll need to be more specific about what this and that reason are.

[Tyriar]

We don't want to introduce too many non-standard VT things to the in/out pipes

[DHowett-MSFT]

So, okay, if I understand this correctly:
You're asking for us to cut this feature request, because you do not believe that we should add support for it; you do, however, still want it to exist?

[Tyriar]

I think there's a bug as @zadjii-msft indicated in the linked issue that I want fixed instead of adding non-standard stuff to conpty. Setting a variable and printing it in Python should not redraw the entire screen, see microsoft/vscode#75141 (comment) for context.

[zadjii-msft]

Well, I think the bug might be making this issue more apparent, but I don't think fixing the bug mentioned will actually properly resolve this issue. It might temporarily hide the issue, by allowing the cursor to start printing text again at the origin. However, the text wouldn't actually be at the origin in conpty's buffer, and it's entirely possible that another action could cause conpty to need to redraw everything - at which point, all the old text would re-appear, and the new text would suddenly shift to where conpty thinks it is.

For *nix, VSCode's terminal clear works just fine because the pty is just a dumb pipe - when you clear the frontend's buffer, there's nothing on the backend that still thinks that there's any text in the buffer, because there is no buffer.

However, with conpty, because there is an actual buffer on the backend, we can't just clear the frontend like that. The conpty buffer will still think something's there. Since this is a problem unique to conpty & windows, I think it's unfortunately something that needs a unique solution.
I'm half way on board with adding another signal to the signal pipe - that'll clearly work unique to Windows, since it'd introduce another API to our API surface. If we want to avoid adding another signal here, we could maybe add the clear by making conpty resond to something like ^[[3J on the input. However, I know that doing the resize on the input pipe also had some problems with making sure the resize sequence wouldn't be broken up by other input sequences, esp for WSL and docker. Hence the addition of the signal pipe. I think the signal would end up being the safest, most robust solution.

I really want to make sure we avoid API creep on the conpty API though. We need to make sure that this is something that's definitely not accomplish-able by some other means.

[Tyriar]

@zadjii-msft I get that there will be times when it fails just as it will on nix (eg. running clear in the alt buffer) and I'm fine with the Terminal Clear command not working properly in those cases. Fixing the conpty-specific cases doesn't seem worth it since it can happen off with a native pty too.

Isn't conpty meant to be pretending to be a dumb pipe here in the same way as a native pty? I tried doing that same action to run the current Python file on a mac and it will just print a bunch of characters, CR, LF and use the character attribute escape (CSI m). In a similar way it would be lovely if conpty did not reprint the screen when resized unless something actually changed, ie. a line in the viewport is wrapped or will be wrapped.

[DHowett-MSFT]

Isn't conpty meant to be pretending to be a dumb pipe here

ConPTY is pretending to be as close to a dumb pipe as it can. That's the major contention here: there is a 35-year history of win32 console applications that have access to the output buffer. We cannot be a dumb pipe, because that's not something that fits into the "dumb pipe" model.

Any concessions we've made in what ConPTY can do and does do are in the pursuit of the broadest possible compatibility.

[maxhora]

Why not to support 2 modes of ConPTY: plain "dumb pipe" and ConPTY with legacy support for 35-year history of win32 console? :)

[zadjii-msft]

@Maxris you might be interested in #1173

[yurenchen000]

it seems that:
clear or cls will clear scroll buffer in PowerShell.
cls will clear scroll buffer in CMD.

but clear OR reset OR CSI sequence not clear scroll buffer in bash.

can we have some workaround to implement buffer clear ?
some thing like,
associate shortcut keys (or something else) to call windows clear or cls to current "tty". (Bypass the current running bash.exe , ssh.exe..),

[DHowett]

@yurenchen000 Are you using Cygwin, MSYS, or MinGW bash? If so, what version of the runtime are you using? Certain versions of the Cygwin runtime delete CSIs.

Clearing the buffer works fine from bash, so long as the CSI is not deleted.

[yurenchen000]

@yurenchen000 Are you using Cygwin, MSYS, or MinGW bash? If so, what version of the runtime are you using? Certain versions of the Cygwin runtime delete CSIs.

Clearing the buffer works fine from bash, so long as the CSI is not deleted.

thanks a lot.

seems printf "\e[H\e[2J\e[3J" will clear scroll buffer, in some situations (not all).

I use lots of cli environment:

• GNU bash, version 4.4.23(1)-release //from git-bash (msys // not clear scroll buff
• GNU bash, version 4.4.23(2)-release (x86_64-pc-msys) //from MSYS2, YES this clear worked ( found it send "\e[H\e[2J\e[3J")
• adb shell
• ssh to ubuntu18

can I ignore the CLI environment,
and have a terminal itself's GUI option (mouse right menu or shortcut key) to clear scroll buffer?

[zadjii-msft]

Alright, worked on this a bit in dev/migrie/fhl-2021/clear-buffer-action. Small bits of trickiness:

• We need to preserve the cursor line. SCREEN_INFORMATION::VtEraseAll is not the thing we'd want to use to do this - that pretty blindly just scrolls away the old buffer contents into scrollback. We could:
  1. Just scroll everything above the cursor out, and leave everything below it. This would be weird for something like vim.
  2. Copy the contents of the cursor row into a temp buffer, scroll everything, then restore that line to the top of the viewport. This would also be weird for something like vim.

I mean, this almost certainly will never do the right thing for a full screen app so we may as well do i. because it's easier now.

• I should revert the changes I made to AdaptDispatch and DispatchCommon. These changes won't be necessary anymore, so better to remove them, and reduce the likelyhood that someone else uses them wrong.
• I should add a new DispatchCommon::s_ClearBuffer method, that PtySignalInputThread::_DoClearBuffer calls.
• That should get plumbed through to something just like SCREEN_INFORMATION::VtEraseAll, but with clearing everything above the cursor, and leaving the cursor on the top row of the viewport.
• Maybe just ignore doing this always when in the alt buffer? Would reduce cases where vim et al. will definitely do the wrong thing.

[ghost]

🎉This issue was addressed in #10906, which has now been successfully released as Windows Terminal Preview v1.12.2922.0.:tada:

Handy links:

• Release Notes
• Store Download