Feature Request: Clients of terminal need a way to inquire about the capabilities of the terminal

[rkeithhill]

Summary of the new feature/enhancement

Maybe this is as simple as an environment variable like VSCode uses for its terminal e.g. TERM_PROGRAM=WindowsTerminal / TERM_PROGRAM_VERSION=0.0.1.0.

Proposed technical implementation details (optional)

Start by creating the environment variables above so those of us using PowerShell can adapt our user experience (via PowerShell profiles) to Windows Terminal.

[DHowett-MSFT]

As of #897, you can look for $Env:WT_SESSION

[oising]

@DHowett-MSFT why not use TERM_PROGRAM also, like vs code, hyper and fluentterminal already do? Not too late to change :)

[vexx32]

Agreed. We can't keep creating new environment variables every single time a new terminal program comes out. Linux already uses $env:TERM and if we have to check several per platform it's just terrible from a UX standpoint. Why reinvent the wheel yet again?

[Jaykul]

An application-specific variable like WT_SESSION is really awful for anyone trying to determine what features they can use. I certainly don't want to have to check a dozen different environment variables to figure out what terminal I'm running in.

TERM_PROGRAM is slightly better, but is really just a lazy version of TERM, which is what all of these terminals should be setting.

Eventually, we're going to get a curses library and a real terminfo database with @TIC@ and @INFOCMP@ macros, shouldn't we start behaving well now?

[DHowett-MSFT]

Hold up, I think there's been a misunderstanding here. I'm sorry about that.

We should definitely support TERM or TERM_PROGRAM. Unfortunately, TERM has become somewhat like a user agent: "I work like this terminal, I promise" -- right up until it doesn't.
TERM_PROGRAM, I've never heard of. What does it do? Who is expected to use it, and for what?

WT_SESSION is something else. Today, it can be used to detect Windows Terminal. It is informative, not normative, as a detection mechanism. WT_SESSION tags an individual session -- it is a GUID that the shell (or some other enlightened application) can use in coordination with the terminal to support application state resumption. This feature was inspired by both Terminal.app (the stock terminal emulator on OS X) and its featureful replacement, iTerm2. There's no new and divergent art here.

The idea there is that if WT eventually supports reloading and showing a session's history, that resumed state will use the same session GUID so the shell can do additional things like restoring the working directory or anything else it might have saved off.

[DHowett-MSFT]

informative, not normative

This is why I threw it out as an example and didn't close the feature request. I'll seek to communicate things that are rattling around in my head a bit better going forward.

[heaths]

It would be nice if conhost would define new TERM values as well, so that we can know if we can use larger color palettes (of course, the fact it gets defined would imply that since it would be only in newer versions), differentiate conhost vs. Terminal, etc. In compiled code, it's easy enough to figure out via GetConsoleMode, but in shell scripts (cmd, powershell, etc.) not so much. For examples, flavors of linux define xterm, xterm-color, and xterm-256color, as well as many others. Some convention similar to that indicating color palette (level of VT support) or even Unicode support would be helpful to provide rich console experiences.

[be5invis]

@DHowett-MSFT Maybe both? TERM_PROGRAM for the application name, and TERM for "feature detection"?

[zadjii-msft]

Sorry, I wasn't clear. The Window in my gif is conhost, not the Terminal. There's no WT_SESSION there, because it is not the Terminal. That's just the console. Emoji can work in the console as well as the terminal. That's part of the whole design of the Terminal - a most of the improvements we make to the Terminal's buffer and internals are things that the console can benefit from as well.

[heaths]

OK, so env vars won't work in all cases. Is there another option?

Is a perfect solution necessary? It's clear from the thread that there's not really an established pattern, but TERM and COLORTERM come close. Close enough? For most indented purposes, seems "yes".

Running some EXE as suggested above or relying on Windows- or Windows Terminal-specific solutions don't make scripts portable, or at least make them harder to write (would always need a precondition of WT_SESSION which, as @zadjii-msft points out above, also isn't accurate).

Seems the sooner a solution is added that is close to what most other terminals do, the less workarounds get invented that might be even worse.

[j4james]

@heaths If you're just trying to detect true color support, you should be able to use the standard method documented here:

https://gist.github.com/XVilka/8346728#querying-the-terminal

I believe that should be supported in Windows Terminal from version 1.12.

[heaths]

My original reason for jumping onto this thread was to find an easy way a script or program can query. Even the suggested gist - which I had found previously - doesn't always work. Even if you replace : with ; in Windows Terminal, the response you get back isn't the same:

echo -e '\e[48;2;1;2;3m\eP$qm\e\\'
^[P1$r0;48;2;1;2;3m^[\

The gist shows "r;48" instead of what Windows Terminal reports back, "r0;48". So depending on what people are checking for when reading back, it seems to vary from terminal to terminal. Whether you use : or ; certainly does. For example, using : in WT gives back ^[P1$r0m^[\, which indicates WT doesn't support truecolor.

This makes it more difficult to write portable shell scripts as well, where an env var is pretty straight forward to use.

[eryksun]

The defterm scenario is never going to work with WT_SESSION. When the Terminal is invoked for a defterm connection, cmd.exe is already running. At this point, it's environment variables are already set. It's started, so there's nothing WT can do to change them now. Then, the OS tries to create a console to host the cmd process. conhost does some work, and hooks up the cmd.exe to the Terminal (or some other terminal!). The Terminal can only add the WT_SESSION variable when WT is the parent, launching cmd.exe directly.

When a console is allocated on the client side, the internal function ConsoleCreateConnectionObject() calls NtCreateFile() to open the connection to conhost. The connection request triggers the handoff in conhost. After connecting, the client side could call NtDeviceIoControlFile() with an IOCTL that requests session environment variables that need to be set (e.g. "TERM", "WT_SESSION", "WT_PROFILE_ID", "WSLENV"). The base API could also make this IOCTL when the console connection is inherited, or when attaching via AttachConsole(). This would ensure that clients always start with relevant session environment variables. The ConPTY API would include functions to get and set these session environment variables in the console server.

For handoff, the delegated console (openconsole) would need to obtain the session environment variables from the delegated terminal when it calls handoff->EstablishPtyHandoff(...), which could be either from an out parameter or a subsequent COM call. This guarantees their availability when the base API requests them.

[DHowett]

I actually really like this. We've been thinking about what it would mean to extend the Console API surface, too, with both inbox consumers (conclnt) and out-of-box ones. The Windows release cycle gives us a bit of trouble, but since DefTerm is locked behind Windows updates as well ... it isn't as much of a problem. Interesting.

[Jaykul]

@heaths wrote:

echo -e '\e[48;2;1;2;3m\eP$qm\e\\'
^[P1$r0;48;2;1;2;3m^[\

The gist shows "r;48" instead of what Windows Terminal reports back, "r0;48".

The leading 0; is the SGR reset which "returns all attributes to the default state prior to modification".

The response for DCS $ q Pt ST according to ctlseqs is supposed to be: DCS 1 $ r Pt ST

When the query Pt is m you're going to get the SGR back as the Pt. Windows Terminal is returning: 0;48;2;1;2;3m which is the reset plus the background. Some other terminals may leave off the reset. For our purposes, you can (should) ignore it. You set the default background (48), so what you're trying to read is the value of 48.

I wrote Test-RgbMode for example, but I don't have a wide test base to verify it works.

[heaths]

Thanks for the explaining the 0 there. In hindsight I probably should've realized that (I tend to reset colors to be sure myself), but wasn't entirely sure what the printed sequence was representing.

But looking at your gist shows the complexity that any script would have to do in Windows Terminal as opposed to most other terminals with a combination of TERM and COLORTERM. Given the acquisition model for Windows Terminal (most people probably get it through the Windows Store) and update push model, is there any big downside of assuming the vast majority of customers have the latest and TERM and COLORTERM are enough? My concern is more about portability of scripts with common terminals and less about 100% accuracy. Wouldn't you agree that the vast majority of devs wanting to detect capabilities are probably most interested in terminal colors? Sure there's some one-offs for detecting if hyperlinks are supported, etc., but given that most customers would have the latest WT simply checking TERM for, say, "xterm-256color" or whatever it would be set to should be approximate.

Even the detection mechanism you demo above has variants for terminals like using : instead of ; so the logic becomes even more complicated. It's all possible, of course, but you start turning otherwise small shell scripts into much larger ones.

[j4james]

@heaths Note that there are three different truecolor formats, and terminals don't necessarily support all of them. If COLORTERM is set to truecolor or 24bit, that indicates that the terminal might support some form of truecolor, but it doesn't tell you which formats will work. It's probably reasonable to assume the semicolon format, but that's not guaranteed.

For example, using : in WT gives back ^[P1$r0m^[\, which indicates WT doesn't support truecolor.

No - that's indicating that WT doesn't support the colon format, which is exactly the point. A COLORTERM variable wouldn't tell you that.

That said, if you're happy with the all the limitations of environment variables, that's fine. I just want to make sure you are actually aware of those limitations.

[heaths]

I appreciate the limitations, yes. Not only have I been following this thread, but working with @DHowett offline with some GitHub CLI (and related modules) issues, along with a couple GitHub developers. The main reason I brought up my concern/question above was questioning whether or not the limitations of env vars like TERM and COLORTERM are worth it for the vast majority of cases. I posit: probably not. In scenarios where those limitations may be problematic, certainly app/script devs can query caps from the terminal as you've suggested - dealing with all the portability issues across shells.

Scenarios where env vars are probably good enough would be colors, I would think. Let's say that you only set TERM=xterm-256color and some shell decides to use 256 indexed colors instead of truecolor even though they could. Is that detrimental? Or if they merely check TERM for "xterm" to assume it supports OSC 8 (hyperlinks), WT would as long as they have a fairly recent (1.10?) version, which given Windows Store's (eventual) push model for updates is likely, especially as more time passes.

IMO, I just don't see a great reason to avoid using common TERM and COLORTERM even if they aren't 100% correct. For simple scenarios they should be good enough, as they have been even before Windows Terminal despite some differences across various terminals. When it matters, devs should be encouraged to query caps on the terminal. Both can be offered.

[tracker1]

@j4james "Running under WT?" is a good enough emoji rendering support detection algorithm as far as I'm concerned.

Not for people actually making terminal programs that won't always be run in windows... not to mention there are different terminals for windows itself. WT_* may indicate that it's "Windows Terminal" that said it's a horrible experience when trying to support different terminals, or possibly even trying to create something that can detect features.

It's very similar to at least being able to get the application name (TERM_PROGRAM) and the version (TERM_PROGRAM_VERSION) so that you can handle, work around or determine specific implementation bugs, in a consistent way with other environments.

Specific, similar example.... Internet Explorer 5.0.0 had a very specific bug in which it implemented a new interface for managing the options in a <select> element that was fixed in 5.0.1 ... but at the time, it was written to every Windows 2000 and Office 2000 cd... that's an example of a very specific work around... that said, it happens more often than most would like to think.

In this case, the terminal is effectively a browser for a command line application, including the shell environment.

Some features that concern me that would be nice to be able to determine in a consistent way....

• Color support, 16, 256, rgb including either/both delimiter options
• UTF-8 or not, can we find out what the character set is otherwise?
• Emojii (none, single color glyph, or color emojii)
• Image rendering support/options
• hyperlinks
• Regions?
• screen size
• screen resize

Doing so in the most consistent way, termcap library support for WT, etc. But short of having at LEAST the TERM_PROGRAM, it's not at all consistent with other terminals.

For that matter, maybe a settings option for adding/setting/overriding custom environment variables, so the user can CHOOSE to add appropriate options for their environment (wsl, ssh, etc).

[j4james]

Color support, 16, 256, rgb including either/both delimiter options

You should be able to determine this with a DECRQSS query.

UTF-8 or not, can we find out what the character set is otherwise?

This could probably be detected by ouputting a UTF-8 character and then measuring the consumed space with a DSR-CPR query.

Image rendering support/options

You can determine Sixel image support from the DA1 report.

hyperlinks

Not possible yet, but I'm hoping to persuade other terminals to use DA1 for this to.

Regions?

Not sure what you mean.

screen size
screen resize

If you mean you want to determine the size of the screen, and whether you can resize the window, a DSR-CPR query would probably suffice.

[heaths]

The issue I was trying to raise in the last couple comments above, though, is why writing to a TTY and checking the results would be the most accurate way to determine capabilities, are you expecting that every program and, more important, every shell script (especially those that want to remain simple but maybe write out some colors) have to write to the TTY, read back, then clear the line just to emit some colored output? The environment variables like TERM, COLORTERM, and TERM_PROGRAM may not be perfect but I imagine for most cases are good enough. So why not support both?

[j4james]

@heaths I'm not expecting anyone to do anything. I'm just saying there are queries you can use if you need an accurate way to determine those features. If asynchronous queries are not appropriate for your application, or you don't particularly care whether your feature detection is accurate or not, then you can of course use any other method you prefer. As I said before, if you're happy with the limitations of environment variable, that's fine - whatever works best for you.

[tig]

Hi all, maintainer of https://gitub.com/gui-cs/Terminal.Gui here.

We would really like a way to ask terminals (not just Windows Term) if a specific unicode glyph is supported at runtime.

The idea being, we have to choose a least-common-denominator glyph for things like the one used for checkboxes. In this example, we choose the square root symbol as it empirically works on every terminal we've tried, with every font we've tried, where other "check mark" symbols do not.

What we'd like to do is emit a DECRQM asking, "Can you render this pretty glyph"? If the answer is no, we'll fall back to our default. If the answer is yes, we'll use the prettier glyph. We could also do this via querying an environment var or something, but it seems to me the most cross-platform way to do this is to extend DECRQM to support such a query?

I know it is possible to ask (at least on Windows) if a particular font supports a particular glyph.

I found this Issue in my searching for existing solutions to this and it seems like a good place to start. Let me know if you have suggestions on how to proceed further. My dream (big dreamer here) is other terminals will also implement such a thing in a standard way...

[j4james]

@tig If I remember correctly, there was a some discussion a while back about coming up with a terminal standard for querying Unicode glyph support. I may be misremembering the details, and I haven't been able to track down the thread where it was discussed, but I think it would probably have covered your use case if it were ever implemented.

That said, this isn't something that Windows Terminal could easily implement, even if such a standard did exist, because these query sequences are handled in conhost, which doesn't know anything about the fonts that the actual conpty client is using. So this might be one of those things that would only work with a pass-through mode (#1173).

In short, it's possible this might be feasible one day, but unlikely in the near future.

[tusharsnx]

It's clear that setting $TERM is not sufficient when an application wants to know if terminal supports FeatureX.

I guess we can have a terminfo for WT, for all kinds of *NIX applications. We can always be TERM="xterm-16color" or even TERM="" (whatever the safest option is) and totally support extra capabilities by announcing it via terminfo.
User has to do one time installation of WT's terminfo on their favourite distro (by curl and install) to support all new capabilities. If that's not possible for whatever reason, you are pretty much using the same terminal what you get today without any modifications.

This is supposed to done by terminals and not ConHost or ConPTY. For any terminal out there, if they want to support all capabilities of ConHost then they should need to announce it. ConHost won't do it for them. Applications should just behave the way they do right now when terminfo is not found or they don't recognise the terminal.

Remaining problems:

• What should ConHost/ConPTY do when default terminal is invoked?
• What should be the mechanism for announcement with Windows console applications? (Assuming terminfo isn't widely used on Windows.)