Taskbar progress reporting via ANSI codes #14615

Gordon01 · 2024-09-29T20:23:37Z

What does this PR try to resolve?

A few terminal emulators support progress output to Windows taskbar. winget uses this to show install progress.

Notably, Windows Terminal recently (2020) added support for ANSI codes specified in ConEmu (another terminal emulator for Windows) documentation. Also, in "Learn Windows".

I've found the previous attempt to add this feature: #11436

As per @weihanglo's request, I've added the config option to enable/disable this feature. It's enabled on supported terminal emulators.

Fixes #11432

FCP: #14615 (comment)

How should we test and review this PR?

Run cargo build in Windows Terminal with configuration option term.progress.taskbar set to true.

Not sure

Should all the code be #[cfg(windows)]? Probably no, because the feature is also usable in WSL.

Solved by introducing heuristic based on environment variable set by terminal

If Ctrl+C is pressed, a progressbar will stay in a last state forever (shown in the ConEmu video). winget is also behaves like alike. I've experimented with ctrl_c handler and it's totally fixable.
Enabled is a sensible default for WSL because it works on linux builds in Windows Terminal too

Solved by introducing heuristic based on environment variable set by terminal

Downloading stage may produce unpleasant blinking due to a rapid 0-100 changes

Solved by not displaying bar when downloading and using indeterminate state in other cases so amination don't reset

TLDR

An term.progress.taskbar option with bool type is added
On Windows Terminal and ConEmu is enabled by default
If enabled reports build progress to taskbar icon and/or tab header

Videos

Windows.PowerShell.2024-09-29.23-21-21.mp4

cmd.2024-09-29.23-36-25.mp4

rustbot · 2024-09-29T20:23:43Z

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @epage (or someone else) some time within the next two weeks.

Please see the contribution instructions for more information. Namely, in order to ensure the minimum review times lag, PR authors and assigned reviewers should ensure that the review label (S-waiting-on-review and S-waiting-on-author) stays updated, invoking these commands when appropriate:

@rustbot author: the review is finished, PR author should check the comments and take action accordingly
@rustbot review: the author is ready for a review, this PR will be queued again in the reviewer's queue

src/doc/src/reference/config.md

src/cargo/util/context/mod.rs

src/cargo/util/progress.rs

weihanglo · 2024-11-15T04:30:41Z

@rustbot author

Gordon01 · 2024-11-18T11:22:28Z

Hello, Ed, thank you for the review and sorry for the late feedback. It's been a lot of work lately.

epage · 2024-11-18T16:46:59Z

Hello, Ed, thank you for the review and sorry for the late feedback. It's been a lot of work lately.

Understandable!

epage · 2024-11-18T16:50:10Z

src/cargo/util/progress.rs

+        // From https://conemu.github.io/en/AnsiEscapeCodes.html#ConEmu_specific_OSC
+        // ESC ] 9 ; 4 ; st ; pr ST
+        // When st is 0: remove progress.
+        // When st is 1: set progress value to pr (number, 0-100).
+        // When st is 2: set error state in taskbar, pr is optional.


From #14615 (comment)

This also raises another small issue: if user stops cargo with ctrl+c, taskbar progress is not cleared and it stays until the next invocation of a program with taskbar progress report capability. Microsoft's own winget behaves like this.
I've tried to set ctrl+c handler in cargo and clear taskbar from there. It works, but it 50-100 LOC which would be used rarely. What do you think of this?

I am hesitant to have a ctrl+c handler but I also have had enough bad experience on Windows with colors being leaked from a program and could see myself being annoyed about progress state leaking as well.

Here's my prvious work on ctrl+c handler: de22a7f

The approach works, but then we need to set the handler depending on Progress.state. Can you please advice on this? Do you want to have a handler in src/bin/cargo/main.rs ?

@ehuss I'd be interested in your opinion on this.

Since we're now detecting a terminal regardless of the OS, a ctrl+c handler needs to be portable too, not like in my first implementation. If we still want to go this way, a few questions:

Can we use external crate like https://crates.io/crates/ctrlc? I assume your policy is to use as few external crates as possible

Will it iterfere with https://github.com/rust-lang/cargo/blob/master/crates/cargo-util/src/process_builder.rs#L259-L276? I don't fully understand when this code is used.

It is used when cargo executes binaries for cargo run or cargo <plugin>. Not familiar with Windows but I would assume for these execvp cases we are not expected to see them returning.

src/cargo/util/progress.rs

epage · 2024-11-21T19:58:01Z

This is looking really great and making me jealous that my terminal doesn't support it!

Gordon01 · 2024-11-27T16:18:49Z

@rustbot review

Gordon01 · 2024-12-07T12:03:12Z

Hello, @epage and everyone interested. Hope, you're well!

I have a good news! The progress in standartisation of the sequence is going well. Soon it's gonna be supported by systemd and GNOME vte. Here is the message from Christian Persch:

So my conclusion after reading through this issue, the windows terminal issue/pr, and conemu implementation, is that while this API has flaws and might be improved, it's reasonably widely implemented, and small enough (and the flaws small enough) so as to make it not worthwhile to try to invent yet another protocol for this.
Therefore I'm going to commit a patch to support OSC 9 ; 4 to vte (plus hook it up in the test app). I've changed it a bit since the last wip/progress branch; please check and test if this still meets the requirements. The parser at least should fully copy conemu behaviour.

And, cosequently support to Ptyxis was added.

From Lennart Poettering:

Since a while systemd has been showing nice terminal progress bars when doing certain slow operations (for example, when systemd-repart initializes a disk). With v257 we go one step further with this. Whenever we show the progress bar we'll now also issue certain terminal ANSI sequences that tell your terminal emulator that a slow operation is going on and what the progress currently is.

And the bug in kitty about the conflict with their OSC 9 implementation was fixed here. Also in foot.

So, to sum it up, Ed, it looks like you will get the support in your terminal earlier than you anticipated.

weihanglo · 2024-12-07T13:41:47Z

Thank you for driving this!

I use Kitty so may gain nothing, but still a great feature!

src/cargo/util/progress.rs

weihanglo · 2024-12-07T14:57:47Z

src/cargo/util/progress.rs

+        // From https://conemu.github.io/en/AnsiEscapeCodes.html#ConEmu_specific_OSC
+        // ESC ] 9 ; 4 ; st ; pr ST
+        // When st is 0: remove progress.
+        // When st is 1: set progress value to pr (number, 0-100).
+        // When st is 2: set error state in taskbar, pr is optional.


It is used when cargo executes binaries for cargo run or cargo <plugin>. Not familiar with Windows but I would assume for these execvp cases we are not expected to see them returning.

src/cargo/util/progress.rs

weihanglo · 2024-12-07T15:03:04Z

src/cargo/util/progress.rs

If we feel too risky (like excessive notification on Kitty), should we roll this out only on nightly for a while?

I agree. It will take time for linux users to upgrade their terminals. And who knows what other bugs systemd will catch. I definitely want people to associate Rust ecosystem with quality.

Unsure if this is needed. This feature is "auto" by default and the only places where it auto-enables are inside of terminals that are known to support this feature.

So the only way this would be a problem is if the user does CARGO_TERM_PROGRESS_TASKBAR = true cargo ... and that is by design so people can experiment with this feature and report back on support so we can expand it.

When it comes to a terminal having bad behavior in an old version and good behavior in a new, we can sometimes do version detection, e.g. https://github.com/zkat/supports-hyperlinks/blob/a2cc083668eb47c046f32ae0bc89d5bbdb8398ef/src/lib.rs#L23-L28

epage · 2024-12-09T18:19:30Z

src/cargo/util/progress.rs

+        let enabled = gctx.progress_config().taskbar.unwrap_or_else(|| {
+            // Windows Terminal session.
+            gctx.get_env("WT_SESSION").is_ok()
+            // Compatibility with ConEmu's ANSI support.
+                || gctx.get_env("ConEmuANSI").ok() == Some("ON".into())
+        });


Can we break the detection out into a function like we do for hyperlinks?

(and this would be allowed to use std::env::var_os, like hyperlinks)

Ideally I see us breaking this out into a crate as some point

I'll see what I can do.

Ideally I see us breaking this out into a crate as some point

After Lennart's push I think that it may be beneficial for the community to have a crate exactly for this one purpose: detecting supported terminal and serializing progress value and status into escape sequences.

I've added cargo::core::shell::supports_progress_report() -> bool, cargo::core::shell::Stream::progress_report and cargo::core::shell::Shell::progress_report() -> bool. Hope this wording is clear:

let enabled = gctx .progress_config() .taskbar .unwrap_or_else(|| gctx.shell().progress_report());

epage · 2024-12-16T18:20:28Z

src/cargo/core/shell.rs

@@ -565,6 +576,14 @@ fn supports_unicode(stream: &dyn IsTerminal) -> bool {
    !stream.is_terminal() || supports_unicode::supports_unicode()
 }

+#[allow(clippy::disallowed_methods)] // ALLOWED: to read terminal app signature
+fn supports_progress_report() -> bool {


Could we rename this (and the variables) to from progress_report to taskbar_progress?

"progress report" is very generic and sounds like its talking about progress indicators in the terminal.

From @joshtriplett:

Different terminal emulators handle this differently; for instance, some of the Linux terminal emulators show this in the terminal's tab bar, instead. Rather than having a name that's specific to the rendering of a subset of terminals, could we use a name that's a little more generic?

Windows Terminal also displays progress in the tab. And here I also started to think about more generic term, because it's really more like a progress and status messaging to terminal.

I'm fine to change it back to taskbar_progress because naming is hard anyway.

epage · 2024-12-16T18:27:52Z

@rfcbot fcp merge

In terms of stability, this adds a new config field

[term]
progress.taskbar = true # whether cargo reports progress to terminal emulator

Like with hyperlinks and unicode, it is auto-detect by default and can be explicitly turned on/off.

For rendering, this ANSI escape code is supported by at least

ConEmu
Windows Terminal
GNOME vte / Ptyxis (not added to cargo's auto-detection yet)

For emitting, this ANSI escape code is supported by at least

systemd

In the current implementation, if the user hits ctrl-c, then nothing will remove the taskbar status. We could add a ctrl-c handler but I'm always hesitant about those and would prefer to see if we can get away without it. It also sounds like other existing programs don't add a ctrl-c handler for this.

rfcbot · 2024-12-16T18:27:54Z

Team member @epage has proposed to merge this. The next step is review by the rest of the tagged team members:

Concerns:

config-name (Taskbar progress reporting via ANSI codes #14615 (comment))

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

src/cargo/core/shell.rs

ehuss · 2024-12-17T15:16:06Z

src/doc/src/reference/config.md

+#### `term.progress.taskbar`
+* Type: bool
+* Default: auto-detect
+* Environment: `CARGO_TERM_PROGRESS_TASKBAR`


Just a mild note, this environment variable doesn't work by itself (because it requires the other progress settings).

BTW, you might be able to fix this by adding #[serde(default)] to the ProgressConfig::when field.

Great point! It's definitely more ergonomic to use now.

This breaks test bad_progress_config_missing_when

Unclear why does ProgressWhen has a Default implementation then...

As for me it feels that any progress setting should imply when = 'auto'. Not sure should 4 years old behavior be changed here.

#8165

I don't recall any particular decisions made regarding bad_progress_config_missing_when. I personally would not object to changing that to allow it to pass. I can't think of a reason it would be required to specify when with width.

joshtriplett · 2024-12-17T15:40:14Z

src/doc/src/reference/config.md

+unicode = true          # whether cargo can render output using non-ASCII unicode characters
+progress.when = 'auto'  # whether cargo shows progress bar
+progress.width = 80     # width of progress bar
+progress.taskbar = true # whether cargo reports progress to terminal emulator


Different terminal emulators handle this differently; for instance, some of the Linux terminal emulators show this in the terminal's tab bar, instead. Rather than having a name that's specific to the rendering of a subset of terminals, could we use a name that's a little more generic?

Perhaps progress.ansi or progress.osc? Or, if we want something less technical, then perhaps progress.escapes, indicating that it sends escape codes?

Here are my points:

progress.taskbar - I like this one for easy interchangeability with taskbar_progress-named fields and functions

progress.osc or progress.osc9_4 - people name PRs and issues mostly with something like "OSC 9;4" so it's somewhat familiar

progress.notification - just high-level

https://gitlab.gnome.org/GNOME/vte/-/issues/2845#note_2299617

For me, my care abouts are

Easy discover for people wanting to change this behavior

Clear meaning when looking at a config or config docs

Consistency with wider community

I feel like ansi, osc, escapes, and osc9_4 do neither unless you are someone already in the nitty gritty implementation details. Also, our other OSC configs do not speak this way (color, unicode, hyperlinks, though color can also support wincon APIs).

progress.taskbar - I like this one for easy interchangeability with taskbar_progress-named fields and functions

That's circular. :)

@epage 👍 to those premises. In addition to those, I'd like to avoid ending up with a name that assumes specifics of how this gets displayed. (This is somewhat similar to https://www.w3.org/QA/Tips/noClickHere .)

Some more possibilities:

term.progress.notification (suggested by @Gordon01)

term.progress.terminal or term.progress.term (slightly odd to repeat but they do have distinct meanings)

term.progress.terminal-ui

term.progress.ui

term.progress.gui

term.progress.report (suggested by @weihanglo)

term.progress.terminal-reporting

Happy to see any other possibility that doesn't specifically assume a UI display mechanism (e.g. not "title" or "taskbar" or "tab-title" or similar).

I don't think there is a clear name to explain itself. notification sounds too far away. osc9_4 is too tied to implementation details. Maybe we'll have other reporting mechanism for some specific terminal emulator in the future (I hope not though).

Since what the feature does is reporting the progress. So, perhaps a verb progress.report is a bit clearer?

(Hey report is also a noun 😅)

notification can also sound like the kitty feature that slightly overlaps om the codes.

Another name I thought of was progress.system to convey that its reporting through the system that calls into cargo.

Eventually I want to add support for multi-line progress which might need its own config and then if we ever get a watch, I would like it to look like bacon. ui and gui might get confusing in those respects (well, more tui).

rfcbot · 2024-12-17T20:59:14Z

🔔 This is now entering its final comment period, as per the review above. 🔔

epage · 2024-12-17T22:07:40Z

src/cargo/core/shell.rs

+                progress_report,
+                stderr_tty,
+                ..
+            } => *progress_report && *stderr_tty,


Generally this check is done in the supports functions, should we make that consistent?

weihanglo · 2024-12-18T01:45:09Z

src/doc/src/reference/config.md

+* Default: auto-detect
+* Environment: `CARGO_TERM_PROGRESS_TASKBAR`
+
+Report progess to the teminal emulator for display in places like the task bar.


Suggested change

Report progess to the teminal emulator for display in places like the task bar.

Report progress to the teminal emulator for display in places like the task bar.

weihanglo · 2024-12-18T01:51:51Z

src/doc/src/reference/config.md

+unicode = true          # whether cargo can render output using non-ASCII unicode characters
+progress.when = 'auto'  # whether cargo shows progress bar
+progress.width = 80     # width of progress bar
+progress.taskbar = true # whether cargo reports progress to terminal emulator


I don't think there is a clear name to explain itself. notification sounds too far away. osc9_4 is too tied to implementation details. Maybe we'll have other reporting mechanism for some specific terminal emulator in the future (I hope not though).

Since what the feature does is reporting the progress. So, perhaps a verb progress.report is a bit clearer?

(Hey report is also a noun 😅)

joshtriplett · 2024-12-18T01:59:22Z

@rfcbot reviewed
@rfcbot concern config-name

Temporary concern until #14615 (comment) gets resolved, because folks may be about to go on holiday soon and I don't want this to get lost. I am not at this time attempting to actually block the FCP if this isn't changed, and I'm happy to defer to consensus; I just want to make sure that this doesn't accidentally FCP without resolving the discussion and reaching some consensus.

rustbot assigned epage Sep 29, 2024

rustbot added A-configuration Area: cargo config files and env vars A-console-output Area: Terminal output, colors, progress bar, etc. A-documenting-cargo-itself Area: Cargo's documentation S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 29, 2024