Support for rendering ANSI-colored text from piped stdin (and use Helix as $PAGER)

[NNBnh]

Although #996 have been merged, support for rendering ANSI-colored text from piped stdin inside fifo buffers is very useful to used as Kitty terminal's scrollback_pager and will enable Helix to be used as $PAGER, $MANPAGER and $GIT_PAGER without the need of a plugin.

Some Kakoune plugins supporting this feature:

• Kakpipe
• kak-ansi

Also for using the main editor as less or bat replacement, Kakoune currently need a dedicated plugin pager.kak. Hope that Helix can somehow implement this be defaults.

[archseer]

This seems a bit orthogonal to a text editor to me. Why not just use a proper pager?

[NNBnh]

Why not just use a proper pager?

It's the same reason why many seaking for way to use Kakoune and Vim as pager:
Pager is just an editor with less feature, way less feature that look and behave badly different, that mean user need to learn and config two different software that do nearly the same jobs: reading document. Having an editor as the pager is way more powerful, consistent and minimal (also, people just love the editor too much and want to use it more).

Btw, Helix has allowed piping from stdin into a buffer, so Helix have somewhat already be usable as pager:

export PAGER="hx"
export MANPAGER="$PAGER"

Although Helix currently wasn't able to handle terminal ansi code which lead to open a buffer with a bunch of ansi-characters-escape (this image from Kakoune is for illustration purpose):

We could add a option in the config.toml file:

ansi = true # Default

When Helix being piped:

• If false, Helix just open a stdin buffer with a bunch of ansi-characters-escape as currently.
• If true, Helix remove all ansi-characters-escape and render stdin exactly the same as it's rendered on the shell.

Here are some more demonstrate (achieved using Kakoune with kak-ansi):

Kakpipe is also be written in rust, we could took/investegate some of its code.
But still "render stdin exactly the same as it's rendered on the shell" seem difficult, I don't expect this feature to come to Helix even in 2022 base on how long it took to have the same kind of plugin for Kakoune:

• Kakoune start in 2012
• 6 year gap...
• kak-ansi start in 2018

If the dev thinks this feature shouldn't be built in to Helix, please label this issue as plugin idea and the ansi = true option will just remove all ansi-characters-escape.

[kbd]

I am also a kitty user. It, cleverly, provides scrollback search by piping scrollback to a pager. With its new shell integration, you can also do things like pipe the last command's output to a pager.

The usual less isn't great for this, for a bunch of reasons I could enumerate.

I currently use this monstrosity with neovim in my kitty config:

scrollback_pager /usr/local/bin/nvim -c "set nonumber nolist showtabline=0 foldcolumn=0 laststatus=0 noshowmode noruler noshowcmd ignorecase smartcase incsearch shortmess+=F" -c "autocmd TermOpen * normal G" -c "map q :qa!<CR>" -c "set clipboard+=unnamedplus" -c "silent write! /tmp/kitty_scrollback_buffer | te echo -n \"$(cat /tmp/kitty_scrollback_buffer)\" && sleep 1000 "

There's a long issues thread for kitty with different people offering up their recipes for how to do this.

It is very powerful to pipe your scrollback into your full editor, have all the incremental search/highlight comforts of your normal editor, be able to copy text out of it from the keyboard, and so on. I think it's within the realm of what a terminal-based text editor can/should do. I'd happily replace the above long nvim command with any other editor's "pager mode" that provides the equivalent.

[ubitux]

On the topic of pager, neovim proposes this in its documentation: MANPAGER=nvim +Man!, which provides color highlights, classic vim bindings, with an extra q binding to quit. Pretty handy.

[cor]

I'd also love to use Helix as my $PAGER!

[bound-variable]

This seems a bit orthogonal to a text editor to me. Why not just use a proper pager?

I'm not interested in using it as a pager. But I would like to output some content of a man page and then edit it (with helix).

[bcspragu]

Another use case for this: code reviews in the terminal. delta generates syntax highlighted diffs (I believe using the same underlying tooling as bat), which look great/substantially like the ones in GitHub, ex:

PRR is a tool for doing GH code reviews in a text file (and I've hacked together a tree-sitter grammar/Helix integration for that). Setting up delta to use Helix as a pager, and then some minor modifications to prr would be a really excellent TUI/CLI code review workflow for GH. Or maybe this is an extremely niche use case 🤷

[pascalkuthe]

We want to build a diffmode into helix which could be used for code review in the future which makes much more sense IMO since it also allow editing.

[bcspragu]

A diffmode sounds great, though the reason I call out delta above is because I don't think one can get that kind of syntax highlighted diff with just a tree-sitter grammar, as it requires (at least when working from a standard diff file) assembling state across disconnected chunks of file.

Are you imagining the diff mode will have it's own system for highlighting diff chunks such that existing tree-sitter grammars work, ala the algorithm described here in delta?

[pascalkuthe]

A diffmode sounds great, though the reason I call out delta above is because I don't think one can get that kind of syntax highlighted diff with just a tree-sitter grammar, as it requires (at least when working from a standard diff file) assembling state across disconnected chunks of file.

Are you imagining the diff mode will have it's own system for highlighting diff chunks such that existing tree-sitter grammars work, ala the algorithm described here in delta?

We have pur own virtual text Infrastrukturen. It's already perfectly possible to rendern multiple documents with different syntax hoghloghting interleaved.

The text rendering in helix is an order of magnitude more complex compared to a simple tool like delta.

[gyreas]

How orthogonal is it to use helix to view "rolling" log files, just like tail -f? Helix log will benefit directly from this since it's integrated into the editor.

[i-am-logger]

personally , i care less about rolling a log... im more into searching, selecting and yanking from it ... which is why id like to see such support

[aemogie]

Just wanted to mention another place where rendering raw ANSI might be useful.

When running :sh (The screenshot runs :sh help help with nushell configured as the shell)

[pascalkuthe]

closing in favor of #2313

[johannesrave]

@pascalkuthe i just read through #2313 trying to understand how it relates to this but couldn't?

[kirawi]

https://matrix.to/#/!zMuVRxoqjyxyjSEBXc:matrix.org/$8yvUoxXuWBnYHoj-172u2Wq5x8CjVJLx_MwFjoFfkDI?via=matrix.org&via=mozilla.org&via=tchncs.de

As soon as you are parsing/showing ansi escape does you are essentialially building a terminal emualtor anyway. So they really are the same thing. You wouldn't need to call a different pager, the builtin terminal could be used as a sort of pager directly

[k4lizen]

Same as the author, I would like to use hx as a pager for kitty, and thus don't need an integrated terminal (since kitty already has window splitting etc). While parsing ANSI codes needs to be done for an integrated terminal, surely its easier than implementing everything in regards to that?

I haven't read the whole thread there, but to me it makes sense that one would want ANSI code processing built in to helix, while an integrated terminal can be a plugin.

( i tried to open the matrix channel but my client (element) doesn't let me (: )

[acuteaangle]

I also do not understand how #2313 is relevant to this issue.

The use case that led me to this issue:
Ideally, I would like to be able to run lynis audit system > report.ansi, then use helix (hx report.ansi) to add comments to the file, documenting which output items are relevant and what actions (if any) need to be taken for each of them.

Workarounds:

• Use vim with the AnsiEsc.vim plugin
  • I do not want to install and manage a vim plugin on every system exclusively for this task
  • I would prefer to use helix rather than vim, when available
• Ignore the escape codes
  • The mangled output can make it very difficult to read in
• Strip the escape codes
  • I would like to both retain a marked-up report.ansi file that can be catted out, and also be able to run ansi2html < report.ansi > report.html to create a coloured html report including my edits.

As for a more general use case, I also use zellij, a terminal multiplexer.
Instead of a ‘copy mode’, like in screen or tmux, zellij has a shortcut to open the scrollback in $EDITOR.
This is intended to be more useful, as you should be able to search, copy, and write (edited) parts out to files, all with your familiar editor.
It would be ideal if this didn’t mangle escape codes, as it is fairly common that I need to page an output, but might also want to edit and/or save it after seeing it.

From vim, I’m used to having my editor be a superset of my pager. I enjoy usingvim’s view symlink, and would love to see more pager functionally as part of helix, including optional ansi support and the buffer behavior described in #11586 (comment).