Add "scroll caching" functionality, similar to mark jumping in iTerm2

[mp3guy]

Ok I think this is ready for review again. It's more complicated but a lot more robust. I suspect you'll want to put this behind a checkbox setting?

So, from the top: Adds two new keyboard shortcuts (ctrl-up/down) for jumping up/down between previous command executions on the shell.

I had to hook into a terminal's scrollbar value-changed signal to handle tracking of the current "location" when the user scrolls.

To detect "command execution", I had to do some relatively hacky stuff. Firstly, like in iTerm2 (AFAIK the only other terminal emulator that supports this?), I had to add a special hidden char to my shell's PS1 prompt. I opted for the whitespace rendered Soft Hyphen (\u00AD), like this in my .bashrc:

PROMPT=$(echo -e '$\u00AD')
PS1=${PROMPT}

Now, I capture the \r character with the VTE terminal's on-commit signal. In order to function in the scenario where a command spans multiple lines, I search backwards in the terminal text until I hit the top of the window or find the prompt string ($\xad). I guess making this string a customiseable option would be a good idea?

All of these signals go into the ScrollCache class that handles all of the logic. It stores a cache of cursor positions, populated every time a command is executed from the shell prompt via \r.

Special consideration is required for a number of cases:

1. When infinite scrollback is disabled, book keeping needs to be done between the scroll bar's position and the VTE terminal's position.
2. When clear/reset is called, more book keeping is required between the scroll bar and VTE terminal position.
3. Full screen applications like vim mess with the cursor position once again and the cache has to be cleverly managed to avoid weirdness when the application is exited.

Overall from testing I think it's working as expected now. I tested with/without infinite scroll, with mouse wheel scrolling, with clear/reset, with vim, with nvim, with another fullscreen terminal application, with fzf. Some fullscreen terminal applications end up erasing the cache history, but I think that's unavoidable.

Looking forward to feedback, I'll continue dogfooding this myself.

edit: I should note I tried a solution that also uses the hidden prompt character but instead uses the VTE built in search functions to jump up and down. This is super robust to any kind of terminal output/programs, but a bit jankier. It creates a whitespace highlight you can't get rid of easily, doesn't reset properly when you scroll and also doesn't put the line at the top of the terminal. So less fragile but clunkier.

[mattrose]

Wow! I haven't had a chance to look over the new code, but this looks fantastic from the description. Unfortunately, I won't have a chance to review it or work on it before Tuesday, but I'll definitely take a look at it. Putting it behind an experimental checkbox at the start is definitely a good idea.

Are you attached to the Soft-Hyphen marker? I was looking at re-using the OSC 1337 ; SetMark escape sequence that iTerm has documented here https://iterm2.com/documentation-escape-codes.html. It's less likely to get confused with an actual soft-hyphen if somebody decides to output that to the terminal, and it will let us re-use shell code easily from iTerm2, as well, as it is also under a GPL2 license.

Thanks again for tackling this, It's something I've been wanting, and poking at, for a while now.

[mp3guy]

I'm not attached to the soft hyphen, that's just the first thing I found that works. I tried the escape sequences but couldn't find a way to capture them, do you know how?

[mattrose]

If you use the 'contents-changed' signal, that captures all output to the terminal, rather than 'commit', which just captures user input, that should capture all the text output to the terminal. That should detect the OSC sequence embedded in a prompt. This is all theoretical, so I can't say for sure it'll work, but I've had better luck using this signal.

[mp3guy]

That was something I had considered too, but that signal only provides the terminal in the callback and not the actual change itself. So as far as I could tell the only way to read the terminal is with get_text_range which doesn't expose the escaped sequences? Unless you have another example showing this, I'd love to try integrating it.

[mp3guy]

Here's a snippet for bash that patches in the Soft Hyphen in your bashrc

_update_ps1() {
    if [ "${PS1: -1}" = " " ]; then
        PS1=${PS1%?}$(echo -e '\u00ad')
    else
        PS1=${PS1}$(echo -e '\u00ad')
    fi
}
_update_ps1

[mp3guy]

Oh, it looks like vte.get_text_range can actually query the entire terminal history. That means it could be re-architected to just do a linear search through the buffer from the current location whenever you want to jump up or down. This would remove the need for any book keeping and still allow good control over where the jump point is positioned in the terminal. Would solve issues with fullscreen applications and simplify the implementation.

[mp3guy]

Well unsurprisingly this is super slow, especially if the previous marker is very far away. I think what I have now is looking like the best compromise.

[mp3guy]

One option I could add is instead of requiring the user to add the _update_ps1 command to their bashrc, is just using https://lazka.github.io/pgi-docs/Vte-2.91/classes/Terminal.html#Vte.Terminal.feed_child to feed the update of the PS1 variable to the shell right after it gets launched.