bat slow when doing a git lg and using it as the pager compared to less

[trajano]

What version of bat are you using?

bat 0.15.4

Describe the bug you encountered:
When using bat as the core pager git config --global core.pager bat and using it with git lg using git config --global alias.lg=log --color --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset' --abbrev-commit

The performance of git lg on a long history is noticeablely slower compared to that of less. This happens in both CMD and Windows Terminal. Though it is more noticeable in Windows

Describe what you expected to happen?
Performance should be comparable to less, maybe about 10% slower, but you can see it rendering.

How did you install bat?
scoop

[paste the output of info.sh here]
Microsoft Windows [Version 10.0.19041.450]

[sharkdp]

Thank you for reporting this. I can confirm this by running

GIT_PAGER="bat --paging=never" git log --color --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset' --abbrev-commit

in the bat repository. It takes around 6 seconds for ~1500 lines.

What you are asking bat to do here is a bit strange. The git log --color output is colorized already. What do you need bat for?

If you use bat --plain as a pager, it should be much faster.

[trajano]

What you are asking bat to do here is a bit strange. The git log --color output is colorized already. What do you need bat for?

line numbers and coloring in general. Basically just want to replace less because less has a few commands that annoy me like 'logging' or 'saving'

[sharkdp]

I looked into this for a bit. Using perf, we can see that most of the time is spent in write:

Further, we can use strace -c to confirm this:

% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ------------------
 90,10    1,884788         111     16978           write
  9,74    0,203841          10     18672           brk
  0,04    0,000764           6       111           read
  0,03    0,000688          11        59           mmap
  0,02    0,000325          18        18           mprotect
  0,01    0,000294          98         3           munmap
  0,01    0,000252           9        28         2 openat
…

bat makes almost 17k calls to write. The problem is that the input is separated into multiple parts, due to the ANSI codes.

If we use --wrap=never, bat is much faster.

A lot of time is actually spent in the terminal emulator that renders the output. We can see this by running

bat out --wrap=character --terminal-width=80 --style=full --color=always --paging=never

which actually outputs the same content, but does not display it on the screen.

FYI @eth-p

[eth-p]

That's a lot of overhead, wow.

What if we keep a preallocated buffer for printing individual lines? Avoid the syscall and alloc overhead until the very end when the output line is generated.

[sharkdp]

Was thinking about that too. Somehow I would have hoped that a BufWriter would do this for us (see stacktrace)?

[trajano]

What would happen if we just replace write with fwrite? won't that do buffering? So it would be less expensive? The only thing is we would likely need a way of flushing after X milliseconds.

[bl-ue]

Any progress? bat is kind of slow in this regard.

[keith-hall]

Some ideas about the way forward was outlined at #1481 (comment)

[eth-p]

I wanted to revisit this issue and see if things have improved as a result of #2544.

I created a test file by using your command, git log --color --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset' --abbrev-commit and cat 25 times to create a 80,000-line "history" of commits. I compared the both release builds from before and after, and these are the results when neither less nor the terminal emulator is involved:

There's still a lot of room for improvement, but it's something. One area that I'm hoping to work on is reducing the redundant color codes emitted by bat (e.g. prevent bat from printing the default theme text color when it's going to be overridden by a color anyways), which should help both less and the terminal struggle a bit less. It's a bit of an involved change, though, and will require careful planning and design to do properly without regressing performance in other areas.