Render using canvas

[Tyriar]

@parisk @mofux I don't expect the code to change much now, it's mainly just tests, documentation and any bugs that pop up at this point. Please review, I know it's big but most of the changes are in new files inside src/renderer 😄

Resolves #935
Fixes #936
Fixes #681
Fixes #730
Fixes #720
Fixes #475
Fixes #937
Fixes #920
Fixes #919

---

New Stuff!

• Ascii characters with their foreground, 8 color and 8 bright/bold colors are pre-rendered and stored in an ImageBitmap. This is used as a texture atlas to draw to canvas really fast.

  

• Multiple canvases are used to layer different things onto the terminal, this method is fast and allows for a nice separation of logic.

  

• Render layers keep track of what they have drawn (when it makes sense) and as such will diff the new contents and only draw when necessary. This turns single character changes in lines to typically take < 0.25ms, down from ~6ms.

  

• Better support for line height, font size and font family. As they're all managed within Terminal now, just setOption and run the fit addon if necessary.

  

• There is now a very clean separation from the model and everything rendering, pretty mcuh everything that renders is within ./src/renderer (viewport being the only exception, which needs a background color set). We should move towards doing similar with the model and the DOM in Terminal to simplify and make testing easier.

• Link handlers are getting reworked quite a bit as previously the validationCallback exposes the HTMLElement of the link to allow tooltips to be attached. There is now a hoverCallback which achieves a similar goal.

• This fixes a bunch of bugs (see the link of "Fixes #x" above), a notable bug fixes is line characters being misaligned in vtop vtop features misaligned lines #937.

API Changes

See complete list of changes in the xterm.d.ts diff.

Limitations & regressions

• Zooming the browser window will zoom the canvas, meaning at > 100% the text may be a little blurry.
• Blink character attribute support was dropped. This is a little difficult to implement and not really worth the effort given its usage. Implement blink character attribute #944
• Link underlines are no longer the same color as the fg color Link underline should be the same as FG color  #957
• Ligatures are no longer supported Support font ligatures #958

Deferred

• Add new features to demo Add new features to demo #939
• The selection render layer should do incremental draws, currently it clears and redraws the entire selection Do incremental draws in the selection render layer #941
• Window resizes that change devicePixelRatio is expensive (60ms+) Startup and canvas render window resize that updates devicePixelRatio is expensive #955

Benchmarks

This table contains the average time it takes to render when running various actions or programs.

Render time	2.9.2	canvas_render	Improvement
Typing a single letter	4-5ms	0.4-0.5ms	10x faster
vtop idle	22-73ms*	0.8-1.6ms	27-46x faster
ls -lR large_folder	17-25ms*	2.3-4.6ms	5-7x faster
yes	17-30ms*	1-1.3ms	17-23x faster

* Rendering took multiple frames

Setup

• Measured on Chrome 60, Macbook Pro.
• Compared cc7c310 (master, 2.9.2) and Tyriar@2d1a1bf
• The times are all rendering-related chunks exposed in devtools added up (render script, rendering, paint, composite layers).
• I tried to take the smallest and largest samples for the range.

In pictures

Here's one of my favorites; comparing vtop's performance 🔥.

Before

After

---

TODO List

• API: Move setting of theme into Terminal.setOption
• Better emoji support
• Add selection color to ITheme
• Resolve/accept TODO's in code
• Document important classes/methods
• Test
  • Add unit tests
  • Chrome
    • macOS
    • Linux
    • Windows
  • Safari/macOS - Does not support ImageBitmap so falls back to HTMLCanvas
  • Firefox - Buggy and bad performance drawing from ImageBitmap, explicit fallback to HTMLCanvas
    • Linux
    • macOS
    • Windows
  • Edge/Windows - seems just as good as Chrome
  • IE11/Windows - 💣
• Redraw terminal when devicePixelRatio changes (causes blurry text)
• Support floating point devicePixelRatio (causes jagged text)
• Fix right padding getting larger when resizing
• Investigate support for ctrl only clicks in the linkifier (a render layer in an addon?)

[mofux]

We still seem to have a problem with at least the _ character that is only 1px high (whereas I would expect it to be 2px on a retina display, or equal the height of one of the = strokes). See the __proto__ here:

[mofux]

This problem might actually be related to the 15px font size in the demo. Using 14px or any of the font sizes that are supported in Terminal.app gives good looking results. I am wondering if that's the reason why Terminal.app restricts the font-sizes to 9,10,11,12,13,14,18,24,... 🤔

[Tyriar]

I don't think we should include this in this PR but one was we could fix this is by have 2 render layers/canvases for the foreground, one for odd rows, one for even. That would allow characters like _ to overlap cells while still being able to clear them without interfering with other cells.

[mofux]

@Tyriar Support for ligatures seems to be broken (works in v3 and master). I think we should check if there is a way to support it with canvas - I'm afraid there are lots of people that use it (and some like powerline users can hardly live without it).

[Tyriar]

Support for ligatures seems to be broken (works in v3 and master)

I didn't think powerline used ligatures? powerline/fonts#42 Do you have a screenshot of the problem?

[mofux]

@Tyriar Just double-checked, I thought some of the glyphs that are used by powerline are coming from the ligatures, but I was wrong, sorry 😓

[parisk]

🎉

[Krinkle]

Left a logger here I think?

[Tyriar]

Nice catch 😅 #983

[arcanis]

I'm curious, does this renderer supports copy/pasting?

[weihanglo]

Hi @arcanis.
I am not from the Xterm.js team. But after reading the sources, I think the selection of canvas renderer was built almost as same as the custom selection logic shown in #670. There is a separate render layer to make selections work. You can also run the demo and experience its performance directly. All copy/pasting stuff works well in my MacBook.

[Tyriar]

@arcanis copy and paste was barely touched at all, only the drawing of the selection and the events that copy/paste listened to were tweaked.

[weihanglo]

Hi. I am curious about why we chose createImageBitmap over passing canvas out directly. I thought ImageBitmap is for worker thread or image decoding on background thread. Does this gain any performance improvement?

[Tyriar]

There's a comment that explains this:

// ImageBitmap's draw about twice as fast as from a canvas

This is when I initially measured it. I also found that ImageBitmap was cripplingly slow on Firefox and wasn't supported on Safari, these fall back to drawing from HTMLCanvas.

[weihanglo]

Got it.ImageBitmap is for fast accessing and drawing. Thanks for the reply!

[tbodt]

Will selection and copy/paste still work on iOS?

[Tyriar]

@tbodt probably not, I'm thinking we should probably bring back the old renderer anyway for environment edge cases. Some computers take much longer to render using canvas which I think it something to do with GPU rendering being disabled or something.