webgl regression: may not render as typing #4665

Tyriar · 2023-08-11T15:59:14Z

VS Code issue: microsoft/vscode#190195

I think this started happened in the last few weeks. Will need to review recent changes/optimizations to webgl

tisilent · 2023-08-12T11:24:02Z

Yesterday, I also discovered this issue in the internal version. One of the two devices had this issue, so I didn't pay much attention The display is only displayed after entering a few characters.

tisilent · 2023-08-12T11:37:27Z

On Monday, I'll see if the device can still reproduce. 😆

tisilent · 2023-08-15T14:13:56Z

~~If there is a time interval, this situation will occur, and if you directly term.write, there will be no such exception.~~ 😵‍💫

Tyriar · 2023-08-15T15:24:06Z

@tisilent what do you mean by time interval?

tisilent · 2023-08-16T10:02:07Z

hi @Tyriar.
Not much progress.
The clues discovered now.

I have generated vscode for version 1.81.1 and master, and 1.81.1 is normal.
This situation can also occur when using earlier versions.

20230816_174117.mp4

This is a virtualization device.

jerch · 2023-08-16T10:14:50Z

This situation can also occur when using earlier versions.

Does it go away if you test with an older release, like 5.2.0? From there on it would be good to pin it down to a certain beta build to get a hold of the commits in question. And if much older releases like 5.1.0 still show the glitch, then it might be a good idea to repeat the testing with an older chrome version and/or other webgl capable engine like firefox to rule out engine version issues.

tisilent · 2023-08-16T10:41:57Z

@jerch I tried two combinations,

xterm 5.2.1 , webgl 0.15.0
xterm 5.3.0-beta.7 , webgl 0.16.0-beta.2 (vscode currently in use)
go away.............
vscode 1.81.1 is normal,I can try using chromium 108.

Tyriar · 2023-08-17T16:27:13Z

I can't reproduce this unfortunately

@traceCall

- New 'trace' log level in the API, you should expect very spammy logs when trace is on and filter them down. As opposed to 'debug' which should be more readable. - @traceCall decorator logs the args and return value of the decorated method. Wecan use this on key functions in various parts of the code to assist debugging issues that are hard to reproduce such as xtermjs#4665. - Decorated a few webgl render methods for xtermjs#4665. - There's an early exit in @traceCall when not on the trace log level so the runtime impact when not in trace is negligable. Part of xtermjs#4665

- New 'trace' log level in the API, you should expect very spammy logs when trace is on and filter them down. As opposed to 'debug' which should be more readable. - traceCall decorator logs the args and return value of the decorated method. Wecan use this on key functions in various parts of the code to assist debugging issues that are hard to reproduce such as xtermjs#4665. - Decorated a few webgl render methods for xtermjs#4665. - There's an early exit in traceCall when not on the trace log level so the runtime impact when not in trace is negligable. Part of xtermjs#4665

tisilent · 2023-08-24T10:03:06Z

https://github.com/xtermjs/xterm.js/assets/48614781/3218fdf8-8ec3-4c33-afe4-10fc1e57188c
left:
electron 22.3.18, chromium 108
right:
electron 26.1.0, chromium 116

tisilent · 2023-08-24T10:11:30Z

I don't know how to track it at the moment. 😭

jerch · 2023-08-24T10:19:56Z

Well I havent seen the issue once here, but I am still on chromium 112 with my tests (linux here). Which makes me wonder, if the recent WebGPU additions in chromium might have to do with this (maybe the drawing/update cycling got affected by those, temporary context loss, things like that...)

Tyriar · 2023-08-24T12:22:41Z

@deepak1556 we're starting to think this webgl issue might be related to the Electron update

Tyriar · 2023-08-24T12:34:03Z

@tisilent since you can reproduce, could you try it on https://insiders.vscode.dev/github/xtermjs/xterm.js in a recent Chrome version? This will help tell if it's a Chromium problem or an Electron problem.

tisilent · 2023-08-24T13:37:30Z

Okay, I'll try it later.

jerch · 2023-08-24T14:07:31Z

Cant repro with chromium 115 either (newest build available here). Could this be a windows only / gc-driver issue?

Below parts of chrome://gpu (currently using intel integrated graphics on ~~Haswell~~ Skylake). Note that it has deactivated WebGPU here due to some "blocklist".

jerch · 2023-08-24T14:24:36Z

Also tested on Chromium 116 (cheated a bit by using custom chrome binary from newest playwright) - cant repro. Also same output from chrome://gpu as above.

If there is a specific pattern to trigger it, I can test it again. What I tested was xterm.js demo (master) and xtermjs.org with switching browser tabs and stealing focus in between. All fine here.

jerch · 2023-08-24T14:32:56Z

Also note that in your demo repro video in #4665 (comment) the cursor on the right side still moves & updates on every input, it is just the char glyphs that come up lazy at once. Hmm.

tisilent · 2023-08-24T14:48:22Z

I saw that fps has fluctuations and is processed, but the characters are not displayed. At first, I checked the code and found that the model output was consistent. This device is Virtualization. emmm.

tisilent · 2023-08-25T02:31:03Z

Verified, older versions of Chromium will not encounter this issue.

jerch · 2023-08-25T11:38:57Z

@tisilent Can you also check, if chrome://gpu --> "Graphics Feature Status" shows significant differences between your tested versions, e.g. WebGPU enabled here but disabled there?

My best guess so far - chromium devs did some fundamental rework on the graphics internals with recent versions, mainly to get WebGPU finally shipped. This may include shifts in handling certain driver/gc constellations (moved from HW to SW support, or vice versa, I think they update their blocklists regularily).

Also saw only windows reports so far, not sure if its platform related as well.

In summary - from xterm.js side there is nothing we can do about that (beside tracking and writing a bug report, if we have enough evidence).

jerch · 2023-08-25T12:11:46Z

This hw acc support issue in browsers is really annoying, it pops up every now and then for us. I think that xterm.js is just a surface vector here, where ppl experience it pretty early / more directly, but in fact is a deeper problem with the engine's hardware support.

At this point I wonder how other popular canvas/webgl driven libraries handle that issue (like d3, three etc), if there are any best practices to collect issues and report upstream.

@tisilent I feel your pain, such a regression is hard to swallow when coming from a flawless working system. If you are really up to find the culprit code change in chromium, I can only suggest to try out the nightly chromium builds one by one until you find the first defective build. From there you can dig deeper into the commits...

deepak1556 · 2023-08-25T12:25:52Z

If you are really up to find the culprit code change in chromium, I can only suggest to try out the nightly chromium builds one by one until you find the first defective build. From there you can dig deeper into the commits...

If there is a reliable repro in Chromium browser, then I recommend to use the following bisect tool https://www.chromium.org/developers/bisect-builds-py/ to narrow down the commit range. It will be very useful when filing upstream bug report.

tisilent · 2023-08-25T12:38:38Z

@jerch There are too many features, some are different..other..

The Graphics Feature Status is the same.

Tyriar · 2023-08-30T15:09:21Z

We looked into this and here's the tracking Chromium bug https://bugs.chromium.org/p/chromium/issues/detail?id=1476475. See microsoft/vscode#191795 for how to workaround this in Electron in the meantime (still needs to be verified once merged).

jerch · 2023-08-30T18:15:34Z

Slightly offtopic (still prolly related to chromium's recent work on the graphics internals) - my sixel-bench test yields these fps numbers (on the same machine/system):

chromium 108 - 3.38s (35.09 MB/s, 202.69 fps)
chromium 116 - 8.97s (13.22 MB/s, 76.34 fps) (WTH?)
epiphany 16.4 Safari/605.1.15 - 3.64s (32.56 MB/s, 188.07 fps)
firefox 117 - 5.83s (20.36 MB/s, 117.58 fps)

Btw chromium started at ~42 MB/s 2ys ago, but the recent drop to 13 MB/s is really hurting. Firefox was always that slow, Webkit is stable at its fast pace.

jerch · 2023-08-30T19:03:55Z

Yes something fishy is going on:

chrome --use_angle - 3.53s (33.64 MB/s, 194.29 fps) (almost back at old values, maybe old default)
chrome --use_angle=egl - 4.68s (25.35 MB/s, 146.41 fps)
chrome --use_angle=gl - 8.67s (13.68 MB/s, 79.0 fps) (prolly the new default?)

Idk the right values for angle, I basically just guessed around, and found an empty value to be the fastest. 🤔

Edit:
And runs really slow with chrome --use-gl=angle --use-angle=swiftshader - now at 25.89s (4.58 MB/s, 26.46 fps). I guess thats mostly software emulation now? Are these settings explained anywhere?

deepak1556 · 2023-08-31T03:20:32Z

Are these settings explained anywhere?

I haven't found a doc for these settings, it was mostly from reading the code https://source.chromium.org/chromium/chromium/src/+/main:ui/gl/init/gl_display_initializer.cc;l=26-170

jerch · 2023-08-31T12:29:58Z

Was finally able to restore the old throughput numbers for the sixel test by disabling vulkan in flags and using the empty --use-angle switch. This penalized a different code path now (png-blob --> bitmap creation), but not by a bothersome amount (only 20% slower).

Its really astonishing how fragile chrome's graphics settings are, while firefox and webkit didn't change fps numbers at all across different updates. Also I dont have to mess with undocumented switches there. (Note that webkit just added alot of stuff in the graphics section like offscreen canvas and bitmap support...)

Tyriar · 2023-09-07T17:03:49Z

Since this is a browser issue I'm closing this off

deepak1556 · 2023-09-19T07:25:08Z

Issue has been addressed in the vulkan backend, refs https://bugs.chromium.org/p/chromium/issues/detail?id=1476475#c14

Tyriar added area/addon/webgl type/bug Something is misbehaving labels Aug 11, 2023

Tyriar added this to the 5.3.0 milestone Aug 11, 2023

Tyriar self-assigned this Aug 11, 2023

Tyriar mentioned this issue Aug 18, 2023

Add trace log level, traceCall decorator and use in webgl #4687

Merged

Tyriar mentioned this issue Aug 23, 2023

Terminal content appears delayed microsoft/vscode#191056

Closed

appleparan mentioned this issue Aug 24, 2023

Text in Terminal does not render properly - V4 jupyterlab/jupyterlab#14687

Closed

appleparan mentioned this issue Aug 30, 2023

terminal freezes with reset or vim jupyterlab/jupyterlab#15043

Closed

Tyriar closed this as completed Sep 7, 2023

Clem-Fern mentioned this issue Sep 20, 2023

When input, characters are not displayed Eugeny/tabby#8884

Closed

ericpaulsen mentioned this issue Oct 5, 2023

web terminal characters are not rendering in v2.1.5+ coder/coder#10082

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

webgl regression: may not render as typing #4665

webgl regression: may not render as typing #4665

Tyriar commented Aug 11, 2023

tisilent commented Aug 12, 2023

tisilent commented Aug 12, 2023

tisilent commented Aug 15, 2023 •

edited

Loading

Tyriar commented Aug 15, 2023

tisilent commented Aug 16, 2023 •

edited

Loading

jerch commented Aug 16, 2023 •

edited

Loading

tisilent commented Aug 16, 2023

Tyriar commented Aug 17, 2023

tisilent commented Aug 24, 2023

tisilent commented Aug 24, 2023

jerch commented Aug 24, 2023 •

edited

Loading

Tyriar commented Aug 24, 2023

Tyriar commented Aug 24, 2023

tisilent commented Aug 24, 2023

jerch commented Aug 24, 2023 •

edited

Loading

jerch commented Aug 24, 2023 •

edited

Loading

jerch commented Aug 24, 2023 •

edited

Loading

tisilent commented Aug 24, 2023

tisilent commented Aug 25, 2023

jerch commented Aug 25, 2023 •

edited

Loading

jerch commented Aug 25, 2023 •

edited

Loading

deepak1556 commented Aug 25, 2023

tisilent commented Aug 25, 2023 •

edited

Loading

Tyriar commented Aug 30, 2023

jerch commented Aug 30, 2023 •

edited

Loading

jerch commented Aug 30, 2023 •

edited

Loading

deepak1556 commented Aug 31, 2023

jerch commented Aug 31, 2023

Tyriar commented Sep 7, 2023

deepak1556 commented Sep 19, 2023

webgl regression: may not render as typing #4665

webgl regression: may not render as typing #4665

Comments

Tyriar commented Aug 11, 2023

tisilent commented Aug 12, 2023

tisilent commented Aug 12, 2023

tisilent commented Aug 15, 2023 • edited Loading

Tyriar commented Aug 15, 2023

tisilent commented Aug 16, 2023 • edited Loading

jerch commented Aug 16, 2023 • edited Loading

tisilent commented Aug 16, 2023

Tyriar commented Aug 17, 2023

tisilent commented Aug 24, 2023

tisilent commented Aug 24, 2023

jerch commented Aug 24, 2023 • edited Loading

Tyriar commented Aug 24, 2023

Tyriar commented Aug 24, 2023

tisilent commented Aug 24, 2023

jerch commented Aug 24, 2023 • edited Loading

jerch commented Aug 24, 2023 • edited Loading

jerch commented Aug 24, 2023 • edited Loading

tisilent commented Aug 24, 2023

tisilent commented Aug 25, 2023

jerch commented Aug 25, 2023 • edited Loading

jerch commented Aug 25, 2023 • edited Loading

deepak1556 commented Aug 25, 2023

tisilent commented Aug 25, 2023 • edited Loading

Tyriar commented Aug 30, 2023

jerch commented Aug 30, 2023 • edited Loading

jerch commented Aug 30, 2023 • edited Loading

deepak1556 commented Aug 31, 2023

jerch commented Aug 31, 2023

Tyriar commented Sep 7, 2023

deepak1556 commented Sep 19, 2023

tisilent commented Aug 15, 2023 •

edited

Loading

tisilent commented Aug 16, 2023 •

edited

Loading

jerch commented Aug 16, 2023 •

edited

Loading

jerch commented Aug 24, 2023 •

edited

Loading

jerch commented Aug 24, 2023 •

edited

Loading

jerch commented Aug 24, 2023 •

edited

Loading

jerch commented Aug 24, 2023 •

edited

Loading

jerch commented Aug 25, 2023 •

edited

Loading

jerch commented Aug 25, 2023 •

edited

Loading

tisilent commented Aug 25, 2023 •

edited

Loading

jerch commented Aug 30, 2023 •

edited

Loading

jerch commented Aug 30, 2023 •

edited

Loading