-
Notifications
You must be signed in to change notification settings - Fork 29.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
WebGPU-based renderer for the editor #221145
Comments
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
Update on my end for last week. WIP branch #225413 General
RasterizationTexture atlas
Explorations
|
Hope this become default soon. |
@faheemstepsharp I suspect it's going to be a long road to be the default (6 months, 1 year+?). We did eventually switch the terminal to default to GPU rendering, it'll be really bad if we ship an editor that breaks text rendering though. |
Update for @hediet and myself for last week. WIP PR #225413 ArchitectureWe came up with a better approach for where to stick the implementation. GPU parts are now regular "view parts" instead of being more tightly tied to the view. vscode/src/vs/editor/browser/view.ts Line 161 in bd21f3c
A new vscode/src/vs/editor/browser/view.ts Lines 146 to 148 in bd21f3c
❔ The term "context" is becoming a little overloaded (ViewContext, ViewGpuContext, GPUContext). Maybe there's a better name for Drawing shapesBuilt out the
This object isn't hooked up yet, just the data structure and tests are mostly done. General
Texture atlas
Debugging |
This may be a silly question but how do you draw glyphs on WebGPU? Are you drawing the glyph map with canvas or render the font manually? |
@vincentdchan fonts are rasterized to a texture atlas using a 2d canvas context (mostly on CPU), then the texture atlas pages are uploaded and drawn by the shader where each cell is a texture mapped to 2 triangles. So we're leveraging the browser's font rendering stack and can avoid getting into that. |
Main updates for last week:
|
I was busy experimenting with the new Copilot API this month, will start back up on this effort in November. |
Update for the last 2 weeks: The main goal was to start self-hosting and I've accomplished that as I started using it 100% of the time on Thursday since selection and mouse events are finally working! It's still rough around the edges but it's definitely usable especially starting from the upcoming Insiders release on Monday (EDIT: #233856 and #234103 are pretty annoying in the Monday build). If you'd like to try it out the setting is currently These are the issues currently being tracked:
editor-gpu
Major changes
Performance
Bug fixes / misc
|
Update for the week:
|
@Tyriar, thanks for your work and updates! However, I believe most people won’t fully comprehend the intricacies of these works. If you could also provide a rough estimate of the progress (even a very rough one is sufficient), that would be fantastic! We eagerly anticipate the availability of this feature in the vscode-prerelease. |
@AurevoirXavier it's hard to say exactly, especially if I get pulled away to do other work. Here's a break down of some of the rough milestones/plans which are a good indicator of progress:
If I was to guess I'd say the "Team self-host" happens by this testing period and "Announce preview in release notes" happens in the following month I work on it. Note though that December is when we do our house-keeping iteration, so I guess this would probably continue in January release and be announced in the January release (aka. early February). You can take it for a spin now with |
Last week was a short one due to holidays in the US. Mostly just fixing some of the bigger self hosting problems.
|
I added the config to enable GPU rendering but all lines that should have been rendered on the GPU disappear. Edit: I did get this message, but as a user this was rather unhelpful as there is no reason why it is disabled or suggested way to enable it. Thanks for your work and looking forward to this. Version: 1.96.0-insider |
@MoonE you need to reload after doing changing the setting.
We don't know the reason, this shows when the symbols are available (ie. it's supported by the engine), but Electron/Chromium doesn't have webgpu enabled. You can check GPU status via No more insight into the problem unfortunately, but it's likely related to Linux, especially if you're in a VM.
Will fix this with #235730 |
I did, the text disappears after that.
Not in a VM, so it probably is a Linux issue, seems like my hardware is on a chromium blocklist.
Thanks for taking your time to respond. |
@KaeLL the info in that screenshot is handy, how did you get it? |
Same as MoonE I'm on Fedora 41 with a Nvidia GPU, which I guess it's to be expected?! :/ |
@KaeLL well that's handy 😅 |
We're finally starting to look at implementing a WebGPU-based rendering in monaco, similar to what xterm.js uses. This issue is used to track all the work which is expected to take several months.
Project: VS Code Editor GPU Renderer (view)
Related issues
Here are some historical links that might be useful:
Below copied from https://github.com/microsoft/vscode-internalbacklog/issues/4906
GPU-based rendering
branch: tyriar/gpu_exploration
How GPU rendering works
It works by assembling array buffers which represent commands to run on the GPU, these are filled on the CPU with information like the texture to use (chracter, fg, bg), location, offset, etc. xterm.js for example allocates a cols x rows array buffer that represents the viewport only and updates it on every frame where the viewport changes.
There are 2 types of shaders:
How the prototype works
The WebGPU prototype works by pre-allocating a buffer that represents up to 3000 lines in a file with a maximum column length of 200. The buffers* are lazily filled in based on what's the viewport. Meaning once a line is loaded, it doesn't need to be modified again. I think it updates more aggressively currently than needed due to my lack of knowledge around finding dirty lines in Monaco.
Texture atlas
Glyphs are rendered on the CPU using the browser's canvas 2d context to draw the characters into a texture atlas. The texture atlas can have multiple pages, this is an optimization problem as uploading images is relative expensive. xterm.js creates multiple small texture atlas pages, allocates using a shelf allocator and eventually merged them into larger immutable pages as they're more expensive to upload.
Currently the prototype uses a single large texture atlas page, but it warms it up in idle callbacks for the current font and all theme token colors in the background (using the
TaskQueue
xterm.js util).Memory usage
In the above, each text_data_buffer cell is 12 bytes (3x 32-bit floats), so 3000x200 would be:
This is pretty insignificant for a modern GPU.
* Double buffering is used as the GPU locks array buffers until it's done with it.
Scrolling
The prototype currently scrolls extremely smoothly as at most a viewport worth of data is filled but often no viewport data will change. Then we just need to update the scroll offset so the shadow knows which cells to render.
Input
So far, the above is highly optimized for readonly scrolling. For input/file changes there are a few cases we need to target. We essentially want to get these updates to take as little CPU time as possible, even if that means leaving stale and no-longer referenced data in the fixed buffers.
Adding new lines or deleting lines
This could be supported by uploading a map whose job is to map line numbers with the index in the fixed buffer:
That way we only need to update indexes, not the whole line data.
Inserting characters
Simple O(n) solution is to just update the entire line. We could do tricks to make this faster but it might not be worth the effort if line length is fixed.
Fixed buffers and long lines
My plan for how the characters will be send to the GPU is to have 1 or more fixed width buffers (eg. 80, 200?) with maps that point to indexes dynamically as described in the input section and then another more dynamic buffer which supports lines of arbitrary length. This dynamic buffer will be a little less optimized as it's the edge case when coding. The fixed buffers could also be dynamically allocated based on the file to save some memory.
Other things we could do
┌───┘
. Whether this looks good in monaco is up to the font settings. Letter spacing and line height will always mess with theseTest results
These were done on terminalInstance.ts. Particularly slow frames of the test are showed.
The
tyriar/gpu_exploration
tests disabled all dom rendering (lines, sticky scroll, etc.) to get an idea of how fast things could be without needed to perform layouts on each frame. It's safe to assume that rendering other components would be less than or equal to the time of the most complex component (minimap is similar, but could potentially share data as well).Scroll to top command
M2 Pro Macbook main
M2 Pro Macbook tyriar/gpu_exploration (all dom rendering disabled)
Windows gaming PC main
Windows gaming PC tyriar/gpu_exploration (all dom rendering disabled)
Scrolling with small text on a huge viewport
fontSize 6, zoomLevel -4
M2 Pro Macbook main
M2 Pro Macbook tyriar/gpu_exploration (all dom rendering disabled)
Windows gaming PC main
Windows gaming PC tyriar/gpu_exploration (all dom rendering disabled)
Very long line
Long lines aren't supported in the gpu renderer currently
Shaders run in parallel to microtasks and layout
The sample below from the Windows scroll to top test above demonstrates how the shaders execute in parallel with layout, as opposed to all after layout.
Before:
After:
Harfbuzz shaping engine is used by lots of programs including Chromium to determine various things about text rendering. This might be needed for good RTL/ligature/grapheme rendering.
The text was updated successfully, but these errors were encountered: