Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Broken shader compilation with mesa update #190437

Closed
RokeJulianLockhart opened this issue Aug 14, 2023 · 53 comments · Fixed by #198142
Closed

Broken shader compilation with mesa update #190437

RokeJulianLockhart opened this issue Aug 14, 2023 · 53 comments · Fixed by #198142
Assignees
Labels
author-verification-requested Issues potentially verifiable by issue author bug Issue identified by VS Code Team member as probable bug chromium Issues and items related to Chromium electron-25-update gpu Issues concerning GPU related items insiders-released Patch has been released in VS Code Insiders linux Issues with VS Code on Linux mitigated Issue has workaround in place upstream-issue-fixed The underlying upstream issue has been fixed verified Verification succeeded
Milestone

Comments

@RokeJulianLockhart
Copy link

RokeJulianLockhart commented Aug 14, 2023

  1. Workaround

    This is an issue in Chromium itself. For now there are the following two workarounds:

    1. Broken shader compilation with mesa update #190437 (comment)

      rm -rf "$HOME/.config/Code/GPUCache"
    2. Broken shader compilation with mesa update #190437 (comment)

      If ($IsLinux) {Remove-Item -Path "$HOME/.config/Code - Insiders/GPUCache"}
  2. Bug

    Does this issue occur when all extensions are disabled?
    I can't test that.
    1. VS Code Version

      Per:

      code-insiders

      ...the affected version is:

      Type Data
      Version 1.82.0-insider
      Commit 76985ae
      Date 2023-08-11T17:10:45.392Z
      Electron 25.4.0
      ElectronBuildId 22958381
      Chromium 114.0.5735.248
      Node.js 18.15.0
      V8 11.4.183.27-electron.0
      OS Linux x64 6.4.9-1-default

      However, per:

      code

      ...note that --channel=latest/stable, that is:

      Type Data
      Version 1.81.1
      Commit 6c3e3db
      Date 2023-08-09T22:18:39.991Z
      Electron 22.3.18
      ElectronBuildId 22689846
      Chromium 108.0.5359.215
      Node.js 16.17.1
      V8 10.8.168.25-electron.0
      OS Linux x64 6.4.9-1-default snap

      ...works:

      image

    2. OS Version

      cat -vbET '/etc/os-release'
           8  CPE_NAME="cpe:/o:opensuse:tumbleweed:20231119"$
      
      uname -a
      PS /home/rokejulianlockhart> uname -a
      Linux RQN6C6 6.4.9-1-default #1 SMP PREEMPT_DYNAMIC Wed Aug  9 05:07:55 UTC 2023 (5b9ad20) x86_64 x86_64 x86_64 GNU/Linux
      PS /home/rokejulianlockhart>
      
  3. Steps to Reproduce

    1. code-insiders
    2. The GUI is invisible:

      image

      ...yet the menubar continues to operate correctly.

@lorand-horvath
Copy link

1.82 updated to Electron 25 and doesn't work on older OS.

@RokeJulianLockhart
Copy link
Author

RokeJulianLockhart commented Aug 15, 2023

#190437 (comment)

@lorand-horvath, I'm using cpe:/o:opensuse:tumbleweed:20230813, which updated 2600 packages yesterday (which I have installed) yet this problem remains. Nobody in their right mind would consider that to be old. Additionally, I am using https://en.opensuse.org/SDB:KDE_repositories#Adding_these_repos_to_an_existing_installation which means that no independent part of the OS should be considered old. This is the most bleeding edge system for miles.

Regardless, should I perhaps open a linked issue at https://github.com/electron/electron/issues/new?assignees=&labels=bug+%3Abeetle%3A&projects=electron%2F90&template=bug_report.yml&title=%5BBug%5D%3A+?

@lorand-horvath
Copy link

lorand-horvath commented Aug 15, 2023

@RokeJulianLockhart Oh, sorry, I thought it's an old OS issue. Before reporting in the electron repo, what is Help > Toggle Developer Tools bringing up?
Have you tried to do a clean reinstall of VSCode Insider? Perhaps it was just a bad build.
Or, alternatively, you could try to download the latest Insider installer and install over your existing broken version (which I assume it's 3-4 days older than what you'd get right now) - that should effectively update it.

@RokeJulianLockhart
Copy link
Author

RokeJulianLockhart commented Aug 15, 2023

@lorand-horvath, #190437 (comment)

Oh, sorry

No problem. Thanks. Apologies for being curt.

what is Help > Toggle Developer Tools bringing up?

image

Have you tried to do a clean reinstall of VSCode Insider? Perhaps it was just a bad build.

# Start-Transcript -Append -IncludeInvocationHeader # Using `script -a -c 'pwsh-preview'` instead.
sudo zypper refresh && `
sudo zypper remove code-insiders && `
sudo zypper install code-insiders && `
code-insiders
PS /home/rokejulianlockhart> sudo zypper refresh && `
>> sudo zypper remove code-insiders && `
>> sudo zypper install code-insiders && `
>> code-insiders
Repository 'KDE:Unstable:Applications' is up to date.                                                                                                                                                                                      
Repository 'KDE:Unstable:Extra' is up to date.                                                                                                                                                                                             
Repository 'KDE:Unstable:Frameworks' is up to date.                                                                                                                                                                                        
Repository 'KDE:Unstable:Qt' is up to date.                                                                                                                                                                                                
Repository 'Copr repo for caprine owned by dusansimic' is up to date.                                                                                                                                                                      
Repository 'dmafanasyev' is up to date.                                                                                                                                                                                                    
Repository 'Main Repository (NON-OSS)' is up to date.                                                                                                                                                                                      
Repository 'Main Repository (DEBUG)' is up to date.                                                                                                                                                                                        
Repository 'Main Repository (Sources)' is up to date.                                                                                                                                                                                      
Repository 'Main Repository (OSS)' is up to date.                                                                                                                                                                                          
Repository 'Main Update Repository' is up to date.                                                                                                                                                                                         
Repository 'google-chrome-unstable' is up to date.                                                                                                                                                                                         
Repository 'Packman Repository' is up to date.                                                                                                                                                                                             
Repository 'libdvdcss repository' is up to date.                                                                                                                                                                                           
Repository 'home:ecsos' is up to date.                                                                                                                                                                                                     
Repository 'openSUSE:Tumbleweed' is up to date.                                                                                                                                                                                            
Repository 'microsoft-edge-dev' is up to date.                                                                                                                                                                                             
Repository 'openSUSE-20230123-0' is up to date.                                                                                                                                                                                            
Repository 'packages-microsoft-com-prod' is up to date.                                                                                                                                                                                    
Repository 'Packman' is up to date.                                                                                                                                                                                                        
Repository 'repo-openh264' is up to date.                                                                                                                                                                                                  
Repository 'snappy' is up to date.                                                                                                                                                                                                         
Repository 'TeamViewer - x86_64' is up to date.                                                                                                                                                                                            
Repository 'vscode' is up to date.                                                                                                                                                                                                         
All repositories have been refreshed.
Reading installed packages...
Resolving package dependencies...

The following package is going to be REMOVED:
  code-insiders

1 package to remove.
After the operation, 363.4 MiB will be freed.
Continue? [y/n/v/...? shows all options] (y): y
(1/1) Removing code-insiders-1.82.0-1691992458.el7.x86_64 ...........................................................................................................................................................................[done]
 
Loading repository data...
Reading installed packages...
Resolving package dependencies...

The following NEW package is going to be installed:
  code-insiders

1 new package to install.
Overall download size: 130.0 MiB. Already cached: 0 B. After the operation, additional 363.5 MiB will be used.
Continue? [y/n/v/...? shows all options] (y): y
Retrieving: code-insiders-1.82.0-1692078853.el7.x86_64 (vscode)                                                                                                                                                        (1/1), 130.0 MiB    
Retrieving: code-insiders-1.82.0-1692078853.el7.x86_64.rpm ..............................................................................................................................................................[done (6.2 MiB/s)]

Checking for file conflicts: ........................................................................................................................................................................................................[done]
(1/1) Installing: code-insiders-1.82.0-1692078853.el7.x86_64 ........................................................................................................................................................................[done]
PS /home/rokejulianlockhart>

image

Appears that it's reproducible.


code-insiders --verbose

code-insiders.txt.log.md

@lorand-horvath
Copy link

Very strange, indeed. I'm out of ideas. Would be interesting to be able to test with a previous Insider version, but for that you would need to have the old package and disable automatic updates.
Electron was updated from 22 to 25 on August 8, so the last version with Electron 22 is the Insider from August 7, which is also the last version that I've tested and still working on Windows 7. Anything after that doesn't start anymore on Win 7. I'm not sure whether the Electron update on August 8 is causing your issue, though...

@RokeJulianLockhart
Copy link
Author

RokeJulianLockhart commented Aug 15, 2023

@lorand-horvath, does the log file explain anything? To me, it seemed a little too verbose to be of much use. I can try other methods to get relevant diagnostic data if you are able to instruct me.

@lorand-horvath
Copy link

lorand-horvath commented Aug 15, 2023

@RokeJulianLockhart I see many GPU-related errors, such as:
Errors: link failed but did not provide an info log [3007:0815/155508.112542:WARNING:angle_platform_impl.cc(48)] ProgramGL.cpp:989 (checkLinkStatus): Program link or binary loading failed with no info log. [3007:0815/155508.112636:ERROR:shared_context_state.cc(81)] Skia shader compilation error

Found these:
https://bugs.chromium.org/p/chromium/issues/detail?id=1442633
ferdium/ferdium-app#1265
https://bbs.archlinux.org/viewtopic.php?id=285507
So it's definitely something related to the new Chromium included in Electron 25 and these issues seem relatively recent.
The first link recommends to disable hardware acceleration, you could try searching how to do that from command line in the new Insider. Unfortunately, I don't have the time to dig deeper right now.

@RokeJulianLockhart

This comment was marked as resolved.

@RokeJulianLockhart
Copy link
Author

RokeJulianLockhart commented Aug 16, 2023

@lorand-horvath, #190437 (comment)

If ($IsLinux) {Remove-Item -Path "$HOME/.config/Code - Insiders/GPUCache"}

remediates the problem. Because indeed this issue is dependent upon https://bugs.chromium.org/p/chromium/issues/detail?id=1442633, before this update is rolled out, there needs to be remediating measures (or the update needs to be temporarily halted) lest it break a great many peoples' installations.

#190437 (comment)

@lorand-horvath
Copy link

lorand-horvath commented Aug 16, 2023

@deepak1556 Can you please take a look at this? It's Chromium/Electron 25 related, so I immediately thought of you. Please see the response just above for a possible solution. Thanks!

@deepak1556 deepak1556 assigned deepak1556 and unassigned sbatten Aug 16, 2023
@deepak1556 deepak1556 added bug Issue identified by VS Code Team member as probable bug gpu Issues concerning GPU related items upstream-issue-linked This is an upstream issue that has been reported upstream chromium Issues and items related to Chromium electron-25-update mitigated Issue has workaround in place labels Aug 16, 2023
@deepak1556
Copy link
Collaborator

Thanks for investigating the issue, given there are workarounds in place this is not necessarily a release blocker. I will wait for chromium to address this issue and backport the fix to Electron 25.

@RokeJulianLockhart
Copy link
Author

RokeJulianLockhart commented Aug 16, 2023

#190437 (comment)

@deepak1556, could a patch at least be added to the affected releases to delete the relevant directory? Surely users shall be affected otherwise? I doubt we can expect them to diagnose a known issue if the solution is so simple.

@deepak1556
Copy link
Collaborator

We don't want to delete the GPUCache for all users and the affected scenario described in https://bugs.chromium.org/p/chromium/issues/detail?id=1442633#c29 is not possible to detect from the application layer. I think the best I can do is to add this issue to the release notes and document the workarounds for visibility.

@RokeJulianLockhart
Copy link
Author

@deepak1556,

We don't want to delete the GPUCache for all users

Why?

@deepak1556
Copy link
Collaborator

It will have impact on startup performance, for now I would wait to see the impact of this issue across our users and decide on a fix forward. The upstream issue has also been open for a while now, signifying its low impact.

@RokeJulianLockhart
Copy link
Author

RokeJulianLockhart commented Sep 8, 2023

Just happened again.

image

rokejulianlockhart@RQN6C6:~> zypper info code-insiders
Loading repository data...
Reading installed packages...


Information for package code-insiders:
--------------------------------------
Repository     : vscode
Name           : code-insiders
Version        : 1.83.0-1694109435.el7
Arch           : x86_64
Vendor         : Microsoft Corporation
Installed Size : 360.6 MiB
Installed      : Yes
Status         : up-to-date
Source package : code-insiders-1.83.0-1694109435.el7.src
Upstream URL   : https://code.visualstudio.com/

rokejulianlockhart@RQN6C6:~>

@lorand-horvath
Copy link

This is mentioned in the 1.82 release notes https://code.visualstudio.com/updates/v1_82#_update-highlights-for-chromium
I guess the issue remains in 1.83-insiders, haven't tried it yet.

@RokeJulianLockhart
Copy link
Author

RokeJulianLockhart commented Sep 8, 2023

It appears weird to me that solely VSCode appears to be affected by this (on my system) because Discord has been very happily updating frequently, and obviously has its own Electron-provided GPUCache directory.

image

@lorand-horvath
Copy link

It appears weird to me that solely VSCode appears to be affected by this (on my system) because Discord has been very happily updating frequently, and obviously has its own Electron-provided GPUCache directory.

They are probably using a different Electron version (that doesn't have this bug).

@jamestiotio
Copy link

This issue has been fixed upstream via this Chromium commit.

It might take a while before it trickles downstream though, since it needs to be picked up by Electron first before it can go into VSCode. I did attempt cherry-picking the commit and generating a local build and it seemed to fix the issue.

@ReillyBrogan
Copy link

The commit is fairly trivial and can probably be safely backported to all supported Electron trees. This is a very widespread issue, affecting virtually every Electron app running on Linux. It would be great to get the fix as widely disseminated as possible considering how slow the app ecosystem is to update versions.

Is there a linked Electron issue for this? Do we need to create one if not or does the Electron project accept drive-by backport PRs?

@jamestiotio
Copy link

jamestiotio commented Nov 2, 2023

Is there a linked Electron issue for this?

I haven't found any linked Electron issues yet.

Do we need to create one if not or does the Electron project accept drive-by backport PRs?

Based on this document, since the patch is temporary, has been committed upstream, and will eventually be removed, I think the Electron folks will accept the patch.

However, do take note that the commit has dependencies on previous commits. On my working local build, I have these 3 commits cherry-picked in chronological order:

Therefore, if anyone is going to bring this up to the Electron team, all 3 patches would need to be merged. Add the patch files to the patches/chromium/ directory and append the filenames of said patch files to the .patches file.

Depending on the Electron build version, there might also be some merge conflicts. Based on my experience, those conflicts should be resolvable, but I did not test all of the older Electron major version release branches. For each branch, we would need to double-check if the 2 dependency commits are already included in their respective Chromium version or not.

@jamestiotio
Copy link

jamestiotio commented Nov 9, 2023

@deepak1556 Thank you for the backporting work to Electron versions 25-28 over the past few days. Downstream applications should now be able to pick those fixes up.

Once VSCode picks up the fix, we should be able to close this issue.

@vscodenpa vscodenpa added the unreleased Patch has not yet been released in VS Code Insiders label Nov 13, 2023
@vscodenpa vscodenpa added this to the November 2023 milestone Nov 13, 2023
@vscodenpa vscodenpa added insiders-released Patch has been released in VS Code Insiders and removed unreleased Patch has not yet been released in VS Code Insiders labels Nov 14, 2023
@deepak1556 deepak1556 added the author-verification-requested Issues potentially verifiable by issue author label Nov 15, 2023
@vscodenpa
Copy link

This bug has been fixed in the latest release of VS Code Insiders!

@RokeJulianLockhart, you can help us out by commenting /verified if things are now working as expected.

If things still don't seem right, please ensure you're on version 54821ee of Insiders (today's or later - you can use Help: About in the command palette to check), and leave a comment letting us know what isn't working as expected.

Happy Coding!

@RokeJulianLockhart
Copy link
Author

#190437 (comment)

@VSCodeTriageBot, I'll verify when the next release reaches me on openSUSE.

@TogoFire
Copy link

@andreamah
Copy link
Contributor

@RokeJulianLockhart by chance, were you able to verify the fix for the issue yet?

@RokeJulianLockhart

This comment was marked as outdated.

@rzhao271
Copy link
Contributor

rzhao271 commented Jan 24, 2024

Considering the issue as verified on 1.85.x. 1.86.0 will go out in around a week.
Feel free to comment on this issue thread if the issue comes up again after that.

@rzhao271 rzhao271 added the verified Verification succeeded label Jan 24, 2024
@github-actions github-actions bot locked and limited conversation to collaborators Jan 27, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
author-verification-requested Issues potentially verifiable by issue author bug Issue identified by VS Code Team member as probable bug chromium Issues and items related to Chromium electron-25-update gpu Issues concerning GPU related items insiders-released Patch has been released in VS Code Insiders linux Issues with VS Code on Linux mitigated Issue has workaround in place upstream-issue-fixed The underlying upstream issue has been fixed verified Verification succeeded
Projects
None yet
Development

Successfully merging a pull request may close this issue.