zombie processes holding a lock on the browse database

[lsmgeb89]

Type: LanguageService

Describe the bug

• OS and Version: Ubuntu 14.04 LTS
• VS Code Version:

  Version: 1.39.2
  Commit: 6ab598523be7a800d7f3eb4d92d7ab9a66069390
  Date: 2019-10-15T15:33:40.634Z
  Electron: 4.2.10
  Chrome: 69.0.3497.128
  Node.js: 10.11.0
  V8: 6.9.427.31-electron.0
  OS: Linux x64 4.4.0-128-generic
  

• C/C++ Extension Version: Version 0.26.1: October 28, 2019
• Other extensions you installed (and if the issue persists after disabling them):
• A clear and concise description of what the bug is.
  All "Go to Definition" returns "No definition found for ...".
  But if my mouse hovers over that function, it will show correct information related to that definition.
  I have three project repositories cloned from the same source.
  And all config files c_cpp_properties.json are the same.
  But only one has this problem.
  Explicit install Version 0.25.1 fixed the problem and Version 0.26.0 shows the same broken behavior of Version 0.26.1.

To Reproduce

• Here is the c_cpp_properties.json dropped sensitive information:

  {
      "configurations": [
          {
              "name": "Linux",
              "compileCommands": "${workspaceFolder}/compile_commands.json",
              "includePath": [
                  "${workspaceFolder}/**"
              ],
              "compilerPath": "/home/bob/toolchain/xxx/yyy/bin/g++ --sysroot=/home/bob/toolchain/xxx/yyy",
              "cStandard": "c11",
              "cppStandard": "c++14",
              "intelliSenseMode": "gcc-x64"
          }
      ],
      "version": 4
  }

1. Open a C++ source code inside work space
2. Click on 'Go to Definition' on a function name
3. See error 'No definition found for ...'
4. 'Peek Definition', 'Go to Declaration' and 'Peek Declaration' also do NOT work.

Expected behavior

It should jump to the definition of that function.

Screenshots

Additional context

[sean-mcmanus]

We're not able to repro this and haven't heard of any other users hitting this yet so we need more repro info. What does Find All References show? Does doing a Reset IntelliSense Database fix it? Does hovering over a 0 show (int)0?

[lsmgeb89]

@sean-mcmanus
Tonight, when I try to reproduce it again by upgrading 0.25.1 to 0.26.1, the bug disappears.
I am guessing the real problem is zombie cpptools processes.

Because in the morning, I saw this error Unable to add file to database, error = 0x8064000a: /xxx/yyy. And then I found this issue #2296.

I did ps -aux | grep 'cpptools', there are around 12 processes with different versions existed.
I killed all related processes by doing pkill -f cpptools and restarted vscode.
These processes must be left because last night I tried to find a working version by clicking "Manage" in the "EXTENSIONS" tab and then "Install Another Version".

Because vscode is running inside company's working vm and then compiling is also happening in this vm. When compiling is running, almost all CPU and memory are used. And then at that time, cpptools processes may crash or become zombie processes, I guess.

[grigorig]

I can confirm this: IntelliSense stopped working for me, and it was caused by old cpptools processes idling, even after VSCode stopped. Killing those processes and starting up VSCode again fixed it.

[lsmgeb89]

@grigorig I agreed with you.

The thing is cpptools processes should not be zombie or left when switching versions or in any other cases.
By the way, just by switching versions back and forth, you can easily reproduce the zombie cpptools processes.

[sean-mcmanus]

@michelleangela @Colengms Are you able to repro the zombie process when switching versions? It isn't reproing for me (on Windows or Linux) -- I see the processes get killed after VS Code reloads with the new extension version.

[willvousden]

Just to confirm, I've had the same symptoms for a week or so – go to definition, find all references, etc. didn't work, even though hovering the mouse over a symbol correctly showed its declared type.

The cause was the same: lots of old processes (just sleeping, not true zombies). Killing them all and restarting VSCode fixed the problem, but this was not an obvious fix!

$ ps aux | grep cpptools | grep -v grep
wvousden         29560   0.0  0.1  4359576  13796   ??  S     8:00am   0:07.90 /Users/wvousden/.vscode/extensions/ms-vscode.cpptools-0.26.2/bin/Microsoft.VSCode.CPP.Extension.darwin
wvousden         66540   0.0  0.2  4380492  35520   ??  S     4:03pm   1:24.28 /Users/wvousden/.vscode/extensions/ms-vscode.cpptools-0.26.2/bin/Microsoft.VSCode.CPP.Extension.darwin
wvousden         48737   0.0  0.0  4357684    644   ??  S     3:46pm   0:12.37 /Users/wvousden/.vscode/extensions/ms-vscode.cpptools-0.26.2/bin/Microsoft.VSCode.CPP.Extension.darwin
wvousden         23811   0.0  0.0  4357028    656   ??  S     3:25pm   0:09.52 /Users/wvousden/.vscode/extensions/ms-vscode.cpptools-0.26.2/bin/Microsoft.VSCode.CPP.Extension.darwin
wvousden         71210   0.0  0.0  4357852    596   ??  S     2:39pm   0:11.24 /Users/wvousden/.vscode/extensions/ms-vscode.cpptools-0.26.2/bin/Microsoft.VSCode.CPP.Extension.darwin
wvousden          9231   0.0  0.0  4358552    640   ??  S    11:47am   0:17.48 /Users/wvousden/.vscode/extensions/ms-vscode.cpptools-0.26.2/bin/Microsoft.VSCode.CPP.Extension.darwin
wvousden         92227   0.0  0.0  4359984    480   ??  S    Tue10am   0:45.63 /Users/wvousden/.vscode/extensions/ms-vscode.cpptools-0.26.2/bin/Microsoft.VSCode.CPP.Extension.darwin
wvousden         76048   0.0  0.0  4354880    480   ??  S    Tue08am   0:29.12 /Users/wvousden/.vscode/extensions/ms-vscode.cpptools-0.26.2/bin/Microsoft.VSCode.CPP.Extension.darwin
wvousden         67455   0.0  0.0  4371412    540   ??  S    Tue07am   1:55.98 /Users/wvousden/.vscode/extensions/ms-vscode.cpptools-0.26.2/bin/Microsoft.VSCode.CPP.Extension.darwin
wvousden         66729   0.0  0.0  4356752    448   ??  S    Tue07am   0:31.33 /Users/wvousden/.vscode/extensions/ms-vscode.cpptools-0.26.2/bin/Microsoft.VSCode.CPP.Extension.darwin
wvousden         66586   0.0  0.0  4356888    536   ??  S    Tue07am   0:32.28 /Users/wvousden/.vscode/extensions/ms-vscode.cpptools-0.26.2-insiders3/bin/Microsoft.VSCode.CPP.Extension.darwin
wvousden         63173   0.0  0.0  4364616    564   ??  S    Mon04pm   0:39.36 /Users/wvousden/.vscode/extensions/ms-vscode.cpptools-0.26.2-insiders3/bin/Microsoft.VSCode.CPP.Extension.darwin
wvousden         59062   0.0  0.0  4361952    408   ??  S    Mon01pm   0:47.64 /Users/wvousden/.vscode/extensions/ms-vscode.cpptools-0.26.2-insiders3/bin/Microsoft.VSCode.CPP.Extension.darwin
wvousden         53641   0.0  0.0  8611012    100   ??  S    Mon11am   0:45.83 /Users/wvousden/.vscode/extensions/ms-vscode.cpptools-0.26.2-insiders3/bin/Microsoft.VSCode.CPP.IntelliSense.Msvc.darwin 53081 3
wvousden         53081   0.0  0.0  4362472    448   ??  S    Mon11am   0:53.01 /Users/wvousden/.vscode/extensions/ms-vscode.cpptools-0.26.2-insiders3/bin/Microsoft.VSCode.CPP.Extension.darwin
wvousden         15069   0.0  0.0  8735960     96   ??  S     6Nov19   2:46.05 /Users/wvousden/.vscode/extensions/ms-vscode.cpptools-0.26.1/bin/Microsoft.VSCode.CPP.IntelliSense.Msvc.darwin 7862 27
wvousden         10639   0.0  0.0  8572340     96   ??  S     6Nov19   0:46.46 /Users/wvousden/.vscode/extensions/ms-vscode.cpptools-0.26.1/bin/Microsoft.VSCode.CPP.IntelliSense.Msvc.darwin 7862 9
wvousden          7862   0.0  0.0  4403960    464   ??  S     6Nov19   6:10.52 /Users/wvousden/.vscode/extensions/ms-vscode.cpptools-0.26.1/bin/Microsoft.VSCode.CPP.Extension.darwin

[sean-mcmanus]

@willvousden Yeah, the dangling process is "locking" the database, causing all the database-based operations to fail from new processes. This sounds like something we should investigate/fix (either detecting/killing the old processes or detecting the database failures and falling back to a different/new one).

[Colengms]

I'm not able to repo an issue with 1.1.3 that results in the process lingering in such a way that prevents other processes from accessing the tag parser database

For example, if I use both VS Code and VS Code Insiders, I can open the same folder from each. When the second one attempts to reference the tag parser database and fails, it will initialize a new database with ".2" inserted into the same. Unfortunately, this means that it's necessary to rebuild the database for that instance, so IntelliSense operations may not provide complete results until the new database is fully populated. This is by design, to enable 2 active uses of the same folder.

We are launching cpptools using a VS Code API (LanguageClient). While we have some options to manage launching/killing cpptools-srv.exe, the cpptools.exe process is managed by VS Code. As explained here, if VS Code is shut down normally, it will wait 5 seconds for a language server process to terminate before forcefully killing it. If this process is lingering after a normal shutdown of VS Code, that would seem to be a VS Code issue. If you are seeing a 'zombie process', it should not be using any memory or holding any files open, as zombie processes do not keep memory or open file handles active.

@lsmgeb89 Are you still able to repro an issue related to this? If so, I think we need to investigate why that process is lingering, in order to compose an issue against VS Code failing to kill it.

[lsmgeb89]

@Colengms I can't reproduce it since I am keeping vscode updated. I am not staying on the old version.

[Colengms]

Thanks @lsmgeb89 . Since you aren't repro'ing with recent versions, I'm going to go ahead and close this issue. However, if anyone has a repro for this issue, please let me know. We can re-open, or we can use this dupe.