Semantic colorization doesn't work on multiroot project

[RaavnAndRina]

Type: LanguageService

Describe the bug

• OS: Fedora 31, Windows 7 64x
• VS Code: 1.47.1
• Extension: v0.29.0
• Issue persists with all other extensions disabled and after clean installation

On multiroot project, semantic colorization works only in the first workspace folder (colorization of custom types, c macros, declared functions). However, all other IntelliSense features (hover text, auto-completion, going to declaration/definition) work. Hover text colorization also works correctly. If you try running "Developer: Inspect Editor Tokens and Scopes" command, it will run correctly in the first workspace folder but will fail silently in the second.

Colorization worked correctly on previous stable release.

Steps to reproduce

1. Create a workspace with two root workspace folders
2. Create .c files in each ("Hello world!" will do)
3. Automatically generate c_cpp_properties.json through "Select configuration" for each (may be needed to set default include path for Windows)
4. In VS Code settings, set some specific style for c functions (for your theme, "Quiet Light" in example):

"editor.tokenColorCustomizations": {
        "[Quiet Light]": {
            "functions": "#9e2939",
            "textMateRules": [
                {
                    "scope": "entity.name.function.c",
                    "settings": {
                        "foreground": "#606080"
                    }
                }
            ]
        }

If semantic coloring works, we will never see it (it will be overrided by scope "entity.name.function").
5. Open first .c file. Open second .c file. There is difference in coloring.
6. Try running "Developer: Inspect Editor Tokens and Scopes" on second .c file. It will fail silently.

Logs

1. Logs from the command C/C++: Log Diagnostics

-------- Diagnostics - 7/21/2020, 2:18:19 PM
Version: 0.29.0
Current Configuration:
{
    "name": "Linux",
    "includePath": [
        "${workspaceFolder}/**"
    ],
    "defines": [],
    "compilerPath": "/usr/bin/clang",
    "cppStandard": "c++14",
    "compilerArgs": [],
    "cStandard": "c99",
    "intelliSenseMode": "gcc-x64",
    "browse": {
        "path": [
            "${workspaceFolder}/**"
        ],
        "limitSymbolsToIncludedHeaders": true
    }
}
Translation Unit Mappings:
[ /home/ealeksenko/testtest/test1/main.c ]:
    /home/ealeksenko/testtest/test1/main.c
[ /home/ealeksenko/testtest/test2/main2.c ]:
    /home/ealeksenko/testtest/test2/main2.c
Translation Unit Configurations:
[ /home/ealeksenko/testtest/test1/main.c ]:
    Process ID: 1591325
    Memory Usage: 11 MB
    Compiler Path: /usr/bin/clang
    Includes:
        /usr/local/include
        /usr/lib64/clang/9.0.0/include
        /usr/include
        /home/ealeksenko/testtest/test1
    Standard Version: c99
    IntelliSense Mode: clang-x64
    Other Flags:
        --clang
        --clang_version=90000
[ /home/ealeksenko/testtest/test2/main2.c ]:
    Process ID: 1591343
    Memory Usage: 9 MB
    Compiler Path: /usr/bin/clang
    Includes:
        /usr/local/include
        /usr/lib64/clang/9.0.0/include
        /usr/include
        /home/ealeksenko/testtest/test2
    Standard Version: c99
    IntelliSense Mode: clang-x64
    Other Flags:
        --clang
        --clang_version=90000
Total Memory Usage: 20 MB

2. Logs from the language server

cpptools/activeDocumentChange: file:///home/ealeksenko/testtest/test2/main2.c
cpptools/textEditorSelectionChange
cpptools/resetDatabase
cpptools/resetDatabaseReady
Database safe to open
Code browsing service initialized
Populate include completion cache.
Discovering files...
  Processing folder (recursive): /usr/local/include/
  Processing folder (recursive): /usr/lib64/clang/9.0.0/include/
  Processing folder (recursive): /usr/include/
  Processing folder (recursive): /home/ealeksenko/testtest/test1/
  Processing folder (recursive): /usr/local/include/
  Processing folder (recursive): /usr/lib64/clang/9.0.0/include/
  Processing folder (recursive): /usr/include/
  Processing folder (recursive): /home/ealeksenko/testtest/test2/
  Discovering files: 4810 file(s) processed
  0 file(s) removed from database
Done discovering files.
Parsing remaining files...
  tag parsing file: /home/ealeksenko/testtest/test2/main2.h
  tag parsing file: /home/ealeksenko/testtest/test2/main2.c
  tag parsing file: /home/ealeksenko/testtest/test1/main.h
  tag parsing file: /usr/include/bits/stdio.h
  tag parsing file: /usr/include/c++/9/tr1/stdio.h
  tag parsing file: /usr/include/stdio.h
  tag parsing file: /usr/include/c++/9/tr1/cstdio
  tag parsing file: /usr/include/bits/libc-header-start.h
  tag parsing file: /usr/lib64/clang/9.0.0/include/stddef.h
  tag parsing file: /usr/include/linux/stddef.h
  tag parsing file: /usr/lib64/clang/9.0.0/include/stdarg.h
  tag parsing file: /usr/include/c++/9/tr1/stdarg.h
  tag parsing file: /usr/include/bits/types.h
  tag parsing file: /usr/include/bits/types/__fpos_t.h
  tag parsing file: /usr/include/bits/types/__fpos64_t.h
  tag parsing file: /usr/include/bits/types/__FILE.h
  tag parsing file: /usr/include/bits/types/FILE.h
  tag parsing file: /usr/include/bits/types/struct_FILE.h
  tag parsing file: /usr/include/bits/types/cookie_io_functions_t.h
  tag parsing file: /usr/include/bits/stdio_lim.h
  tag parsing file: /usr/include/bits/sys_errlist.h
  tag parsing file: /usr/include/bits/getopt_posix.h
  tag parsing file: /usr/include/bits/stdio2.h
  tag parsing file: /usr/include/bits/stdio-ldbl.h
  tag parsing file: /usr/include/c++/9/cstdio
  tag parsing file: /usr/include/c++/9/parallel/features.h
  tag parsing file: /usr/include/features.h
  tag parsing file: /usr/lib64/clang/9.0.0/include/__stddef_max_align_t.h
  tag parsing file: /usr/include/c++/9/tr1/cstdarg
  tag parsing file: /usr/include/bits/wordsize.h
  tag parsing file: /usr/include/bits/timesize.h
  tag parsing file: /usr/include/bits/typesizes.h
  tag parsing file: /usr/include/bits/time64.h
  tag parsing file: /usr/include/bits/types/__mbstate_t.h
  tag parsing file: /usr/include/bits/getopt_core.h
  tag parsing file: /usr/include/c++/9/x86_64-redhat-linux/32/bits/c++config.h
  tag parsing file: /usr/include/c++/9/x86_64-redhat-linux/bits/c++config.h
  tag parsing file: /usr/include/stdc-predef.h
  tag parsing file: /usr/include/sys/cdefs.h
  tag parsing file: /usr/include/gnu/stubs.h
  tag parsing file: /usr/include/c++/9/cstdarg
  tag parsing file: /usr/include/c++/9/x86_64-redhat-linux/32/bits/os_defines.h
  tag parsing file: /usr/include/c++/9/x86_64-redhat-linux/bits/os_defines.h
  tag parsing file: /usr/include/c++/9/x86_64-redhat-linux/32/bits/cpu_defines.h
  tag parsing file: /usr/include/c++/9/x86_64-redhat-linux/bits/cpu_defines.h
  tag parsing file: /usr/include/c++/9/pstl/pstl_config.h
  tag parsing file: /usr/include/bits/long-double.h
  tag parsing file: /usr/include/gnu/stubs-32.h
  tag parsing file: /usr/include/gnu/stubs-64.h
  tag parsing file: /home/ealeksenko/testtest/test1/main.c
  Parsing: 50 files(s) processed
Done parsing remaining files.
idle loop: reparsing the active document
Checking for syntax errors: file:///home/ealeksenko/testtest/test2/main2.c
Queueing IntelliSense update for files in translation unit of: /home/ealeksenko/testtest/test2/main2.c
cpptools/finishUpdateSquiggles
Error squiggle count: 0
Update IntelliSense time (sec): 0.252
cpptools/getSemanticTokens: file:///home/ealeksenko/testtest/test1/main.c (id: 45)
cpptools/getSemanticTokens: file:///home/ealeksenko/testtest/test2/main2.c (id: 46)
Database safe to open
cpptools/getSemanticTokens: file:///home/ealeksenko/testtest/test2/main2.c (id: 47)

Last line appeared after running "Developer: Inspect Editor Tokens and Scopes".

Screenshots
File in first workspace folder:

File in second workspace folder:

[Colengms]

Hi @RinaAndRaven . Thanks for reporting this. We indeed have an issue with semantic tokens not being handled properly in multiroot. We've made a fix, which should be available in our very next release or insiders build.

[sean-mcmanus]

@Colengms I'm seeing this not fixed with https://github.com/microsoft/vscode-cpptools/releases/tag/0.30.0-insiders. Same as #5828 . Are you able to repro that?

[sean-mcmanus]

@RinaAndRaven Are you seeing this issue fixed with 0.30.0-insiders?

I'm seeing this being fixed now -- I don't know why I was not seeing the fix yesterday (with the same project and settings). Maybe there's some particular scenario that still has an issue, but I don't know the repro requirements.