Windows build produces empty symbol zip file

[mbacchi]

The zip file should have the brave.breakpad.syms directory as it does in the muon builds. Yet it only contains two files:

$ zipinfo -l brave-v0.54.0-win32-x64-symbols.zip
Archive:  brave-v0.54.0-win32-x64-symbols.zip
Zip file size: 339337 bytes, number of entries: 2
-rw-rw-rw-  2.0 fat  1995657 b-   333530 defN 18-Aug-15 22:07 LICENSES.chromium.html
-rw-rw-rw-  2.0 fat    17501 b-     5575 defN 18-Aug-15 21:54 LICENSE
2 files, 2013158 bytes uncompressed, 339105 bytes compressed:  83.2%

[mbacchi]

The build appears to run the generate_breakpad_symbols.py script very early in the process, as shown here it is step 5832 out of 34025 steps in the build. Output from the build log:

11:37:29 [5832/34025] ACTION //brave/app/win:generate_breakpad_symbols(//build/toolchain/win:win_clang_x64)
11:37:29 Failed to get symbols for C:/jenkins/win/src/out/Release\VkICD_mock_icd.dll.pdb
11:37:29 Failed to get symbols for C:/jenkins/win/src/out/Release\brotli.exe.pdb
11:37:29 Failed to get symbols for C:/jenkins/win/src/out/Release\dump_syms.exe.pdb
11:37:30 [5833/34025] STAMP obj/components/resources/compressed_about_credits.stamp
11:37:30 [5834/34025] STAMP obj/brave/app/win/generate_breakpad_symbols.stamp

I don't know where it should run but in my opinion it needs to be much later in the process to pick up all the pdb files that are intended to be symbolized.

[mbacchi]

This is potentially an issue with dump_syms.exe, I get the following when I run it by hand on any .pdb file:

PS C:\jenkins\win\src\out\Release> .\dump_syms.exe .\brave.exe.pdb
CoCreateInstance CLSID_DiaSource {E6756135-1E65-4D17-8576-610761398C3C} failed (msdia*.dll unregistered?)
Open failed

According to this, we need MSVC 2013 installed: https://github.com/JPNaude/dev_notes/wiki/Using-Google-Breakpad-with-Qt#windows

If you don't have MSVC 2013 available on you system (more specifically msdia120.dll) you will be greeded with the following output when attempting to use dump_syms: CoCreateInstance CLSID_DiaSource failed (msdia*.dll unregistered?)

But this issue documents that it might be a breaking change with VS 2017: csoltenborn/GoogleTestAdapter#124

Pointing to this VS 2017 readme: https://docs.microsoft.com/en-us/visualstudio/extensibility/breaking-changes-2017#change-reduce-registry-impact

It might be something that the dump_syms.exe authors need to fix instead of us?

[mbacchi]

According to this the path needed to be set to find the msdia140.dll file before running dump_syms.exe. After setting the path, the process works as expected producing symbols in the directory:

PS C:\jenkins\win\src\out\Release> $env:PATH = "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\ID
E\Remote Debugger\x64;$env:PATH"
PS C:\jenkins\win\src\out\Release> python ..\..\brave\tools\win\generate_breakpad_symbols.py --jobs=16 --build-dir="C:\j
enkins\win\src\out\Release" --verbose --symbols-dir="C:\jenkins\win\src\out\Release\dist\brave.breakpad.symbols" "C:\jen
kins\win\src\out\Release\"
Generating symbols for C:\jenkins\win\src\out\Release\brave.exe.pdb
Generating symbols for C:\jenkins\win\src\out\Release\brotli.exe.pdb
Generating symbols for C:\jenkins\win\src\out\Release\character_data_generator.exe.pdb
...
PS C:\jenkins\win\src\out\Release> dir .\dist\brave.breakpad.symbols\


    Directory: C:\jenkins\win\src\out\Release\dist\brave.breakpad.symbols


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----        8/22/2018   7:45 PM                brave.exe.pdb
d-----        8/22/2018   7:45 PM                brotli.exe.pdb
d-----        8/22/2018   7:45 PM                character_data_generator.exe.pdb
d-----        8/22/2018   7:45 PM                chrome.exe.pdb
d-----        8/22/2018   7:45 PM                chromedriver.exe.pdb
d-----        8/22/2018   7:45 PM                chrome_watcher.dll.pdb
d-----        8/22/2018   7:45 PM                dump_syms.exe.pdb
d-----        8/22/2018   7:45 PM                elevation_service.exe.pdb
d-----        8/22/2018   7:45 PM                flatc.exe.pdb
d-----        8/22/2018   7:45 PM                genmacro.exe.pdb
d-----        8/22/2018   7:45 PM                genmodule.exe.pdb
d-----        8/22/2018   7:45 PM                genperf.exe.pdb
d-----        8/22/2018   7:45 PM                genstring.exe.pdb
d-----        8/22/2018   7:45 PM                genversion.exe.pdb
d-----        8/22/2018   7:45 PM                libEGL.dll.pdb
d-----        8/22/2018   7:45 PM                libGLESv2.dll.pdb
d-----        8/22/2018   7:45 PM                mini_installer.exe.pdb
d-----        8/22/2018   7:45 PM                mksnapshot.exe.pdb
d-----        8/22/2018   7:45 PM                mojo_core.dll.pdb
d-----        8/22/2018   7:45 PM                notification_helper.exe.pdb
d-----        8/22/2018   7:45 PM                osmesa.dll.pdb
d-----        8/22/2018   7:45 PM                protoc.exe.pdb
d-----        8/22/2018   7:45 PM                protoc_plugin.exe.pdb
d-----        8/22/2018   7:45 PM                re2c.exe.pdb
d-----        8/22/2018   7:45 PM                setup.exe.pdb
d-----        8/22/2018   7:45 PM                top_domain_generator.exe.pdb
d-----        8/22/2018   7:45 PM                torque.exe.pdb
d-----        8/22/2018   7:45 PM                VkICD_mock_icd.dll.pdb
d-----        8/22/2018   7:45 PM                VkLayer_core_validation.dll.pdb
d-----        8/22/2018   7:45 PM                VkLayer_object_tracker.dll.pdb
d-----        8/22/2018   7:45 PM                VkLayer_parameter_validation.dll.pdb
d-----        8/22/2018   7:45 PM                VkLayer_threading.dll.pdb
d-----        8/22/2018   7:45 PM                VkLayer_unique_objects.dll.pdb

I will add this to the path in our build job.

[mbacchi]

Ok, after adding the path above to the build job, it produces a zip file with 3 symbols, while it should contain dozens. It appears its still running early in the build process, as step 5584 of 34028, I believe its so early that there are only a few pdb files to process:

19:24:06 [5584/34028] ACTION //brave/app/win:generate_breakpad_symbols(//build/toolchain/win:win_clang_x64)

The zip only contains 3 symbols:

$ zipinfo -l brave-v0.54.1-win32-x64-symbols.zip
Archive:  brave-v0.54.1-win32-x64-symbols.zip
Zip file size: 636721 bytes, number of entries: 5
-rw-rw-rw-  2.0 fat  1995657 b-   333530 defN 18-Aug-22 23:21 LICENSES.chromium.html
-rw-rw-rw-  2.0 fat    17501 b-     5575 defN 18-Aug-22 23:08 LICENSE
-rw-rw-rw-  2.0 fat   315894 b-    91958 defN 18-Aug-22 23:24 brave.breakpad.syms/brotli.exe.pdb/2978ABBE73A4D5DEEBA0D4EEBCAF6B881/brotli.exe.sym
-rw-rw-rw-  2.0 fat   267100 b-    70563 defN 18-Aug-22 23:24 brave.breakpad.syms/dump_syms.exe.pdb/95A7960365C9D1043BEF3D90336E2AE41/dump_syms.exe.sym
-rw-rw-rw-  2.0 fat   534775 b-   134093 defN 18-Aug-22 23:24 brave.breakpad.syms/VkICD_mock_icd.dll.pdb/465A3A95A031F12B58982BACC64830EA1/VkICD_mock_icd.dll.sym
5 files, 3130927 bytes uncompressed, 635719 bytes compressed:  79.7%

I will continue to work on this to run the symbol generation later in the process.