Test: Smoke Tests

[Tyriar]

#91278

• Windows @sbatten
• OS X @mjbvz
• Linux @aeschli
• Web (Chromium) @RMacfarlane
• Web (Firefox) @joaomoreno
• Web (macOS Webkit) @chrmarti

Complexity: 2

Run the smoke tests and report issues for any test failures https://github.com/Microsoft/vscode/blob/master/test/smoke/README.md#run. ⚠️ Desktop tests MUST run with --stable-build argument ⚠️

If the automated tests fail, create and issue for that and run the tests manually: https://github.com/microsoft/vscode/wiki/Smoke-Test

[joaomoreno]

@chrmarti I switched my slot with yours since I do not have macOS.

[bpasero]

@sbatten @mjbvz fyi those tests that target --stable will not function for this iteration because the smoke test was improved to detect opened editors more reliably. Unfortunately that required changes in the workbench itself which are not part of the 1.42 release.

So I suggest to not run with --stable for this time and do the following manual test:

• open stable on a user data dir and build up some UI state
• verify this UI state is restored opening that in insiders

Same for dirty files (untitled and normal files).

[bpasero]

@Tyriar I am not sure these instructions make it clear that we must run with --build to test against insiders (both for Electron and Web!) and not out of sources. Not sure if you copied this from some template but then we should update that.

[joaomoreno]

Yeah, Search tests fail on Linux Web Firefox as well... and that triggers everything else to fail, so I'll wait for Search to be fixed and run this again tomorrow. cc @JacksonKearl @roblourens

[Tyriar]

@joaomoreno you could comment out or use .skip on the search test to unblock yourself?

[joaomoreno]

Commenting out search just makes others fail. Anything related to quick open seems not to work at all. More specifically: anything related to input boxes.

I know people have tested web on Firefox (@bpasero ?), so this is maybe Linux specific?

[joaomoreno]

Created #91609