Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Cli after "Run Test" with on-toggled "Show Browser" does not start with --headed parameter #19881

Closed
joma74 opened this issue Jan 4, 2023 · 6 comments
Assignees
Labels

Comments

@joma74
Copy link

joma74 commented Jan 4, 2023

Context:

  • Playwright Version: 1.29.1
  • Operating System: Ubuntu 16
  • Node.js version: 16.14.2
  • Visual Studio Code version: 1.74.2
  • Playwright for VSCode extension version: 1.0.3
  • Browser: All
  • Extra: No

Retrace

  • Open Test Explorer
  • Toggle "Show Browser" on
  • Write test that assumes the headless parameter to the args of the test function is false
  • "Run Test"
  • See the test fail

Describe the bug

Playwright Cli after "Start Test" with on-toggled "Show Browser" does not start with --headed parameter.

Here is my Playwright Cli process call:

joma 24808 19258 9 22:41 ? 00:00:00 /home/joma/.nvm/versions/node/v16.14.2/bin/node /home/joma/entwicklung/nodews/tallpad-headlessui-sidebar-vite/node_modules/playwright-core/lib/cli/cli test -c playwright.config.ts --project=Desktop Firefox /home/joma/entwicklung/nodews/tallpad-headlessui-sidebar-vite/tests/e2e/basic\.spec\.ts:12 --repeat-each 1 --retries 0 --workers 1

It shows no --headed parameter.

Hence the headless parameter to the args of the test function is set to true. On the contrary, any browser is started headed, because of the on-toggled "Show Browser".

This issue bothers me as screenshots of at least different sizes(t.i. heights) are created in headed mode vs headless mode(poss. related #12683). Have to maintain two screenshots for each mode. Therefore, on image comparison, i have to take theheadless parameter from the args to the test function into account, which fails for matching the headless golden snapshot image with the headfull browser snapshot image. Which gives following Test Report snippet

...
1) [Desktop Firefox] › basic.spec.ts:12:5 › looks like expected screenshot =======================

    Error: Screenshot comparison failed:

      Expected an image 1280px by 3602px, received 1280px by 3630px. 

    Call log:
      - expect.toHaveScreenshot with timeout 5000ms
      -   verifying given screenshot expectation
      - taking page screenshot
      -   disabled all CSS animations
      - Expected an image 1280px by 3602px, received 1280px by 3630px. 
      - waiting 100ms before taking screenshot
      - taking page screenshot
      -   disabled all CSS animations
      - captured a stable screenshot
      - Expected an image 1280px by 3602px, received 1280px by 3630px.
...
@mxschmitt
Copy link
Member

Could you show us how your test looks like? Are you using the built-in fixtures?

@joma74
Copy link
Author

joma74 commented Jan 6, 2023

Yeess, think so 😅 For reproduction

git clone -b microsoft_playwright_gh_issue_19881 git@github.com:joma74/tallpad-headlessui-sidebar-vite.git microsoft_playwright_gh_issue_19881
cd microsoft_playwright_gh_issue_19881
yarn install --frozen-lockfile

then
tallpad-headlessui-sidebar-vite_microsoft_playwright_gh_issue_19881

Shown test is "looks like expected screenshot" in https://github.com/joma74/tallpad-headlessui-sidebar-vite/blob/d6ee4fa409c2f2ae5af600ec1a0ab352d48f9cdd/tests/e2e/basic.spec.ts

As proof of validity only of the test case as such - "same" from command line, in headed mode, works on my workstation as expected

yarn test_e2e-headed

BTW PW project "Desktop Chrome" is configured to launch /usr/bin/google-chrome. Anyway, "stock browser" PW project "Desktop Firefox" behaves the same.

@mxschmitt
Copy link
Member

For Firefox we have it already filed in #4295 seems like we can merge it into it.

@joma74
Copy link
Author

joma74 commented Jan 6, 2023

The issue is not about #4295, or #12683.

It is about the VS Code extension.

Issue is, using the extension:

  • You toogle on "Show Browser"
  • You start "Run test"
  • You observe that in the test the headless parameter is set to true. But it should be false. As Browsers are shown, hence not headless.

Fix proposal: In the given situation, the VS Code Extension has to start it's per "Run test" initated PW cli call with the --headed parameter.

@mxschmitt
Copy link
Member

Oh you are right, thats a bad bug!

@mxschmitt
Copy link
Member

Looks like its already fixed by microsoft/playwright-vscode#293.

If you want you can follow the instructions on how to use a beta version of the vscode plugin or wait until we released an official version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants