Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lighthouse emulation for desktop sites is currently broken #1554

Closed
rposbo opened this issue Sep 27, 2021 · 1 comment
Closed

Lighthouse emulation for desktop sites is currently broken #1554

rposbo opened this issue Sep 27, 2021 · 1 comment
Labels
Type: Bug An issue or pull request relating to a bug in WebPageTest

Comments

@rposbo
Copy link
Contributor

rposbo commented Sep 27, 2021

Description

When choosing to get lighthouse results for desktop runs, WPT passes in --emulated-form-factor which is no longer suported in LH version 7; I think this needs to be formFactor, according to the "Changes made in v7" LH notes: https://github.com/GoogleChrome/lighthouse/blob/master/docs/emulation.md#changes-made-in-v7

Steps to reproduce

Choose to run a test for a desktop configuration, tab to "Chromium" and select the lighthouse option.

https://webpagetest.org/result/210927_AiDcEH_0d8a0d38162131dc12a83a9d27971593/

Expected result

WPT test to complete with a link to a LH test for the same configuration (but with emulated mobile connectivity, I believe)

Actual result

The WPT "lighthouse score" box is a question mark, clicking this shows an error page:

Sorry, Lighthouse had some issues gathering your report on WebPageTest. Please try again or try using Lighthouse through another way

Lighthouse test log:

lighthouse "https://news.bbc.co.uk" --channel wpt --enable-error-reporting --max-wait-for-load 80000 --port 9246 --output html --output json --output-path "/home/ubuntu/wptagent/work/wpt_euw3_instance-10.10.1.205/210927_AiDcEH_0d8a0d38162131dc12a83a9d27971593.1.0/lighthouse.json" --throttling-method provided --save-assets --skip-audits screenshot-thumbnails,final-screenshot --emulated-form-factor desktop --chrome-flags="--user-agent='Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/94.0.4606.61 Safari/537.36 PTST/210914.204639'"
Runtime error encountered: Invalid emulation flag. Emulation configuration changed in LH 7.0. See https://github.com/GoogleChrome/lighthouse/blob/master/docs/emulation.md
Error: Invalid emulation flag. Emulation configuration changed in LH 7.0. See https://github.com/GoogleChrome/lighthouse/blob/master/docs/emulation.md
    at assertValidFlags (/usr/lib/node_modules/lighthouse/lighthouse-core/config/config.js:147:11)
    at new Config (/usr/lib/node_modules/lighthouse/lighthouse-core/config/config.js:193:7)
    at generateConfig (/usr/lib/node_modules/lighthouse/lighthouse-core/index.js:65:10)
    at lighthouse (/usr/lib/node_modules/lighthouse/lighthouse-core/index.js:43:18)
    at runLighthouse (file:///usr/lib/node_modules/lighthouse/lighthouse-cli/run.js:244:14)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)

Related Issues

GoogleChrome/lighthouse#10910

@rposbo rposbo added the Type: Bug An issue or pull request relating to a bug in WebPageTest label Sep 27, 2021
@tkadlec
Copy link
Contributor

tkadlec commented Sep 27, 2021

Thanks @rposbo for a super helpfully detailed issue! :)

Lighthouse's version check, which we used to determine which params to send, is acting funky post v 8.5.0 (GoogleChrome/lighthouse#13129) which was causing us to fallback to the older params.

I removed the version check altogether in catchpoint/WebPageTest.agent#434 to just use the updated params (Seems fair given that v7 came out last year :) ). Once the test agents update, we should be good to go again.

@tkadlec tkadlec closed this as completed Sep 27, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: Bug An issue or pull request relating to a bug in WebPageTest
Projects
None yet
Development

No branches or pull requests

2 participants