Fails to run silently in CI

[jooj123]

Not entirely sure if this is a bug - or incorrect usage of pwmetrics.

If i try and run pwmetrics within my CI environment it seems to fireup headless chrome and then silently fails.

Looking for suggestions on how to at least debug this (even if its not a bug).

Here is the output:

19:14:44 + docker run --env-file .env 570761704186.dkr.ecr.ap-southeast-2.amazonaws.com/mytest:PR-50-merge-5 /bin/sh -c yarn run test:perf
19:14:45 yarn run v0.23.2
19:14:45 $ pwmetrics http://www.google.com.au --config=test/performance/pwmetrics-config.js 
19:14:46 Launching Chrome
19:14:51 Done in 6.01s.
..
THEN IT JUST GOES TO THE NEXT STEP IN PIPELINE .. :(

When i run this locally (non headless) i get output like:

Which is what im after.

What I dont get is - if i run lighthouse (outside of pwmetrics) inside the CI environment - it runs headless fine when running a command like:
lighthouse http://www.google.com.au --chrome-flags='--headless'

Config / CLI options

My config file looks like:

module.exports = {
  flags: {
    runs: '1', // number or runs
    submit: false, // turn on submitting to Google Sheets
    upload: false, // turn on uploading to Google Drive
    view: false, // open uploaded traces to Google Drive in DevTools
    expectations: true, // turn on assertation metrics results against provides values
    'chrome-flags': '--headless',
  },
  expectations: {
 
    ttfcp: {
      warn: '>=1500',
      error: '>=0001',
    },
    ttfmp: {
      warn: '>=2000',
      error: '>=3000',
    },
    fv: {
      warn: '>=1500',
      error: '>=2000',
    },
    psi: {
      warn: '>=1500',
      error: '>=2000',
    },
    vc85: {
      warn: '>=1500',
      error: '>=2000',
    },
    vs100: {
      warn: '>=1500',
      error: '>=2000',
    },
    tti: {
      warn: '>=1500',
      error: '>=2000',
    },
  },
};

Steps to reproduce.

Run against headless chrome

Stack trace

None.

Screenshot

Environment

1. pwmetrics version: 2.0.2
2. Chrome version: Latest Stable
3. OS version: Docker image node:7.9.0-slim

Thanks !
ps. this tool is awesome !

[pedro93]

Currently pwmetrics uses a commit-specific version of lighthouse see package.json.

Lighthouse has had in the past some problems runnning in headless mode GoogleChrome/lighthouse#2008, ensure if the differences between our lighthouse version and stable one has an impact on the headless mode.

We should converge with the stable lighthouse version as soon as a new major version is released (expected to be during Google I/O this week).

Also your config file might be wrong, chrome-flags should be without '' I think. Never tested chrome-specific flags in pwmetrics.

[jooj123]

Ok maybe ill just wait till #119 is resolved

If there is anything i can help with let me know ..

thanks

[jooj123]

@pedro93 are we still waiting for #120 to be resolved?
Anything i can do to help ?

[pedro93]

We are waiting for GoogleChrome/lighthouse#2251 to be resolved before we move to lighthouse v2 official non-alpha. There has been frequent daily work on the PR so it shouldn't be long. Maybe see if you can speed it along there?

[denar90]

@jooj123 we've published v3. Can you still reproduce this issue?

[jooj123]

@denar90 ill try to test this out again tommorrow - thanks

[young]

@jooj123 Just ran into the same issue as you. Checked the source and I noticed chrome flags has to pass an array into pwmetrics and the name is camel-cased.

When I changed my options to:

flags: {
    'chromeFlags': ['--headless']
  }

I can get pwmetrics to run in headless Chrome.

[young]

Updated the readme with #126.

[jooj123]

ok thanks @denar90 & @young - can close this one now