WebPageTest plugin doesn't work - code 404 #3771

bgruszka · 2023-02-27T12:01:36Z

Have you read the documentation?

Yes, I've read the how to make a reproducable bug guide
Yes, I've read the how to debug my script guide

URL

https://www.onet.pl/

What are you trying to accomplish

I want to run sitespeed with WebPageTest plugin enabled, but unfortunately I have this: {"name":"WPTAPIError","code":404,"message":"Not Found"}

I set proper API key.

What browser did you use?

Other

How to reproduce

docker run --shm-size=1g --rm -v "$(pwd)":/sitespeed.io sitespeedio/sitespeed.io:26.1.0-webpagetest -b chrome --mobile -d 1 -n 1 -c 3g --webpagetest.key <REAL_KEY_HERE> --webpagetest.runs 1 --plugins.add /webpagetest https://www.onet.pl

Relevant log output

Chromium 97.0.4692.20 Ubuntu 20.04
Mozilla Firefox 105.0
[2023-02-27 11:55:50] INFO: Versions OS: linux 5.10.104-linuxkit nodejs: v16.16.0 sitespeed.io: 26.1.0 browsertime: 16.17.0 coach: 7.1.2
[2023-02-27 11:55:50] INFO: Sending url https://www.onet.pl to test on https://www.webpagetest.org
[2023-02-27 11:55:51] INFO: Running tests using Chrome - 1 iteration(s)
[2023-02-27 11:55:52] INFO: Testing url https://www.onet.pl iteration 1
[2023-02-27 11:56:03] ERROR: Could not run test for WebPageTest {"name":"WPTAPIError","code":404,"message":"Not Found"}
    at Object.<anonymous> (/webpagetest/node_modules/webpagetest/lib/helper.js:213:25)
    at Module._compile (node:internal/modules/cjs/loader:1105:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/webpagetest/node_modules/webpagetest/lib/webpagetest.js:14:15)
    at Module._compile (node:internal/modules/cjs/loader:1105:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/webpagetest/analyzer.js:5:21)
    at Module._compile (node:internal/modules/cjs/loader:1105:14)
[2023-02-27 11:56:04] INFO: Take after page complete check screenshot
[2023-02-27 11:56:05] INFO: Take cumulative layout shift screenshot
[2023-02-27 11:56:06] INFO: Take largest contentful paint screenshot
[2023-02-27 11:56:09] INFO: Get visual metrics from the video
[2023-02-27 11:56:15] INFO: https://www.onet.pl 205 requests, TTFB: 288ms, firstPaint: 424ms, firstVisualChange: 500ms, FCP: 424ms, DOMContentLoaded: 1.23s, LCP: 662ms, CLS: 0, TBT: 668ms, Load: 3.00s, speedIndex: 1.10s, visualComplete85: 2.04s, lastVisualChange: 4.50s
[2023-02-27 11:56:15] INFO: The server responded with a 401 status code for https://www.onet.pl/api/okonto/me?fields=userId%2Clogin%2Cav...
[2023-02-27 11:56:18] INFO: HTML stored in /sitespeed.io/sitespeed-result/www.onet.pl/2023-02-27-11-55-50

The text was updated successfully, but these errors were encountered:

Vodrech · 2023-02-27T14:39:47Z

https://github.com/WebPageTest/webpagetest-api/pull/171/files
Seems like they are deprecating sending in the webpagetest key as query parameters and replacing it by sending it in
as a header, I feel this is already merged to the webpagetest as I also get 404, is this an easy fix? @soulgalore sorry for tag, but it has been like this for a while :)

soulgalore · 2023-02-27T16:40:27Z

I can help out, but what has changed and when? When I look in https://github.com/WPO-Foundation/webpagetest/tree/apache it looks like the last change was done three years ago. The https://github.com/WebPageTest/webpagetest-api seems to get changes all the time, but it's not versioned (like what version of the API works against which version of WebPageTest and more important what version works agains the Open Source version?

soulgalore · 2023-02-28T12:43:41Z

So then If I understand correctly they intentionally broke the API (by calling it "deprecated" but removing the functionality)?

Vodrech · 2023-03-01T09:09:27Z

Might have confused you a bit with above message, it's still possible to send in the api key via the query, I manually tested setting up the webpagetest plugin and running it in the docker with sitespeed.io image, tho I needed to use an older version because the documentation on sitespeed didn't work with setting up plugins manually.

The tests get passed into the webpagetest and I can see the result in there, but I think something is trowing in the try block in the analyzer.js, didn't have time to look further, as I don't have best practice with debugging in such an environment.

Vodrech · 2023-03-01T09:15:55Z

docker run --rm -v /Users/<user>/:/sitespeed.io sitespeedio/sitespeed.io:19.4.1 --plugins.add /sitespeed.io/test/plugin-webpagetest/ --webpagetest.location ec2-eu-north-1 --graphite.host <graphite-host> --webpagetest.browser chrome --webpagetest.connectivity 4G --webpagetest.lighthouse --webpagetest.runs 1 --webpagetest.key <api-key> --webpagetest.timeout 10000 --webpagetest.includeRepeatView --webpagetest.device iPhoneX --outputFolder <folder> --graphite.messages run --graphite.messages pageSummary --config /sitespeed.io/test/plugin-webpagetest/config.json /sitespeed.io/test/plugin-webpagetest/pages.txt

config.json:

	"plugins": {
		"remove": ["browsertime", "budget", "coach", "domains", "harstorer", "pagexray", "remove", "thirdparty", "tracestorer"]
	},
	"metrics": {
		"filter": [
		      "webpagetest.pageSummary.data.lighthouse.categories.performance.score",
		      "webpagetest.pageSummary.data.lighthouse.categories.accessibility.score",
		      "webpagetest.pageSummary.data.lighthouse.audits.aria-allowed-attr.score",
		      "webpagetest.pageSummary.data.lighthouse.audits.aria-required-attr.score",
		      "webpagetest.pageSummary.data.lighthouse.audits.aria-required-children.score",
		      "webpagetest.pageSummary.data.lighthouse.audits.aria-required-parent.score",
		      "webpagetest.pageSummary.data.lighthouse.audits.aria-roles.score",
		      "webpagetest.pageSummary.data.lighthouse.audits.aria-valid-attr-value.score",
		      "webpagetest.pageSummary.data.lighthouse.audits.aria-valid-attr.score",
		      "webpagetest.pageSummary.data.lighthouse.audits.errors-in-console.score",
		      "webpagetest.pageSummary.data.lighthouse.audits.errors-in-console.numericValue",
		      "webpagetest.pageSummary.data.median.firstView.chromeUserTiming_CumulativeLayoutShift"
		]
	}

pages.txt:

bgruszka · 2023-03-02T08:51:14Z

@soulgalore @Vodrech Can I help you somehow?

soulgalore · 2023-03-13T10:09:13Z

@bgruszka what version of WebPageTest do you run against? I haven't looked yet what changed, I only looked in the API and I wanted to understand if that change killed the Apache branch or maybe I misunderstood. The webpagetest-api isn't versioned, and I think the problem is that if we upgrade it can work again their main branch (I don't remember what they call the main branch) but it will break the Apache branch integration?

Comeonautomaten · 2023-03-20T13:54:19Z

@soulgalore I have used sitespeed.io.26.1.0. It is over a month since it stopped working now (around Feb 8th, 2023), and hence losing metrics to graphana.

image: "sitespeedio/sitespeed.io:26.1.0-webpagetest"

Google Chrome 106.0.5249.119
Mozilla Firefox 106.0
Microsoft Edge 106.0.1370.52
[2023-02-27 05:45:25] INFO: Versions OS: linux 5.10.161+ nodejs: v16.16.0 sitespeed.io: 26.1.0 browsertime: 16.17.0 coach: 7.1.2

[2023-02-27 05:45:38] ERROR: Could not run test for WebPageTest {"name":"WPTAPIError","code":404,"message":"Not Found"}
    at Object.<anonymous> (/webpagetest/node_modules/webpagetest/lib/helper.js:213:25)
    at Module._compile (node:internal/modules/cjs/loader:1105:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/webpagetest/node_modules/webpagetest/lib/webpagetest.js:14:15)
    at Module._compile (node:internal/modules/cjs/loader:1105:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/webpagetest/analyzer.js:5:21)
    at Module._compile (node:internal/modules/cjs/loader:1105:14)

soulgalore · 2023-03-20T14:00:58Z

@Comeonautomaten what version are you using of WebPageTest? We didn't change anything in the API so I think it broke because of a change in WebPageTest?

Comeonautomaten · 2023-03-20T14:15:05Z

@soulgalore I do not know the exact version of WebPageTest. I used this image: sitespeedio/sitespeed.io:26.1.0-webpagetest as per documentation: https://www.sitespeed.io/documentation/sitespeed.io/webpagetest/#using-webpagetest
This has been working for years now.

And, I see this merge request on WebPageTest: catchpoint/WebPageTest.api-nodejs#171

soulgalore · 2023-03-20T15:44:47Z

If I understand it correctly this is what has happened:

WebPageTest runs un-versionen and without a changelog in the master branch. The WebPageTest version got updated or autoupdated, the API doesn't work anymore since WebPageTest doesn't version handle the API against the WebPageTest server. The version of the WebPageTest API work against the Apache branch (or any of you using the Apache branch?). When that PR will be merged, it will work for WebPageTest API users that do tests against the master branch. Is that correct?

I don't want to update the WebPageTest API so we break compatibility against the Apache-branch?

soulgalore · 2023-03-28T18:47:04Z

Do you run private instances of WebPageTest? I tried to get an up and running using https://github.com/WPO-Foundation/webpagetest/tree/master/docker/local - I can make it run, but couldn't connect to the API, I always get the " Could not run test for WebPageTest {"statusCode":400,"statusText":"Invalid API key. To continue running tests via the WebPageTest API, you'll need to update your current key for the enhanced WebPageTest API. Read more here: https://product.webpagetest.org/api"" error.

Vodrech · 2023-04-18T16:16:01Z

Hello once again, I tried to follow the newly updated docs that you created, it was very beneficial, but somehow it seem like I'm doing something wrong.

I'm having trouble with adding the webpagetest plugin, I could not figure out if this actually was one of the 'default' plugins or not, but I tried both adding it via the cli and via config.json but neither did work, tried with the / slash as well and nothing changed, could you give me some guidance? @soulgalore

Image: sitespeedio/sitespeed.io:27.3.0-webpagetest
Environment: Kubernetes

/start.sh --webpagetest.host <webpage_host> --webpagetest.location <location> --graphite.host <host> --webpagetest.key <secret> --webpagetest.connectivity 4G --webpagetest.lighthouse --webpagetest.mobile true --webpagetest.runs 1 --webpagetest.timeout 10000 --webpagetest.device iPhoneX --outputFolder <folder> --graphite.messages run --graphite.messages pageSummary --config <path>/config.json pages.txt

config.json

{
	"slug": "<slug>",
	"graphite": {
		"host": "<graphite_host>",
		"namespace": "sitespeed_io/webpagetest",
		"addSlugToKey": true
	},
	"plugins": {
		"list": true,
		"remove": ["browsertime"],
		"add": ["webpagetest"]
	},
	"metrics": {
		"filter": [
		<some_filters>
		]
	}
}

Google Chrome 112.0.5615.49 
Mozilla Firefox 112.0
Microsoft Edge 112.0.1722.39 
[2023-04-18 15:48:56] INFO: Versions OS: linux 5.10.162+ nodejs: v18.14.2 sitespeed.io: 27.3.0 browsertime: 17.8.0 coach: 7.1.3
[2023-04-18 15:48:56] INFO: The following plugins are enabled: assets, budget, coach, domains, graphite, harstorer, html, lateststorer, metrics, pagexray, remove, text, thirdparty, tracestorer, webpagetest
Couldn't load plugin webpagetest: Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/usr/src/app/lib/plugins/webpagetest/index.js' imported from /usr/src/app/lib/core/pluginLoader.js
Error [ERR_MODULE_NOT_FOUND]: Cannot find package 'webpagetest' imported from /usr/src/app/lib/core/pluginLoader.js
    at new NodeError (node:internal/errors:399:5)
    at packageResolve (node:internal/modules/esm/resolve:889:9)
    at moduleResolve (node:internal/modules/esm/resolve:938:20)
    at defaultResolve (node:internal/modules/esm/resolve:1153:11)
    at nextResolve (node:internal/modules/esm/loader:163:28)
    at ESMLoader.resolve (node:internal/modules/esm/loader:838:30)
    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:424:18)
    at ESMLoader.import (node:internal/modules/esm/loader:525:22)
    at importModuleDynamically (node:internal/modules/esm/translators:110:35)
    at importModuleDynamicallyCallback (node:internal/process/esm_loader:35:14) {
  code: 'ERR_MODULE_NOT_FOUND'

soulgalore · 2023-04-18T17:22:39Z

Hi @Vodrech hmm I think I missed updating the documentation for the plugin when sitespeed.io 27.0.0 was released. I fixed that now. If you use the prebuilt container the plugin path is /webpagetest/index.js

Comeonautomaten · 2023-05-17T13:56:28Z

Hi,

The issue seems to be fixed.

Google Chrome 112.0.5615.49 
Mozilla Firefox 112.0
Microsoft Edge 112.0.1722.39 
[2023-05-17 13:13:52] INFO: Versions OS: linux 5.10.162+ nodejs: v18.14.2 sitespeed.io: 27.3.0 browsertime: 17.8.0 coach: 7.1.3
[2023-05-17 13:13:52] INFO: The following plugins are enabled: assets, budget, coach, domains, graphite, harstorer, html, lateststorer, metrics, pagexray, remove, text, thirdparty, tracestorer, /webpagetest/index.js
[2023-05-17 13:13:53] INFO: Sending url https://www.xxxxxxx.com/xx to test on https://www.webpagetest.org/

soulgalore · 2023-05-18T11:23:08Z

Hi @Comeonautomaten thanks of sharing, it seems there was a bug then at the WebPageTest side.

bgruszka added the bug label Feb 27, 2023

soulgalore mentioned this issue Apr 25, 2023

Update to 0.7.1 sitespeedio/plugin-webpagetest#13

Open

soulgalore closed this as completed Dec 28, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WebPageTest plugin doesn't work - code 404 #3771

WebPageTest plugin doesn't work - code 404 #3771

bgruszka commented Feb 27, 2023

Vodrech commented Feb 27, 2023

soulgalore commented Feb 27, 2023

soulgalore commented Feb 28, 2023

Vodrech commented Mar 1, 2023

Vodrech commented Mar 1, 2023

bgruszka commented Mar 2, 2023

soulgalore commented Mar 13, 2023

Comeonautomaten commented Mar 20, 2023

soulgalore commented Mar 20, 2023

Comeonautomaten commented Mar 20, 2023 •

edited

Loading

soulgalore commented Mar 20, 2023 •

edited

Loading

soulgalore commented Mar 28, 2023

Vodrech commented Apr 18, 2023

soulgalore commented Apr 18, 2023

Comeonautomaten commented May 17, 2023

soulgalore commented May 18, 2023

WebPageTest plugin doesn't work - code 404 #3771

WebPageTest plugin doesn't work - code 404 #3771

Comments

bgruszka commented Feb 27, 2023

Have you read the documentation?

URL

What are you trying to accomplish

What browser did you use?

How to reproduce

Relevant log output

Vodrech commented Feb 27, 2023

soulgalore commented Feb 27, 2023

soulgalore commented Feb 28, 2023

Vodrech commented Mar 1, 2023

Vodrech commented Mar 1, 2023

bgruszka commented Mar 2, 2023

soulgalore commented Mar 13, 2023

Comeonautomaten commented Mar 20, 2023

soulgalore commented Mar 20, 2023

Comeonautomaten commented Mar 20, 2023 • edited Loading

soulgalore commented Mar 20, 2023 • edited Loading

soulgalore commented Mar 28, 2023

Vodrech commented Apr 18, 2023

soulgalore commented Apr 18, 2023

Comeonautomaten commented May 17, 2023

soulgalore commented May 18, 2023

Comeonautomaten commented Mar 20, 2023 •

edited

Loading

soulgalore commented Mar 20, 2023 •

edited

Loading