Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core(driver): create eval code using interface #10816

Merged
merged 30 commits into from
Dec 2, 2020
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
90018b2
core(driver): create page code using structured interface
connorjclark May 20, 2020
f7e38f5
rename type
connorjclark May 20, 2020
78f386a
pr feedback
connorjclark May 20, 2020
32856e7
no strings
connorjclark May 20, 2020
e2dd63d
test
connorjclark May 20, 2020
f38bf41
Merge remote-tracking branch 'origin/master' into driver-eval-enhanced
connorjclark Jun 4, 2020
ce9839b
test
connorjclark Jun 4, 2020
4c903de
restrucutre
connorjclark Jun 5, 2020
0413a4a
Merge remote-tracking branch 'origin/master' into driver-eval-enhanced
connorjclark Jun 12, 2020
d4bd2bf
Merge remote-tracking branch 'origin/master' into driver-eval-enhanced
connorjclark Jun 17, 2020
02dc215
remove obj
connorjclark Jun 17, 2020
ce2827a
Merge remote-tracking branch 'origin/master' into driver-eval-enhanced
connorjclark Jul 15, 2020
c81a636
update master
connorjclark Nov 5, 2020
70d8210
rm dead code
connorjclark Nov 5, 2020
df5e8dd
Merge remote-tracking branch 'origin/master' into driver-eval-enhanced
connorjclark Nov 10, 2020
4e59476
adam feedback
connorjclark Nov 10, 2020
147b98c
fix tests
connorjclark Nov 10, 2020
a4051a1
fix mangle issues
connorjclark Nov 10, 2020
77afd88
fix
connorjclark Nov 10, 2020
fd94564
fix nasty types by using tuples
connorjclark Nov 10, 2020
9e7644e
no snapshot
connorjclark Nov 10, 2020
f42ca84
Merge remote-tracking branch 'origin/master' into driver-eval-enhanced
connorjclark Nov 12, 2020
b1fb5d3
fix pr
connorjclark Nov 12, 2020
c4d3a58
oops
connorjclark Nov 12, 2020
ee6f930
require empty args array
connorjclark Nov 13, 2020
6be38eb
inline
connorjclark Nov 13, 2020
97b0525
last bits
connorjclark Nov 13, 2020
81ea5b5
Merge remote-tracking branch 'origin/master' into driver-eval-enhanced
connorjclark Nov 13, 2020
4843291
Merge remote-tracking branch 'origin/master' into driver-eval-enhanced
connorjclark Dec 1, 2020
3c9f573
tests
connorjclark Dec 2, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 23 additions & 1 deletion lighthouse-core/gather/driver.js
Original file line number Diff line number Diff line change
Expand Up @@ -438,16 +438,19 @@ class Driver {
}

/**
* Note: Prefer `evaluate` instead.
* Evaluate an expression in the context of the current page. If useIsolation is true, the expression
* will be evaluated in a content script that has access to the page's DOM but whose JavaScript state
* is completely separate.
* Returns a promise that resolves on the expression's value.
* @template T
connorjclark marked this conversation as resolved.
Show resolved Hide resolved
* @param {string} expression
* @param {{useIsolation?: boolean}=} options
* @return {Promise<*>}
*/
async evaluateAsync(expression, options = {}) {
const contextId = options.useIsolation ? await this._getOrCreateIsolatedContextId() : undefined;
const contextId =
brendankenny marked this conversation as resolved.
Show resolved Hide resolved
options.useIsolation ? await this._getOrCreateIsolatedContextId() : undefined;

try {
// `await` is not redundant here because we want to `catch` the async errors
Expand All @@ -464,6 +467,25 @@ class Driver {
}
}

/**
* Evaluate an expression (optionally defined in a structured manner, see `createEvalCode`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see createEvalCode

this is the interface exposed to gathering. Are there any docs on createEvalCode that could move here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. But we could copy the contents of createEvalCode here?

I'm surprised @see doesn't do anything useful in vscode.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

@brendankenny brendankenny Nov 12, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But we could copy the contents of createEvalCode here?

sorry, that's what I meant :) Even @see is an extra click (and won't appear on hover)

* in `page-functions.js`).
* See `evaluateAsync`.
* @template T, R
* @param {string | ((...args: T[]) => R)} expressionOrMainFn
connorjclark marked this conversation as resolved.
Show resolved Hide resolved
* @param {{useIsolation?: boolean, args?: T[], deps?: Array<Function|string>}=} options
* @return {Promise<R>}
*/
async evaluate(expressionOrMainFn, options = {}) {
if (typeof expressionOrMainFn !== 'string') {
expressionOrMainFn = pageFunctions.createEvalCode(expressionOrMainFn, {
mode: 'iife',
...options,
});
}
return this.evaluateAsync(expressionOrMainFn, options);
}

/**
* Evaluate an expression in the given execution context; an undefined contextId implies the main
* page without isolation.
Expand Down
3 changes: 2 additions & 1 deletion lighthouse-core/gather/fetcher.js
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,8 @@ class Fetcher {
requestInterceptionPromise,
]).finally(() => clearTimeout(timeoutHandle));

const injectionPromise = this.driver.evaluateAsync(`${injectIframe}(${JSON.stringify(url)})`, {
const injectionPromise = this.driver.evaluate(injectIframe, {
args: [url],
useIsolation: true,
});

Expand Down
16 changes: 8 additions & 8 deletions lighthouse-core/gather/gatherers/link-elements.js
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ const LinkHeader = require('http-link-header');
const Gatherer = require('./gatherer.js');
const {URL} = require('../../lib/url-shim.js');
const NetworkAnalyzer = require('../../lib/dependency-graph/simulator/network-analyzer.js');
const {getElementsInDocumentString, getNodeDetailsString} = require('../../lib/page-functions.js');
const pageFunctions = require('../../lib/page-functions.js');

/* globals HTMLLinkElement getNodeDetails */

Expand Down Expand Up @@ -86,13 +86,13 @@ class LinkElements extends Gatherer {
static getLinkElementsInDOM(passContext) {
// We'll use evaluateAsync because the `node.getAttribute` method doesn't actually normalize
// the values like access from JavaScript does.
return passContext.driver.evaluateAsync(`(() => {
${getElementsInDocumentString};
${getLinkElementsInDOM};
${getNodeDetailsString};

return getLinkElementsInDOM();
})()`, {useIsolation: true});
return passContext.driver.evaluate(getLinkElementsInDOM, {
useIsolation: true,
deps: [
pageFunctions.getNodeDetailsString,
pageFunctions.getElementsInDocument,
],
});
}

/**
Expand Down
42 changes: 28 additions & 14 deletions lighthouse-core/gather/gatherers/meta-elements.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,30 @@
'use strict';

const Gatherer = require('./gatherer.js');
const getElementsInDocumentString = require('../../lib/page-functions.js').getElementsInDocumentString; // eslint-disable-line max-len
const pageFunctions = require('../../lib/page-functions.js');

/* globals getElementsInDocument */

/* istanbul ignore next */
function collectMetaElements() {
// @ts-expect-error - getElementsInDocument put into scope via stringification
const metas = /** @type {HTMLMetaElement[]} */ (getElementsInDocument('head meta'));
return metas.map(meta => {
/** @param {string} name */
const getAttribute = name => {
const attr = meta.attributes.getNamedItem(name);
if (!attr) return;
return attr.value;
};
return {
name: meta.name.toLowerCase(),
content: meta.content,
property: getAttribute('property'),
httpEquiv: meta.httpEquiv ? meta.httpEquiv.toLowerCase() : undefined,
charset: getAttribute('charset'),
};
});
}

class MetaElements extends Gatherer {
/**
Expand All @@ -18,19 +41,10 @@ class MetaElements extends Gatherer {

// We'll use evaluateAsync because the `node.getAttribute` method doesn't actually normalize
// the values like access from JavaScript does.
return driver.evaluateAsync(`(() => {
${getElementsInDocumentString};

return getElementsInDocument('head meta').map(meta => {
return {
name: meta.name.toLowerCase(),
content: meta.content,
property: meta.attributes.property ? meta.attributes.property.value : undefined,
httpEquiv: meta.httpEquiv ? meta.httpEquiv.toLowerCase() : undefined,
charset: meta.attributes.charset ? meta.attributes.charset.value : undefined,
};
});
})()`, {useIsolation: true});
return driver.evaluate(collectMetaElements, {
useIsolation: true,
deps: [pageFunctions.getElementsInDocument],
});
}
}

Expand Down
17 changes: 8 additions & 9 deletions lighthouse-core/gather/gatherers/script-elements.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,7 @@
const Gatherer = require('./gatherer.js');
const NetworkAnalyzer = require('../../lib/dependency-graph/simulator/network-analyzer.js');
const NetworkRequest = require('../../lib/network-request.js');
const getElementsInDocumentString = require('../../lib/page-functions.js').getElementsInDocumentString; // eslint-disable-line max-len
const pageFunctions = require('../../lib/page-functions.js');
const {getElementsInDocument, getNodeDetailsString} = require('../../lib/page-functions.js');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we keep this consistent with the style of link-elements?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we keep this consistent with the style of link-elements?

yeah, which is the way that breaks when minified? If we're getting confused we should definitely document this :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, and it is necessary, due to mangling #11463

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, and it is necessary, due to mangling #11463

this is unfortunate, since we lose all deps type checking benefits then :/

(see also #11463 (comment) for an example of the mangling)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea, reminder tho that we could get it back if we namespace the page functions (idea 3 #10781 )


/* global getNodeDetails */

Expand All @@ -19,7 +18,6 @@ const pageFunctions = require('../../lib/page-functions.js');
/* istanbul ignore next */
function collectAllScriptElements() {
/** @type {HTMLScriptElement[]} */
// @ts-expect-error - getElementsInDocument put into scope via stringification
const scripts = getElementsInDocument('script'); // eslint-disable-line no-undef

return scripts.map(script => {
Expand Down Expand Up @@ -72,12 +70,13 @@ class ScriptElements extends Gatherer {
const driver = passContext.driver;
const mainResource = NetworkAnalyzer.findMainDocument(loadData.networkRecords, passContext.url);

/** @type {LH.Artifacts['ScriptElements']} */
const scripts = await driver.evaluateAsync(`(() => {
${getElementsInDocumentString}
${pageFunctions.getNodeDetailsString};
return (${collectAllScriptElements.toString()})();
})()`, {useIsolation: true});
const scripts = await driver.evaluate(collectAllScriptElements, {
useIsolation: true,
deps: [
getNodeDetailsString,
getElementsInDocument,
],
});

for (const script of scripts) {
if (script.content) script.requestId = mainResource.requestId;
Expand Down
48 changes: 25 additions & 23 deletions lighthouse-core/gather/gatherers/seo/tap-targets.js
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ const tapTargetsSelector = TARGET_SELECTORS.join(',');

/**
* @param {HTMLElement} element
* @returns {boolean}
* @return {boolean}
*/
/* istanbul ignore next */
function elementIsVisible(element) {
Expand All @@ -42,7 +42,7 @@ function elementIsVisible(element) {

/**
* @param {Element} element
* @returns {LH.Artifacts.Rect[]}
* @return {LH.Artifacts.Rect[]}
*/
/* istanbul ignore next */
function getClientRects(element) {
Expand All @@ -65,7 +65,7 @@ function getClientRects(element) {
/**
* @param {Element} element
* @param {string} tapTargetsSelector
* @returns {boolean}
* @return {boolean}
*/
/* istanbul ignore next */
function elementHasAncestorTapTarget(element, tapTargetsSelector) {
Expand Down Expand Up @@ -119,7 +119,7 @@ function hasTextNodeSiblingsFormingTextBlock(element) {
* Makes a reasonable guess, but for example gets it wrong if the element is surrounded by other
* HTML elements instead of direct text nodes.
* @param {Element} element
* @returns {boolean}
* @return {boolean}
*/
/* istanbul ignore next */
function elementIsInTextBlock(element) {
Expand Down Expand Up @@ -160,7 +160,7 @@ function elementCenterIsAtZAxisTop(el, elCenterPoint) {
/**
* Finds all position sticky/absolute elements on the page and adds a class
* that disables pointer events on them.
* @returns {() => void} - undo function to re-enable pointer events
* @return {() => void} - undo function to re-enable pointer events
*/
/* istanbul ignore next */
function disableFixedAndStickyElementPointerEvents() {
Expand All @@ -186,7 +186,7 @@ function disableFixedAndStickyElementPointerEvents() {

/**
* @param {string} tapTargetsSelector
* @returns {LH.Artifacts.TapTarget[]}
* @return {LH.Artifacts.TapTarget[]}
*/
/* istanbul ignore next */
function gatherTapTargets(tapTargetsSelector) {
Expand Down Expand Up @@ -285,23 +285,25 @@ class TapTargets extends Gatherer {
* @return {Promise<LH.Artifacts.TapTarget[]>} All visible tap targets with their positions and sizes
*/
afterPass(passContext) {
const expression = `(function() {
${pageFunctions.getElementsInDocumentString};
${disableFixedAndStickyElementPointerEvents.toString()};
${elementIsVisible.toString()};
${elementHasAncestorTapTarget.toString()};
${elementCenterIsAtZAxisTop.toString()}
${getClientRects.toString()};
${hasTextNodeSiblingsFormingTextBlock.toString()};
${elementIsInTextBlock.toString()};
${RectHelpers.getRectCenterPoint.toString()};
${pageFunctions.getNodeDetailsString};
${gatherTapTargets.toString()};

return gatherTapTargets("${tapTargetsSelector}");
})()`;

return passContext.driver.evaluateAsync(expression, {useIsolation: true});
return passContext.driver.evaluate(gatherTapTargets, {
args: [tapTargetsSelector],
useIsolation: true,
deps: [
pageFunctions.getNodeDetailsString,
pageFunctions.getElementsInDocument,
disableFixedAndStickyElementPointerEvents,
elementIsVisible,
elementHasAncestorTapTarget,
elementCenterIsAtZAxisTop,
getClientRects,
hasTextNodeSiblingsFormingTextBlock,
elementIsInTextBlock,
RectHelpers.getRectCenterPoint,
pageFunctions.getNodePath,
pageFunctions.getNodeSelector,
pageFunctions.getNodeLabel,
],
});
}
}

Expand Down
38 changes: 36 additions & 2 deletions lighthouse-core/lib/page-functions.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,39 @@
// @ts-nocheck
'use strict';

/** @typedef {HTMLElementTagNameMap & {[id: string]: HTMLElement}} HTMLElementByTagName */

/* global window document Node ShadowRoot */

/**
* Creates valid JavaScript code given functions, strings of valid code, and arguments.
* @template T, R
* @param {(...args: T[]) => R} mainFn The main function to call. It's return value will be the return value
* of `createEvalCode`, wrapped in a Promise.
* @param {{mode?: 'iife'|'function', args?: T[], deps?: Array<Function|string>}} _ Set mode to `iife` to
* create a self-executing function expression, set to `function` to create just a function
* declaration statement. Args should match the args of `mainFn`, and can be any serializable
* value. `deps` are functions that must be defined for `mainFn` to work.
*/
function createEvalCode(mainFn, {mode, args, deps} = {}) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could this be directly incorporated into evaluate() (and evaluateFunctionOnObject) instead of splitting it into this file? It's mostly not shared code since the two callers take different branches in the conditional (and it's not really a "page function" in the same way as the other functions in this file are).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer it here bc/ it's easier to test (driver has some overhead with beforeEach etc.) and
keeps the driver file smaller (it's so big). Maybe lib/eval.js?

And then export two function: createFunctionEvalCode and createIffeEvalCode?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer it here bc/ it's easier to test (driver has some overhead with beforeEach etc.) and
keeps the driver file smaller (it's so big). Maybe lib/eval.js?

And then export two function: createFunctionEvalCode and createIffeEvalCode?

well it would definitely be good to split the two, but it'll also have a new home after #11633, and it also looks trivial to add to .evaluateAsync in driver-test?

const {expression} = connectionStub.sendCommand.findInvocation('Runtime.evaluate');
expect(expression).toEqual(`(() => {
function mainFn() {
  return true;
}
return mainFn();
})()`);

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@patrickhulce would you prefer this going into driver.js or a new file lib/eval.js?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

new lib/eval.js file SGTM, though I'll also +1 @brendankenny's test suggest to make sure .evaluate uses it :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

new lib/eval.js file

Isn't the (currently named) RuntimeController going to be this already in #11633, though? I guess I don't understand what the problem is just inlining the five lines or so :)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't the (currently named) RuntimeController going to be this already in #11633, though?

Ya, that's why I don't really care much where it lives in this PR :)

I guess I don't understand what the problem is just inlining the five lines or so

I mean I don't feel terribly strongly about it, the rebase gets slightly more annoying with inline but not by much.

const argsSerialized = args ? args.map(arg => JSON.stringify(arg)).join(',') : '';
const depsSerialized = deps ? deps.join('\n') : '';
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: join() on an empty list is an empty string anyway. You can get rid of the conditionals.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

args and deps are optional params.

strangely, typecheck found no error when I removed the conditionals, even though both were marked as Array | undefined ...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

strangely, typecheck found no error when I removed the conditionals, even though both were marked as Array | undefined ...

it's because this file is // @ts-nocheck. Maybe after this change and followup(s), we could drop that with a minimum of expect-errors


if (!mode || mode === 'iife') {
return `(() => {
${depsSerialized}
${mainFn}
return ${mainFn.name}(${argsSerialized});
})()`;
} else {
return `function () {
${depsSerialized}
${mainFn}
return ${mainFn.name}.call(this, ${argsSerialized});
}`;
}
}

/**
* Helper functions that are passed by `toString()` by Driver to be evaluated in target page.
*/
Expand Down Expand Up @@ -78,9 +109,10 @@ function checkTimeSinceLastLongTask() {
}

/**
* @param {string=} selector Optional simple CSS selector to filter nodes on.
* @template {string} T
* @param {T} selector Optional simple CSS selector to filter nodes on.
* Combinators are not supported.
* @return {Array<HTMLElement>}
* @return {Array<HTMLElementByTagName[T]>}
*/
/* istanbul ignore next */
function getElementsInDocument(selector) {
Expand Down Expand Up @@ -463,9 +495,11 @@ const getNodeDetailsString = `function getNodeDetails(elem) {
}`;

module.exports = {
createEvalCode,
wrapRuntimeEvalErrorInBrowserString: wrapRuntimeEvalErrorInBrowser.toString(),
registerPerformanceObserverInPageString: registerPerformanceObserverInPage.toString(),
checkTimeSinceLastLongTaskString: checkTimeSinceLastLongTask.toString(),
getElementsInDocument,
getElementsInDocumentString: getElementsInDocument.toString(),
getOuterHTMLSnippetString: getOuterHTMLSnippet.toString(),
getOuterHTMLSnippet: getOuterHTMLSnippet,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ describe('Link Elements gatherer', () => {
function getPassData({linkElementsInDOM = [], headers = []}) {
const url = 'https://example.com';
const loadData = {networkRecords: [{url, responseHeaders: headers, resourceType: 'Document'}]};
const driver = {evaluateAsync: () => Promise.resolve(linkElementsInDOM)};
const driver = {evaluate: () => Promise.resolve(linkElementsInDOM)};
const passContext = {driver, url};
return [passContext, loadData];
}
Expand Down
Loading