[Feature]: locator.evaluateAll to have timeout option

[v-mwalk]

🚀 Feature Request

Currently, locator.evaluateAll does not have a timeout feature. Other evaluate's (page/locator) have these options, but evaluateAll does not.

Issue #13253 discusses it but not really sure what this issue is requesting now as it is very old (> 3 years) and seems to have a number branches within.

I am using a workaround (wrapping in a promise evaluator with setTimeout), but - as #13253 (comment) points out - it makes the test code quite unreadable.

Example

const attributeValues = await locator.evaluateAll((elements, attribute) => {
return elements.map(el => el.getAttribute(attribute) || '');
}, attributeName, {timeout: 10000});

Motivation

Add consistency in Playwright, so all functions that interact with the browser have a local overriding timeout ability.

[dgozman]

@v-mwalk What do you expect to happen when this timeout triggers? An error to be thrown? Note that evaluation in the page, once started, will not be cancelled, unlike other Playwright actions. This makes the behavior surprising, because you probably won't expect an evaluate() action that throws an error to actually continue in the background and finish.

I think your workaround with racing against setTimeout is a good one. Let me know what you think.

[v-mwalk]

Must admit, I dont fully understand your question. locator.evaluate() has a timeout parameter, so I would expect the functionality for evaluateAll to be the same; expect it takes all the matching elements rather than just the singular element..

How does the locator.evaluate() cancel (or not) the evaluation in the page?

Ass with locator.evaluate(), if the evaluateAll reaches its timeout I would expect it to throw the same error as locator.evaluate()

[dgozman]

@v-mwalk locator.evaluate() only applies timeout to finding the element, e.g. waiting for the locator to appear on the page, and after that the evaluate itself does not respect the timeout. That does not make sense for evaluateAll() because it does not wait for any elements to appear/disappear, and instead just takes all the matching elements available in the page at the moment.

[v-mwalk]

That's not what I/we infered from the documentation; Locator Evaluate timeout.

So I did a quick test and an evaluate with infinate loop;

try {
   await validLocator.evaluate((el) => { while (true) { console.log('Run forever...') } }, 1000);
}
 catch{
     throw new Error(`Yippeee - Bombed`);
}

As you said, it hangs there, even though I gave it a 1 second timeout (proving that it is just the locator wire-up that has the timeout). I kinda think the doco should make this clear (or am I missing something somewhere) as a few of our guys disagreed with you until I did the above proof.

I'll use my own timeout wrappers for evaluates & evaluateAlls. Yes, I appreciate that it would leave the browser-side code running, but I'd consider that to be the responsibility of the Playwright consumer code (In my framework's case, it aborts the current test and tears down the browser with appropriate messaging informing results log that there was a timeout.