WaitTimeoutError during test #537

ryuran · 2017-09-02T11:59:21Z

Helo!

After an error on Travis (OnetapInc/chromy#62), I try to reduce async limit to pass test.

asyncCaptureLimit: 5,
asyncCompareLimit: 50

But nothing change 😢 : https://travis-ci.org/20minutes/colette/builds/271137302

Then I try with

asyncCaptureLimit: 1,
asyncCompareLimit: 1

https://travis-ci.org/20minutes/colette/builds/271199288
I don’t know why it’s green even if i have no result of the test…

The text was updated successfully, but these errors were encountered:

garris · 2017-09-02T17:47:01Z

Did you try to reduce to 1?

ryuran · 2017-09-02T22:07:35Z

@garris yes, my second try is at 1:1
It’s better but travis return positive whereas backstop was not able to finish test and I have some errors in logs.

ChromyJS returned an unexpected error while attempting to capture a selector. { WaitTimeoutError: evaluate() timeout
    at EvaluateTimeoutError.ExtendableBuiltin (/home/travis/build/20minutes/colette/node_modules/chromy/dist/error.js:44:28)
    at EvaluateTimeoutError.TimeoutError (/home/travis/build/20minutes/colette/node_modules/chromy/dist/error.js:73:128)
    at new EvaluateTimeoutError (/home/travis/build/20minutes/colette/node_modules/chromy/dist/error.js:118:145)
    at Chromy._evaluateWithReplaces$ (/home/travis/build/20minutes/colette/node_modules/chromy/dist/document.js:484:21)
    at tryCatch (/home/travis/build/20minutes/colette/node_modules/babel-runtime/node_modules/regenerator-runtime/runtime.js:65:40)
    at Generator.invoke [as _invoke] (/home/travis/build/20minutes/colette/node_modules/babel-runtime/node_modules/regenerator-runtime/runtime.js:299:22)
    at Generator.prototype.(anonymous function) [as throw] (/home/travis/build/20minutes/colette/node_modules/babel-runtime/node_modules/regenerator-runtime/runtime.js:117:21)
    at tryCatch (/home/travis/build/20minutes/colette/node_modules/babel-runtime/node_modules/regenerator-runtime/runtime.js:65:40)
    at invoke (/home/travis/build/20minutes/colette/node_modules/babel-runtime/node_modules/regenerator-runtime/runtime.js:155:20)
    at /home/travis/build/20minutes/colette/node_modules/babel-runtime/node_modules/regenerator-runtime/runtime.js:167:13
    at <anonymous> name: 'WaitTimeoutError' }

garris · 2017-09-05T04:09:24Z

I have just verified that asyncCaptureLimit and asyncCompareLimit are working as designed. At this point I think you need to look at why your particular test is failing.

ryuran · 2017-09-05T09:09:03Z

I add log for test scenario label.
It seems fail on ./docs/item-theme-skins.html or the next one ./docs/item-theme-styles.html.
I check this files with html validator, they are valid.
So I’m blocked. All is fine in local environment.

kiran-redhat · 2017-09-05T09:18:48Z

@ryuran 1. are you about to find that in chrome console ?

Do you see sale error with asyncCaptureLimit: 1,?

If 1 &2 are yes, clone latest master and test. asyncCaptureLimit and asyncCompareLimit we're working fine for me

ryuran · 2017-09-05T09:24:15Z

@kiran-redhat

I don’t know how to return more info from browser in travis-CI logs
This travis build is running with asyncCaptureLimit: 1 https://travis-ci.org/20minutes/colette/builds/271948792 I have the error

kiran-redhat · 2017-09-05T09:38:48Z

for number 1: open your app in Chrome browser press f12 then click on console then copy and paste your selector and hit enter. if you find element wait for your selector to be visible (probably chromy.visible() )

…

On Sep 5, 2017 9:24 PM, "Yvain Liechti" ***@***.***> wrote: @kiran-redhat <https://github.com/kiran-redhat> 1. I don’t know how to return more info from browser from travis-CI 2. This travis build is running with asyncCaptureLimit: 1 https://travis-ci.org/20minutes/colette/builds/271948792 <https://travis-ci.org/20minutes/colette/builds/271948792> I have the error — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#537 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AQUrIz_uD5ROOyWVDt21w2YFEBIEIiIwks5sfRNCgaJpZM4PK4Y4> .

ryuran · 2017-09-05T11:12:58Z

No error in my local chrome.
Backstop.js run fine in local environment with all test to green.

I update backstopjs:

└─┬ backstopjs@3.0.24
  └── chromy@0.5.2

And I try to change delay to 200 (it was 100), but it stops always on the same test.
Without error this time but with travis timeout:
https://travis-ci.org/20minutes/colette/builds/271982035

ryuran · 2017-09-05T12:21:21Z

It seems fail most often on test with <img>

reynaldoapb · 2017-09-05T18:20:32Z

Try to kill chrome with shell command taskkill /F /IM chrome.exe and run everything again.

axe312ger · 2017-10-10T12:27:15Z

Turning it down to

asyncCaptureLimit: 1,
asyncCompareLimit: 1

worked for me on travis :)

mhavelant · 2017-10-17T11:34:15Z

Hi!

I got the same error. The async limits at 1 did not help, however setting "selectors" to "viewport" from "document" did. So it might be something with chromy with 'document' as the selector.

Edit: Setting asyncCaptureLimit to 1 AND not using "document" together worked. Setting the capture limit to something higher broke it.

garris · 2017-10-17T13:26:12Z

What version of chrome and backstop are you using?

mhavelant · 2017-10-17T13:35:31Z

The newest ones:

Google Chrome 62.0.3202.52 beta
BackstopJS v3.0.29

Note: I'm running under a dockerized Ubuntu 16.04

tfrijsewijk · 2017-10-19T14:43:28Z

I'm experiencing the same issue, with the same versions as mhavelent

garris · 2017-10-19T15:01:24Z

Hello all. Thank you very much for posting.

It may be possible that #561 (running out of memory) is related to this issue — which only seems to happen on Linux.

@vdekov posted a fix for #561 here... #571

Would it be possible for someone to pull this PR and see if the problem resolves?

mhavelant · 2017-10-19T15:18:27Z

For me, the error

ChromyJS returned an unexpected error while attempting to capture a selector. { WaitTimeoutError: evaluate() timeout...

is thrown before the 'BackstopTools have been installed.' step in chromy. So long before comparisons are even considered, in the reference command.

So the PR, although sounding insanely good overall, I don't think will help here.

At least for me, it seems that the 'document' selector is not working with Chromy.

garris · 2017-10-19T16:24:28Z

Oh. Crap. You’re right. Ok — scratch that. 😕

kevineye · 2017-10-26T15:48:42Z

I was getting this WaitTimeoutError, too, not running natively, but when switching to run in docker. Happened more often with higher asyncCaptureLimit and more complex pages, but even sometimes with asyncCaptureLimit: 1.

Eventually tracked the problem down to chrome dying with a bus error, and finally tracked this down to running out of space in /dev/shm. Docker allocates 64MB to /dev/shm by default.

Running docker with --shm-size 512m fixed the issue for me. Lets me run with default asyncCaptureLimit: 10.

garris · 2017-10-26T17:10:44Z

@kevineye Nice debugging — thanks for posting!

garris · 2017-11-22T21:48:22Z

cross referencing this issue... #603 (comment)

- Changed base image to none alpine. Allowing new chrome version. - Updated Chrome to the latest (64). Potentially fixing issues garris#603 garris#537

- Changed base image to none alpine. Allowing new chrome version. - Updated Chrome to the latest (64). Potentially fixing issues #603 #537

ryuran mentioned this issue Sep 3, 2017

ChromyJS returned an unexpected error while attempting to capture a selector. OnetapInc/chromy#62

Open

garris changed the title ~~asyncCaptureLimit and asyncCompareLimit seems not working~~ WaitTimeoutError during test Sep 5, 2017

Kenith mentioned this issue Nov 15, 2017

Possible EventEmitter memory leak detected #594

Open

garris mentioned this issue Nov 22, 2017

BackstopJS hangs in Docker container #603

Closed

iain17 added a commit to iain17/BackstopJS that referenced this issue Feb 13, 2018

Changes in dockerfile:

aacd85b

- Changed base image to none alpine. Allowing new chrome version. - Updated Chrome to the latest (64). Potentially fixing issues garris#603 garris#537

iain17 mentioned this issue Feb 13, 2018

Fixing Docker build issues. Updated chrome, Potentially fixing issues #603 #537. #668

Merged

garris pushed a commit that referenced this issue Feb 13, 2018

Changes in dockerfile: (#668)

6b89ad2

- Changed base image to none alpine. Allowing new chrome version. - Updated Chrome to the latest (64). Potentially fixing issues #603 #537

aaemnnosttv mentioned this issue Aug 21, 2024

Fix issue where VRT test fails in CI when launching Chromium google/site-kit-wp#9211

Closed

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WaitTimeoutError during test #537

WaitTimeoutError during test #537

ryuran commented Sep 2, 2017 •

edited

Loading

garris commented Sep 2, 2017

ryuran commented Sep 2, 2017 •

edited

Loading

garris commented Sep 5, 2017

ryuran commented Sep 5, 2017

kiran-redhat commented Sep 5, 2017

ryuran commented Sep 5, 2017 •

edited

Loading

kiran-redhat commented Sep 5, 2017 via email

ryuran commented Sep 5, 2017

ryuran commented Sep 5, 2017

reynaldoapb commented Sep 5, 2017

axe312ger commented Oct 10, 2017

mhavelant commented Oct 17, 2017 •

edited

Loading

garris commented Oct 17, 2017

mhavelant commented Oct 17, 2017

tfrijsewijk commented Oct 19, 2017

garris commented Oct 19, 2017

mhavelant commented Oct 19, 2017

garris commented Oct 19, 2017

kevineye commented Oct 26, 2017

garris commented Oct 26, 2017

garris commented Nov 22, 2017

WaitTimeoutError during test #537

WaitTimeoutError during test #537

Comments

ryuran commented Sep 2, 2017 • edited Loading

garris commented Sep 2, 2017

ryuran commented Sep 2, 2017 • edited Loading

garris commented Sep 5, 2017

ryuran commented Sep 5, 2017

kiran-redhat commented Sep 5, 2017

ryuran commented Sep 5, 2017 • edited Loading

kiran-redhat commented Sep 5, 2017 via email

ryuran commented Sep 5, 2017

ryuran commented Sep 5, 2017

reynaldoapb commented Sep 5, 2017

axe312ger commented Oct 10, 2017

mhavelant commented Oct 17, 2017 • edited Loading

garris commented Oct 17, 2017

mhavelant commented Oct 17, 2017

tfrijsewijk commented Oct 19, 2017

garris commented Oct 19, 2017

mhavelant commented Oct 19, 2017

garris commented Oct 19, 2017

kevineye commented Oct 26, 2017

garris commented Oct 26, 2017

garris commented Nov 22, 2017

ryuran commented Sep 2, 2017 •

edited

Loading

ryuran commented Sep 2, 2017 •

edited

Loading

ryuran commented Sep 5, 2017 •

edited

Loading

mhavelant commented Oct 17, 2017 •

edited

Loading