fix(server): launch browsers when file_list is ready

[jpommerening]

After weeks of intermittent timeouts starting PhantomJS I finally found out what's wrong–

We're having a lot of files (well, not that many, maybe a hundred) and we're using grunt-karma to trigger single-run tests with PhantomJS in a continuous integration environment.

Correct me if I'm wrong, but it looks like the web server is not ready to reply to the browser's connect request until the file list is finished. Since PhantomJS' startup is really quick, it sometimes tries to connect before fileList.refresh() completed.

After applying this patch, the problem seems to disappear… what do you think?

[maksimr]

@jpommerening Thanks!

Do you use preprocessors in your project?

[jpommerening]

None that I'm aware of. I think we even disabled (or tried to disable? I don't remember…) the built-in HTML preprocessor because it wasn't playing well with RequireJS.

PS: It looks like this. First the timeout, then a pretty fast test-run:

[vojtajina]

Short story: I agree we need to fix this issue, but I don't think this PR is the right solution. I'm not sure what is the cause of the issue is.

---

Long story: Here is how Karma works:
It starts the webserver, once the webserver is listening on given port, start browsers.
A browser opens / url, the webserver serves static/client.html for that url, which establishes socket.io connection - the browser is captured. Responding to this request does not wait for resolving the files.

When a test run starts, all browsers refresh an iframe with actual tests. This iframe points to /context.html. Webserver does delay serving of /context.html, until the file_list promise is resolved (it needs updated list of files in order to generate the /context.html response).

Thus, capturing of the browser is not delayed until the files are resolved.

I'm not sure what the actual cause of your problem is. Sounds like the Karma process is too busy resolving all the files (and possibly preprocessing them) and the webserver does not get scheduled to handle the requests from browser. Can you try it with some GUI browser, eg. Chrome and check out the timeline - how long does it take to serve the initial / request, including its resources (such as static/karma.js, socket.io, etc)? Then the browser sends a message (through socket.io), "register", how long does that message take? Also, how many files do you have (I would like to reproduce the issue)?

Let's figure out what the cause is and then we can fix it properly.

[jpommerening]

Hey Vojta,

Thanks for that excellent summary. I think I understand the whole process a lot better now!

I'll follow your suggestions and post the results next week. Also, I'll try to build a small repo to trigger the problem.

Have a nice weekend!

On 02 Jan 2015, at 23:20, Vojta Jina notifications@github.com wrote:

Short story: I agree we need to fix this issue, but I don't think this PR is the right solution. I'm not sure what is the cause of the issue is.

Long story: Here is how Karma works:
It starts the webserver, once the webserver is listening on given port, start browsers.
A browser opens / url, the webserver serves static/client.html for that url, which establishes socket.io connection - the browser is captured. Responding to this request does not wait for resolving the files.

When a test run starts, all browsers refresh an iframe with actual tests. This iframe points to /context.html. Webserver does delay serving of /context.html, until the file_list promise is resolved (it needs updated list of files in order to generate the /context.html response).

Thus, capturing of the browser is not delayed until the files are resolved.

I'm not sure what the actual cause of your problem is. Sounds like the Karma process is too busy resolving all the files (and possibly preprocessing them) and the webserver does not get scheduled to handle the requests from browser. Can you try it with some GUI browser, eg. Chrome and check out the timeline - how long does it take to serve the initial / request, including its resources (such as static/karma.js, socket.io, etc)? Then the browser sends a message (through socket.io), "register", how long does that message take? Also, how many files do you have (I would like to reproduce the issue?

Let's figure out what the cause is and then we can fix it properly.

Can you open the timeline in the browser
This PR will make capturing browsers much slower (waiting for all files to be resolved and then starting browsers; it takes significant time for the browser to actually open the url). So I don't think this PR is a good solution.

�
Reply to this email directly or view it on GitHub.

[dignifiedquire]

@jpommerening any updates on this?

[dignifiedquire]

Closing for now.