Upnode for reconnects

[Iristyle]

I was looking through the issue tracker for TestEm and came across this issue, mentioning upnode
testem/testem#36

Thought it might be useful for some of the reconnection issues mentioned in other issues.

[dignifiedquire]

I'll have a look at this as upnode seems to be much more reliable than the current system.

[dignifiedquire]

After looking a little bit into the matter, I think it would be great to rewrite the connections between server and browser to use SockJS with shoe, and dnode with upnode.
A short explanation for those unfamiliar with the topic and the libs

• SockJS is a library that gives WebSocket support without falling back to flash it's pure javascript
• shoe is a thin wrapper for SockJS for use with dnode
• dnode is an asynchronous rpc system for node.js that lets you call remote functions
• upnode keeps a dnode connection alive and re-establishes state between reconnects with a transactional message queue

Here is a simple example from the readme of dnode.

I think this would greatly improve connection times and reliability. I would consider doing this rewrite if @vojtajina says that this is something worthwhile to pursue.

[Iristyle]

Thanks for digging in a little more... +1 that this seems like a better solution.

[Iristyle]

FYI -- here's a bit more info on the socket.io vs SockJS
https://groups.google.com/forum/#!topic/sockjs/lgzxVnlth54

[vojtajina]

We still need to be able to support other transports (eg. flashsocket seems to be best solution for IE) and we have other devices (eg. mobile phones, etc...) that does not support web sockets... But I think shoe supports fallbacks as well.

What do you expect to get better ? Because I don't see any connection issues (with browsers supporting websockets).

I'm open to this, but would be good to have some metrics. Seems to be worthy research...

[dignifiedquire]

I think we have to differentiate between shoe and sockjs as a replacement for socket.io and updnode + dnode. The last two can also be used in combination with sockjs.
The main reasons I see für upnode + dnode is that first they seem to me well tested and would allow to extract some of the connection logic that is handled manually in testacular. Second they allow for a in my opinion a much cleaner connection interface that what is now in place.

For more about sock.js you can have a look here for a overview of the fallbacks and possible transport layers. I haven't found any benchmarks comparing the two but this is definitively something that should be done before switching.

Update: I've found two benchmark reports. The one that is more relevant for us is probably the first one

• SockJS vs Socket.IO - Benchmarked
• sockjs bench

[vojtajina]

I'm totally for trying SockJS. Thanks for bringing this in.

Not sure, how would dnode make Testacular better ?
What connection logic could be gone ?

Why do you find dnode RPC nicer than just events (socket.io) ?

[dignifiedquire]

Why I would prefer RPC is just a personal taste, if you have a look at this example I personally just find this more elegant and cleaner to write than just events. But if you say you want to stay with the events I can totally understand it.

Having said that I think that upnode would give the connections more flexibillity in the way the connections are established and make them more resistent to disconnects.
For example with upnode you get a que of method calls when disconnected, so that when someone accidentally closes the browser in the middle of running the tests it should be possible to continue as soon as he reopens the browser.

[vojtajina]

I prefer events, but agree, it's not a big deal.

If user closes browser (or it crashes), reconnecting does not help you, you need to start the test run again, as you lost all the state (what tests were executed, etc..).

But we should handle the case, where the browser just disconnects and quickly reconnects (but browser itself is ok, running the tests). That's possible to solve even with socket.io - we just need to have a small timeout, when browser disconnects during the run, wait this timeout and fail the build. If the browser reconnects in given timeout, keep going. AFAIK socket.io should not lost any messages.

Let's do some benchmarks with SockJS. I only read about it, but I think we should definitely try it. Sounds promising.

[dignifiedquire]

Yes some benchmarks are the first thing we need. I'd say some simple ones without testacular involved and some that bench testacular as it is right now so we can see if it really makes a diffference in the real world.

[vojtajina]

Yep, I would be curious, what are the benefits if we switch, because
currently I don't see any issues on browsers with websockets. But any
improvement (eg. speed), would be a reason for switching. SockJS looks
promising and little bit lightweight.

On Thu, Oct 18, 2012 at 11:20 PM, Friedel Ziegelmayer <
notifications@github.com> wrote:

Yes some benchmarks are the first thing we need. I'd say some simple ones
without testacular involved and some that bench testacular as it is right
now so we can see if it really makes a diffference in the real world.

—
Reply to this email directly or view it on GitHubhttps://github.com/vojtajina/testacular/issues/95#issuecomment-9590926.

[dignifiedquire]

I've started looking more into this. SockJS is much more lightweight and only provides a raw stream and in combination with shoe it's mostly like unixy stream. There is no such thing like events build into this, so we need an additional layer to use theses streams. I've got a simple example of the combination upnode + shoe working here.
Will try to implement this into testacular and see what the performance says.

This might also help regarding #203 as sockjs only uses jsonp polling in ie and no flash stuff.

[dignifiedquire]

I've been doing some research and upnode and all this stuff might be a total overkill. A small event emitter layer for SockJS would be implemented quite easy and wouldn't much less overhead than adding all these additional moving parts.

[ilyavolodin]

I haven't seen SockJS before, but after reading the description, and looking through the code, my guess is that it will be significantly slower than Sockets.IO for IE if your testacular server is located on the different machine from the browser. My guess is for IE they maintain Comet connection from the server, and initialize a new connection from the client every time there's a request, which means open connection, handshake, send cookies, etc. Flash is not a great technology for something like WebSockets, but at least it does maintain an actual sockets connection to the server.
I would defiantly benchmark IE before committing to SockJS.

[dignifiedquire]

Thanks to @ilyavolodin I've investigated a little bit further into the different fallbacks and unfortunately he is right. The flash fallbacks are much faster than the jsonp polling used by SockJS. So I have to advise against using SockJS at this point.

Upnode and dnode could still be leveraged if the interest is there. One would just switch out Socket.IO with Engine.IO (the basic transport layer of Socket.IO) But I don't think that at this point there would be real value there.

I will investigate the reconnection issues further and try to solve them using Socket.IO but for now I'll close this issue. Please reopen if any other ideas regarding this come to mind.

[vojtajina]

@dignifiedquire I agree that investigating SockJS sounds interesting, I'm curious as well. But it's not a big deal for me now.

However, I don't think not using flashsocket is an advantage, my experience is that flashsocket works pretty well on IE (definitely faster than jsonp). The #203 is more likely not an issue of socket.io, I guess the browser process returns immediately for some reason.

[Raynos]

There are three points here.

• is socket.io a good communication transport between a browser and a server for a test runner
• is an RPC abstraction a better way to communicate (dnode)
• does upnode handle reconnections better.

For the first I highly recommend you look into alternatives like engine.io or SockJS. My personal experience with SockJS and talking to @3rdeden makes me think engine.io may be better suited but I have no evidence for that. Engine.io is superior to socket.io because it's far lighter and newer, it also makes different trade offs.

An RPC system like dnode is probably not optimum to the test runner so don't switch to dnode because of that.

For the reconnection issue, handling reconnections is hard. I've done it a few times and have had the best experience with upnode. This however has a dependency on dnode so you will need to rewire your communication through RPC which isn't that hard.

I recommend you switch to engine.io from socket.io as it's generally a good idea. Once you have engine.io in place and find reconnections to still be an issue consider using something like engine.io-stream to hook up upnode communication, which may or may not make reconnection issues go away for you.

Reconnection issues are only a big deal if one of your workflows is to start a testacular server, hook up your browsers and keep it running the entire day on a single browser <-> server connection whilst doing your TDD flow.

[vojtajina]

Hey @Raynos - thanks for this info. Very valuable!

engine.io is basically next generation of socket.io, respectively the low level of socket.io, it's done by Guillermo as well.

I totally agree with upgrading to engine.io.