Reconnect to Twitter

[masterkain]

Hello,
I'm being plagued by disconnects, the program keeps running but I get 0 entities from the twitter stream.

I already put down my error, end, close events to no avail.

There's a solution proposed here: https://github.com/iStrategyLabs/twitter-node/commit/3adc1686e2925fce16a416c3672bca3b9f507a7e

What do you think about it?
I suspect it might violate twitter rules about reconnecting as per documentation, but it can be a starting point.

Cheers

[yinshanyang]

Hi, I too was plagued by disconnects, of the silent flavour. I have been running my program on no.de and after a couple of hours, the stream would simply stop receiving tweets.

What I have found is that the 'destroy' event helps here, since some types of disconnections would not fire the 'error' or 'end' events. i.e. those outlined at https://dev.twitter.com/docs/streaming-api/concepts#connecting

I need to let my program run for a couple more days for confirmation, but I thought you might find the 'destroy' event useful.

[martindale]

This should probably be upgraded, with respect to Twitter's official recommendations regarding connection failure: https://dev.twitter.com/docs/streaming-api/user-streams/suggestions#failure

This proves to be an issue in all deployments of ntwitter, and should be considered a standard feature.

[AvianFlu]

This is important, but I haven't had the time lately to work on it. I was given a gist of a sample algorithm that follows Twitter's recommendations very well, though, so this will be implemented soon.

[pixeldrew]

What's the status of this? Should I go ahead and test 570c65f?

[pixeldrew]

I've tested 570c65f and i'm still getting disconnects but at least I can determine if that's happened with PR#29. Can I get a version bump and a release to npm?

[lukekarrys]

@pixeldrew I believe that 570c65f only adds to ntwitter the ability to differentiate between http and tcp/ip errors and a silent destroy method. However, these two things give the ability to detect the possible types of disconnects and account for them appropriately and inline with Twitter's suggestions. Check out this repo it helped me out a lot when trying to code and account for my disconnects.

_Note: I'm right in the middle of coding this for a project due by the end of the week, so I don't know _for sure* that this will work, but from reading, it looks like it will.*

[pixeldrew]

Ah, awesome! Thanks Luke.

I did something similar with my app but I was having some issues with properly destroying the socket on errors and developed a race condition that made re-connecting to the stream fail because twitter only allows one connection to the stream per access token. Long story short, pressed for time I backed out of reconnecting the stream if it fails. So far twitter hasn't disconnected in 4 days so i'm not in trouble yet...

I'll look at refactoring my app to use your module when I get some time but I'd really like to have something similar added to ntwitter and for a version bump and release to npm. Right now my build process involves having to add a patch to the npm build :(

[lukekarrys]

Agreed on the version bump and release to npm :). I'm doing a similar thing as you currently to get around that. Also, the repo I linked to wasn't mine, just something that I found in my searches about keeping streams open.

In my app I also have the requirement of needing to manually destroy the stream at certain times, which the code I linked to doesn't allow. Had to come up with a hackish way of setting a flag that wouldn't let the .on('end' and .on('destroy' reconnect.

Will be glad to see this make it's way into ntwitter. Thinking about trying to implement it myself once my app has launched and can see how my code fares. @AvianFlu would that be welcomed?