Using faye-websocket, receiving multiple copies of same Slack message

[jcraigk]

I'm using

client = Slack::RealTime::Client.new(token: <team_token>)

where <team_token> is an access token for a specific team. I'm then using

Thread.new { client.start! }

OR

client.start_async

to start the client in its own thread. Both produce the same results.

After a period of a few minutes of uptime, the messages being received through client.on :message appear multiple times. After a few minutes, I'm receiving upwards of 10 copies of every message. An app restart resets the issue. After a few minutes I see the duplicates building up again.

When booting multiple instances of the app with different team tokens hooked into the same Slack channel (2 different bots), it's clear the duplicates are happening on the client side. At one point one of the app instances will be reporting only one copy of each message and the other instance will report multiple copies of the same message.

By adding Thread.current.object_id to the debug logging, it's clear the duplicate messages are being received by the same thread. Here are the details of an event that I see multiple times, for example:

Event: channel=CL2UC3ABE, client_msg_id=2dbceee9-1b81-475c-aea7-9e145b8aa176, event_ts=1563745122.053100, source_team=TKS0LJV6W, suppress_notification=false, team=TKS0LJV6W, text=<@UKX13JK26> test, ts=1563745122.053100, type=message, user=UL5H1BQTG, user_team=TKS0LJV6W

I will see this multiple times in a row, all with the same exact info. It's as if there are multiple instances of the websocket open for a team when there should be only one...?

See https://github.com/jcraigk/phishin-chatbot/blob/master/app/services/websockets/slack.rb for my current implementation.

[dblock]

I am going to guess that you're spawning these multiple times, possibly through some loop on a disconnect? Save the threads that you're spawning and try to join them.

If you want a tested implementation for multiple teams try https://github.com/slack-ruby/slack-ruby-bot-server?

[jcraigk]

@dblock Thank you for the rapid response. After further debugging it appears to be an issue with faye-websocket. Since switching to 'async-websocket', '~> 0.8.0' in my Gemfile, as suggested in the README, the issue seems to have resolved itself.

Some additional info: after extensive manual and automated testing including logging thread IDs and Slack message IDs from multiple app instances using multiple bots, I am quite certain my app logic is maintaining only one thread per team and only one call to client.start! per team, so it does appear to have been an issue with faye-websocket. I will just stick with async-websocket going forward.

[dblock]

The ping thread causes a disconnect, properly sends :close, then calls restart_async. This re-registers a driver with all its :on handlers, but fails to shutdown the existing one. I couldn't find an easy way to do that.

[dblock]

We should remove EM support via #338

[dblock]

#357