Send messages from Elixir to Tortoise.Connection

[m1dnight]

Hi all,

I'm writing a system where I would like to process events coming from an MQTT server, through the Tortoise connection, into my regular Elixir code. From my regular Elixir code I would like to send commands back to the MQTT server.

So this would look something like this:

+----+     +--------+     +------+
|mqtt|<--->|tortoise|<--->|elixir|
+--+-+     +--------+     +------+

However, I notice that it doesn't seem possible to send messages from the application to the Tortoise connection handler.
I do know that you can manually send events directly to the connection using Tortoise.publish/3, but that, i think, splits the code concerned with MQTT communication in two. It would be nicer if all that logic would be contained within a single module.

Currently the diagram looks more like this:

+----+          +--------+           +------+
|mqtt|<-------->|tortoise|---------->|elixir|
+--+-+          +--------+           +-+----+
   ^                                   |
   +-----------------------------------+

So perhaps adding support for handle_info/2 or something would be useful?

This is all under the assumption that I understand what I'm doing. I might as well be missing something?

[brianmay]

Maybe just my state of mind right now, after a stressful morning, but unfortunately I am finding this request hard to follow. :-(

You are talking about outgoing messages, right? Or are you talking about something else?

I might have some ideas to help, but first I have to be able to understand the problem :-).

[m1dnight]

That's quite alright, I can't explain all that well, but I tried.

My initial problem was that receiving events from MQTT is a concern that you handle with functions in the Tortoise handler (i.e., implement the right handle_message/3 callbacks). But the concern of sending an event from Elixir to the MQTT (e.g., "turn on lamp xyz") should also be a concern that you implement in that same handler module. But you cannot send a regular message to the handler process (e.g., send(HandlerId, {:foo, :bar})) because you cannot implement callbacks for those messages in the Tortoise.Handler module. In GenServer these are the handle_info/3 callbacks.

The thing I did not notice right away is that sending a message is just a function call (Tortoise.publish/3), so I ended up adding functions in the module myself that use Tortoise.publish/3, and that works too, but I was wondering if there was a more idiomatic approach.

[gausby]

The module you use for your tortoise handler is supposed to do very little, because it is running in the tortoise controller process, which means that an error in this part of the code would result in a tortoise crash, and while the code is running the controller is blocked: While most MQTT connections are not high volume I recommend doing as little as possible in the tortoise controller, and just route the messages to a process where you have all your logic for your application.

Would that work for you ?

[brianmay]

For myself[1], I ended up creating a process that receives subscribe requests from processes (e.g. scenic scenes, Phoenix live handlers, etc), and it routes the messages to the processes that have requested them. See https://github.com/brianmay/robotica-elixir/blob/master/robotica_plugins/lib/subscriptions.ex [2]. The tortoise handler is here: https://github.com/brianmay/robotica-elixir/blob/master/robotica/lib/robotica/client.ex#L84

It also keeps track on the last received message, and re-sends it on a new subscription. Ideally this probably should only happen for retained messages, but I couldn't work out how to tell them apart. For my use case all messages are retained.

It also doesn't support wildcard subscriptions. I haven't needed them.

Unfortunately, I couldn't get dynamic MQTT subscriptions to work reliably. See #130. So the mqtt subscriptions need to be hardwired into the initialization code. But regardless, this code still serves my purpose. The filtering of unwanted messages will occur in my code, not the mqtt server. If I ever got dynamic subscriptions working, I probably would want to listen to connection(:up, state) and reconnect to all subscriptions. As currently I think tortoise might forget dynamic subscriptions on reconnect (although I guess this might change also).

Thinking about what @gausby said, possible the message() function should be a cast not a call. But the routing shouldn't really take too long, and the actual sending is done via cast.

Notes:

[1] Code is "first working draft" status. It works, but readability probably can/should be improved now.
[2] Name subject to change because module name is misleading. I initially created RoboticaPlugins to do plugins stuff, but now it is everything except plugins stuff!