Proposal to add inter process communication using mqtt protocol to Jamulus

[cdmahoney]

Proposal

Currently the only way to extend the functionality of Jamulus is by modifying the source code, or running external applications on the server that "know" where jamulus resources are, and can start systems programs and/or services in order to start/stop recording, download recordings, etc. As far as I am aware there is no way for these external applications to determine automatically the running Jamulus servers, or the current connections of those servers.

This proposal is to add inter process communication to Jamulus using the mqtt protocol. This would enable external applications to receive information and communicate with Jamulus in a platform-independant way.

MQTT subscriptions, publications, topics, and message format

There would be two separate options configurable when starting a Jamulus server:

1. Indicate a host and port to use to publish and subscribe to MQTT topics. Subscribed topics are requests for information only, and do not change the state of the server.
2. Indicate a host and port to use to subscribe to Jamulus messages which can change the state of the server.

This would enable a server to publish messages while not subscribing to messages which might for example start or stop recording on the server.

The Jamulus server would publish messages using topics such as:
jamulus/22124/recording/start
jamulus/22124/recording/stop
jamulus/22124/newconnection
jamulus/22124/channelinfo

"22124" would be replaced by the server's port number.

The Jamulus server would subscribe to topic:
jamulus/22124/request/#

Client code would send messages with topics such as:
jamulus/22124/request/configuration
jamulus/22124/request/connections

For messages which might change the state of the server Jamulus would subscribe to topic:
jamulus/22124/put/#

In all cases message content would be in JSON format.

Use cases

Examples of use cases for this functionality would be an application to provide detailed logging of Jamulus events, or a web-application providing a user-friendly interface to the server state, and the ability to start and stop Jamulus recordings.

In both cases, by subscribing and optionally publishing to an mqtt broker, the applications could determine which Jamulus servers are running, and obtain information about the current state of the servers. The applications would not need to know in advance the ports used by the servers, or the folders used for example to store recordings.

Implementation

I have a proof-of-concept implementation which I could publish as a fork of Jamulus (I implemented the functionality before making the proposal to avoid proposing something that I wasn't then able to implement - this has been my first experience with the Jamulus code base.)

The MQTT implementation is based on the current recording module implementation. A subfolder contains the implementation of the mqtt controller and mqtt connections for publish and subscribe. Changes to the core Jamulus code are minimal, and can be divided into two groups:

1. Code to read command line parameters, pass the parameters to the server instance, and instantiate the mqtt controller if connections have been defined.
2. Changes to make public additional information about the Jamulus server state. This includes new signals in CServer (ServerStarted, ServerStarted, ...), adding a method GetServerInfo() to CServerListManager, and similar changes. In all cases the idea is to minimize the performance impact on the server, especially when no mqtt connection is configured.

Attached is a screenshot of MQTT Explorer showing messages from a Jamulus session.

[ann0see]

Hi! Thank you for this detailed explanation.

Basically this would make it possible to have a server API? I think we thought about this a while ago, but never got further than having a vague idea. Having an API would make Jamulus a lot more flexible!

In theory this could be extended to also control Jamulus clients, I think?

[cdmahoney]

Yes, effectively this would provide an API to the Jamulus server. As you say, it should also be possible to do the same for Jamulus clients, though I haven't tried this. However, this would require access to a MQTT broker on the client side, which would limit its usefulness.

One thing I didn't make clear in the proposal is that the MQTT broker that the Jamulus server connects to is not meant to be public. On testing on a lightsail-type server I enable the MQTT broker on the server, then connect Jamulus server and a web application to the MQTT broker. The web application uses OAuth2 for authorization, then publishes information over its own MQTT implementation using WebSockets. The client only connects to the web application, using its API and WebSocket to get and send information. All of this means that the Jamulus server doesn't have to worry about authorization or security - they are outsourced to the web application. There is no direct external access to the Jamulus server.

[ann0see]

Yes. That sounds good. So e.g. a PHP Script would be able talk to Jamulus locally.

Having some kind of API on the client would make the use of headless clients a lot more useful ;-)

[hoffie]

Cool!
I certainly welcome the idea of creating a method for better API control for both the client and the server. @pljones had some thoughts in that area as well, I think.

Personally, I've rather thought of some HTTP API. I haven't thought about MQTT and I've never used it so far. I guess at least for the pubsub model, HTTP is less tuned (e.g. "stream all recording events to me").

Can you describe what your reasoning for choosing MQTT was? How is (cross-platform) library support? How complex is it, what licenses would be involved? Are there similar use cases where MQTT is used?

[cdmahoney]

I think MQTT has some clear advantages over HTTP:

1. The publish/subscribe model means clients can get instant updates on the state of the Jamulus server. No need for polling of the server.
2. MQTT uses an external broker - no need to implement server functionality in Jamulus, as would be the case with HTTP. Eclipse Mosquitto (https://mosquitto.org/) is an open source (EPL) MQTT broker implementation with versions for Linux, Windows, and Mac, but any other MQTT broker could be used,

My experience with MQTT has been with Internet of Things applications, which combine a REST API with MQTT messaging to create interactive web applications. My idea for Jamulus was to drop the rest API and use pre-defined message topics for what would normally bet GET/PUT/POST requests - this is just to simplify the implementation, as Jamulus only has to implement publishing/subscribing to the MQTT broker.

My code uses the Qt MQTT library (https://doc.qt.io/QtMQTT/index.html) to connect to the MQTT broker. The library fits in very nicely with the Qt event mechanism, is available under the GNU General Public License, which I assume is compatible with Jamulus's current use of Qt.

I'd consider MQTT code to be of low complexity. For Jamulus the most complex case might be to subscribe to a topic (for example, client requests server configuration) and publish the required information when a message is received. The following code sample is adapted from my test implementation:

void CMqttConnectionRequest::OnMqttConnectedInternal()
{
    // Subscribe to topic jamulus/<server port>/request/#
    QMqttTopicFilter topicFilter = GetTopicFilter("jamulus", "request/#");
    QMqttSubscription* subscription = Subscribe(topicFilter, &CMqttConnectionRequest::OnMqttSubscriptionMessageReceived, &CMqttConnectionRequest::OnMqttSubscriptionStateChanged);
}


void CMqttConnectionRequest::OnMqttSubscriptionMessageReceived(const QMqttMessage &msg)
{
    QMqttTopicName topic(msg.topic());
    QMqttTopicFilter topicFilterGetConfiguration = GetTopicFilter("jamulus", "request/configuration/get/#");
    if(topicFilterGetConfiguration.match(topic))
    {
        if(Server && ServerListManager)
        {
            const CServerListEntry* serverInfo = ServerListManager->GetServerInfo();

            QJsonObject jsonObject;
            // Populate jsonObject

            QMqttTopicName topicName =  GetTopicName("jamulus", "response/configuration/get");
            PublishMessage(topicName, jsonObject);
        }
    }
}

There is some code behind the scenes here (Subscribe, PublishMessage) which wraps frequently used actions, but doesn't add significantly to the complexity. Publishing messages for Jamulus server state changes is even simpler - just listen for the events triggered by the server and publish a message when the event ocurrs.

[pljones]

My goal is clear - move to a Controller/Service model, with the Controller layer replaceable and the Service layer exposing a full API. The aim would be to allow replacing the existing Client and Server GUIs with a variety of remote control interfaces, as needed.

[cdmahoney]

OK, so when 3.8.0 is out I'll make any required changes to my code and put it on github. Then you can decide if it's suitable for including in Jamulus.

[ann0see]

Yes. This sounds good. You can soon open the PR (probably later this evening)

[cdmahoney]

Great! It'll probably be the weekend before I can get it posted, will add a comment when it's done.

[ann0see]

Although it's not yet officially announced, the 3.8.0 binaries are out now. Unless we really have a serious issue and need to fix it immediately, you can open it now.

[cdmahoney]

OK, Downloaded code last night and compiled project, now to add changes and put on github...

[DetlefHennings]

Maybe MIDI channels between the server and all clients, and also to and from the 'external world' via midi inputs and outputs, could be an alternative system. For Jamulus control sysex messages could be defined. The same system could also be used for messaging between musicians, e.g. instrument remote control, timing, etc.

[cdmahoney]

OK, but on the server side MIDI would be complementary to MQTT - MQTT would be required for automating the server from a web application, for example.

[pljones]

This is why separating the Control layer from the Service layer is needed as a first step before any further development, really.

[cdmahoney]

I've added the code for publishing/subscribing to an mqtt broker to branch jamulus-mqtt on my fork of jamulus:
https://github.com/cdmahoney/jamulus/tree/jamulus-mqtt/

This code is intended for testing and as 'proof of principal', it's not ready for prime time yet, but does demonstrate how you can have your jamulus server send and receive messaged over mqtt.

The new code has some additional dependencies which need to be installed to get it to compile.

qtmqtt library

The GPL version of qtmqtt needs to be downloaded, compiled, and installed. On ubuntu running QT v5.12.8 the corresponding version of qtmqtt is v5.15.10. I cloned the mqtt code into a folder in parallel with my jamulus project directory:

git clone https://github.com/qt/qtmqtt qtmqtt-github
cd qtmqtt-github/
git checkout v5.12.10

# You may also need to install dependency qtbase5-private-dev
sudo apt-get install qtbase5-private-dev

qmake qtmqtt.pro
make clean
make
# Install compiled library to system
sudo make install

After installing qtmqtt you should be able to compile the project from branch jamulus-mqtt.

New command line arguments

mqtt code adds two new command line arguments:

1. --mqtthostpub: Host of mqtt broker to connect to for publishing messages. If not defined mqtt code is not executed and overhead on server should be minimal. As well as for publishing messages the host is also used for subscribing to the jamulus/ping topic.
2. --mqttportpub: Port of mqtt broker for publishing messages. Defaults to 1883.

When code is added to enable activation of recording or other functionality on the server my idea is to add a second pair of arguments so that the "read-only" functionality can be enabled without having to enable state changing functionality.

So to connect to a local mqtt broker on the default port you can start jamulus as:

Jamulus -s -p 22124 ... --mqtthostpub "localhost" --mqttportpub 1883

Jamulus will publish messages using topics starting jamulus/port_number/, except for the topic jamulus/pingack used to respond to incoming jamulus/ping messages. The content of the messages published by jamulus is in JSON format

Testing mqtt messaging with mosquitto mqtt broker

Eclipse mosquitto (https://mosquitto.org/) provides an open source implementation of an mqtt broker and tools and is available for Linux, Windows, and Mac.

To install the latest version of mosquitto (v2.0.10) add the mosquitto-dev PPA to your repositories

sudo apt-add-repository ppa:mosquitto-dev/mosquitto-ppa
sudo apt-get update
sudo apt-install mosquitto

# Start mosquitto with default settings
mosquitto

With mosquitto running you can start Jamulus and it will connect to the broker - you can check the mosquitto logs to confirm that jamulus has connected. On my installation:

sudo cat  /var/log/mosquitto/mosquitto.log

A more convenient way to test the codes is to connect another client to the mqtt broker and subscribe to the topic jamulus/# to receive notification of all messages published by jamulus. You can do this using the mosquitto_sub utility, which has to be installed separately from mosquitto:

sudo apt install mosquitto-clients
mosquitto_sub -d -t jamulus/#

'#' is a wildcard that subscribes to all topics beginning with jamulus/. If you connect a client to the jamulus server you should see messages appearing the in mosquitto_sub console.

The mosquitto clients tools also include mosquitto_pub, which is used to publish messages to a broker, indicating the topic to publish too and the content of the message. We can use mosquitto_pub in a separate terminal to publish messages requesting state information from jamulus:

mosquitto_pub -t jamulus/ping -m ""
mosquitto_pub -t jamulus/22124/request/configuration/get -m ""
mosquitto_pub -t jamulus/22124/request/connections/get -m ""

These messages should appear in the mosquito_sub terminal, along with the responses from jamulus.

I hope that's all clear, let me know of any problems encountered or if further information is required.

[ann0see]

Thanks! This sounds great! I'm not sure if I have time to test it (my ToDo list is currently full with non Jamulus stuff...), but once this is stable enough, you can open a draft PR to this repo to discuss the code.

[cdmahoney]

OK, I'll add the code to enable/disable recording, clean things up a bit, and give it a some testing on my server, then open up a PR.

[ann0see]

There’s also another approach: #1847 which might be interesting

cc: @cdmahoney

[cdmahoney]

OK, just added a reply on that thread!