Idea/Question: realtime data, MQTT only or HTTP/MQTT combination?

[rduivenvoorde]

We are discussing about creating a native QGIS data provider for STA, see
qgis/QGIS-Enhancement-Proposals#257

One use-case that I'm personally interested in is retrieving realtime data/status from sensors, best suited by the MQTT interface of FROST.

I'm not sure if that should/could be part of the QGIS http-STA-provider, or that it should be a totally separate data provider (as MQTT/message bus is a very general principle off course).

But my question here would be: should a MQTT -data absorbing client, be MQTT only? Or should it first use the HTTP api to retrieve a starting state?

More precise: when you want to show realtime results on a Map, you off course need coordinates (Locations) first...
So: should these come from the MQTT messages (meaning that only topic+result in a message is not enough, there should be geometry info in it then too)? <= mmm, this would fit for mobile/driving sensors too...
OR would the following scenario be better: first retrieve the locations as features (having the topics/id's info in it's attributes), and THEN retrieve the results (to be mapped to locations then)...

OR would be need both, depending on the moving/static use case...

Any input appreciated.

[hylkevds]

In almost[1] all cases a generic MQTT plugin for QGIS will need some non-MQTT source for the metadata that explains which topic to subscribe to and how to decode the data received over MQTT. Even if the user directly provides the topics, the data coming over the MQTT messages is generally not geoJSON, and would require custom parsing.

In the case of the SensorThings API which data comes over HTTP and which over MQTT depends on the use case:

• For non-moving sensors, the Location and initial Observation (result) comes over HTTP. New Observations are then received over MQTT, using a subscription on v1.1/Datastreams(xyz)/Observations.
• For moving sensors, the initial Location and initial Observation comes over HTTP, while updates to the Location come from the topic v1.1/Things(abc)/Locations and updates to the Observations from v1.1/Datastreams(xyz)/Observations.

There are other scenarios, but these two cover most use cases. Unfortunately $expand is not allowed in MQTT topics, so it is not possible to receive Observations with their FeatureOfInterest included.

Going back to the generic QGIS plugin architecture, the nicest implementation would be a generic MQTT plugin that can be configured/controlled from specific API plugins. That way the STA plugin doesn't need to re-implement all MQTT handling, but can tell the MQTT plugin which topics to subscribe to, and how to handle the data received.

[1] Almost, since with MQTT5 it is possible to do request/response, but I don't think many APIs fully use this functionality to provide full data access. This is planned for STA 2.0.