Ordered processing of entity registration messages before any other messages

[albinsuresh]

Is your feature improvement request related to a problem? Please describe.

Since MQTT protocol doesn't offer any guarantees about ordering of messages across different topics, even if a device sends a registration message to te/device/<device-id>// with an explicit @id value (the desired external ID) followed by a telemetry message to te/device/<device-id>///m/temperature topic, there is no guarantee that the mapper will receive and process the registration message first, before processing the telemetry message. If the telemetry message is processed first, the auto-registration mechanism would register that entity wrongly with the wrong external ID and other details(name and type if provided), and when the real device registration message is processed afterwards, it won't be able to undo/correct some of the damage that's already done like registering the entity in the cloud with the wrong auto-derived external ID, instead of the one provided with @id.

Describe the solution you'd like

1. A mechanism to turn-off auto-registration mechanism, when someone decides to use explicit registration. Having auto-registration turned ON while using explicit registration is forbidden as that'll lead to unnecessary race conditions.
2. When auto-registration is turned OFF, introduce an ordering mechanism in the mapper that'll wait for an entity registration message, even if it receives the telemetry message first.

Describe alternatives you've considered

An alternative thought was to have separate message channels for "entity registration messages" and "other messages" and to always give higher priority to the entity registration message channel, while fetching the next available MQTT message, with the hope of always reading the registration message first, before the other message.

But, this isn't sufficient as the priority mechanism will work only when the both the registration message and the other message are delivered to the MQTT actor, by the broker, so that they're both available in their own respective message channels when we try to read the next message. But, if the broker delivers only the telemetry message and delays the delivery of the registration message, even this "priority read" mechanism will fail, as there is only one message in the "other message channel" and nothing in the entity registration message channel and hence the only available message will be read anyway.
 
If we want to keep both auto-registration and explicit registration turned ON together(which would be ideal, if possible), then we can fall back to auto-registration after waiting for a fixed amount of time for the registration message. But, this solution is highly sensitive to message delivery time and hence not very predictable/deterministic. So, we might still end up with undesired entity metadata whenever there's an unexpected delay in message delivery(when the system is under high load).

[albinsuresh]

Test plan

This issue was addressed by two different PRs: #2466 and #2522, which were validated with independent test plans. So, this issue can be closed once both those PRs and validated.

[gligorisaev]

QA has thoroughly checked the feature and here are the results:

• Test for ticket exists in the test suite.
• tests/RobotFramework/tests/cumulocity/registration/device_registration.robot
• QA has tested the function and it's functioning according description.