Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid resending all entity registration messages on mapper restart #2314

Closed
albinsuresh opened this issue Oct 5, 2023 · 1 comment
Closed
Labels
improvement User value

Comments

@albinsuresh
Copy link
Contributor

Is your feature improvement request related to a problem? Please describe.

Since entity registration messages are retained messages on the MQTT broker, every time the broker is started, these messages are re-delivered to the mapper and the mapper will convert those to C8y registration messages and send them again to the cloud, even if they were sent earlier. When there are a large number of entities registered, this leads to unnecessarily resending a lot of messages to the cloud on every mapper startup. Even though duplicate registration messages are ignored by the cloud, it would be better to avoid resending them unnecessarily.

Describe the solution you'd like

When the mapper receives entity registration messages, persist that information on the file system as well, so that on restart, the mapper can reload the previously known state from the file system, compare it against the retained messages that are delivered and only send the differences to the cloud (any new registration messages that may have arrived while the mapper was down).

Describe alternatives you've considered

Whenever the mapper processes a registration message, replace the original registration message on the broker with a copy of it with an additional "@c8y_mapped": "true" flag added to it. When these messages are re-delivered on mapper restart, the ones with this flag can be ignored as they are already processed.

But this approach is more intrusive in the sense that every component subscribed to entity registration messages will get these duplicate messages as well with this additional, which they may not know what to do with. They can just ignore it, but that update itself was unnecessary from their POV.

@albinsuresh
Copy link
Contributor Author

This feature was implemented in #2522 and can be enabled by setting the c8y.entity_store.clean_start config setting to false.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement User value
Projects
None yet
Development

No branches or pull requests

1 participant