Database corruption caused by power loss

[marcinstrzelczyk]

Hello,
I have mosquitto v2.0.18 running on Windows 10 as a service. Few days ago I experienced sudden power loss on my computer and then mosquitto failed to start with

Error: Unable to restore persistent database. Unrecognised file format.
Error: Couldn't open database.

errors in log file. Most likely mosquitto was trying to update the database file when the power loss happened, but I'm not 100% sure.

The mosquitto.db file exists and contains only NULLs. There is no mosquitto.db.new file, which I saw being mentioned in other issues.

This issue is similar to #189, I looked at PR for that issue and the solution was to add fflush(db_fptr); before closing the file (https://github.com/eclipse-mosquitto/mosquitto/pull/206/files), but it was done inside #ifndef WIN32 directive, so it was not applied to Windows version. I don't know if that fflush call could fix problems on Windows as well, but maybe it's worth looking at.

[sitaalbeMarp]

Hello, I've experienced similar behaviour also with version 2.0.20.

[bartoszsurowiec]

@marcinstrzelczyk did you find a solution/fix?

[marcinstrzelczyk]

@bartoszsurowiec nope

[NorbertHeusser]

One of the general problems in windows is the absence of a real atomic and posix compliant file rename.
The fflush might reduce the risk to loose data, but we would need to introduce an additional fsync (FlushFileBuffers as well. And this would introduce a massiv delay.

In general Mosquitto broker has introduced an extension of the plugin API on the development branch to allow persistence based on a plugin. And added a sqlite based persistence plugin. With the next major release it will be possible to replace the snapshot based blocking persistence by a more continuous persistence making use of a SQLite DB.

[bartoszsurowiec]

When is the next major release planned?