Fail safety with invalid external resources

[azymohliad]

Hey! I'm working on external resources support for WatchMate companion app, and by loading invalid resources content during testing I managed to brick my PineTime (stuck in the boot loop). To bring it back to life I had to unseal it, flash InfiniTime 1.10, remove invalid resources, and then upgrade back to 1.11.

This was due to a bug in my early implementation (each file when loaded form the zip archive was prepended by the content of all previous files), but it could also happen with correct app but malicious or maybe corrupted resources files. So I'm wondering how resilient the firmware is / can be at handling invalid resources, and what safety checks are expected from the companion app.

1. Is it possible / reasonable for the firmware to be able to recover by itself from such situation (e.g. automatically remove problematic resource, or fallback to a default watch face that doesn't depend on external resources)?
2. If writing a single resource file failed mid-way, what's the best thing for the companion app to do?
   • Nothing (partially written file won't cause such boot loop)?
   • Remove partially written file?
   • Remove partially written file and all other resources already written from the same archive?
   • Or anything else?
3. What other safety checks would be reasonable for a companion app to perform before writing? Maybe limit locations that resources can be written to (/fonts, /images, etc)? Maybe check for maximum resource file size? etc

[JF002]

Hi @azymohliad, and thanks for your work on Watchmate!

I'm sorry about your bricked PineTime, and, to be honest, I have a hard time understanding how this could have happened. Could you maybe send such corrupted resource file to this conversation? Or point me to a buggy version of Watchmate that would allow me to reproduce the issue on my devkit?

Currently, we use 2 types of resources : pictures and fonts.
For pictures, LVGL will try to open the file. If the file does not exist, LVGL will simply ignore it, and display nothing where the picture would have been located. If the content of the picture is not fully uploaded, lvgl will simply display garbage for the missing parts.
Regarding fonts, lv_font_load() returns null if it couldn't load the font. In this case, we have to check the return value and fallback to a built-in font.

Is it possible / reasonable for the firmware to be able to recover by itself from such situation (e.g. automatically remove problematic resource, or fallback to a default watch face that doesn't depend on external resources)?

Yes, InfiniTime should definitely be resilient to invalid/corrupted resources, especially if we want to allow more customizations by the users.

If writing a single resource file failed mid-way, what's the best thing for the companion app to do?

I guess the bare minimum would be to notify the user that something wrong happened and that they should try again. As I said earlier, we should ensure that invalid/corrupt files do not prevent infinitime from working properly.
Of course, the companion app could remove the failed file, try again, ask the user what they want to do (erase all new files, try again,...).

What other safety checks would be reasonable for a companion app to perform before writing? Maybe limit locations that resources can be written to (/fonts, /images, etc)? Maybe check for maximum resource file size? etc

That's an interesting question... We currently haven't fixed any limitation, the FS is wide open, and the companion app can access to everything. We (InfiniTime maintainers) take care to publish a valid resource pack (.json file + binary files), but nothing prevent a user from crafting an invalid resource pack, indeed. That's probably something we need to talk about. We've already talked about this topic in this review, but took no decision so far.

[azymohliad]

Thanks! That's the best answer I could hope for! So the companion app doesn't really have to worry about interrupted transaction, besides communicating it to the user. It would be pretty annoying to clean up failed file reliably (handle SIGKILL and SIGTERM, override window close action, etc), and even then there's hard reset.

No worries about me bricking my PineTime. I'm sorry if my wording caused you to be sorry. I was actually prepared for that the moment I started working on WatchMate, and I'm very pleasantly surprised it happened just now for the first time. I furiously crash tested it with the firmware upgrade, and it's been rock solid!

Here's the resource archive that should reproduce the issue. I hand-crafted it, but the content written by the buggy WatchMate version should be the same (it appended every next file to a buffer containing all previous files, and then flashed the whole buffer to the PineTime, so that the first file would be correct, the second file would be a concatenation of the first and the second file, and the last file would be a concatenation of all files).

Steps to reproduce:

1. Flash corrupted resources
2. Select Infineat watch face.

I used InfiniTime 1.11.0 release. Let me know if it doesn't reproduce it (I'll try again myself), or if I can help to debug it however else.

Btw, would it be more convenient if I moved it to an issue report?

And just in case, don't feel pressured to debug it. My questions are answered, my work on WatchMate isn't blocked. Thank you!

[JF002]

Thanks for those additional information! I definitely reproduce this issue on my setup thanks to your hand-crafted resource package! I confirm that this is caused by a bug in InfiniTime. I'll follow up in the issue you've just created (#1568)

[Riksu9000]

This is definitely something worth opening an issue about.

[azymohliad]

Ok, I'll do it later today or tomorrow. I see there's "Create issue from discussion" button, and I assume it will move the whole conversation there, so I'll better create it manually to make the description concise and relevant.

[azymohliad]

Reported #1568