New Update API

[jomjol]

@caco3: The newest rolling has a new REST API to make the update easier. User interface (html.zip) not updated yet.

Description of the interface:
The update of the firmware.bin, html.zip, any.tflite is done with only one API.

The following procedure is planned:

1. Upload the new file to the device via the current upload to the directory /firmware
   Make sure, that you have delete old files before actively!
   This can be:

• new firmware (ANY.bin)
• new user interface (html.zip)
• both (firmware + user interface in one zip file (e.g. update.zip)

2. Activate the update via the following API: /ota?task=update&file=FILENAME
   Where FILENAME is the name of the previously uploaded file

According to the filetype (bin, zip) the algo does the following

1. if it is a bin, it starts the firmware flashing
2. if it is tflite it is copied to /config
3. if it is a zip, it unzips all files to /html with one exeption: filetype *.bin this is extracted to /firmware and is used in the same step to initiate a firmware flashing directly after extracting the complete zip file.

With this mechanism we an make the OTA interface much easier.
What do you think of this? Should we update the ota-page according to this new API?

[caco3]

That sounds great.

Some thoughts:

• To point 1: What happens, if you upload the partitions.bin or bootloader.bin? Is there a validation mechanism? I fear a wrong file would brick the device!
• To point 1: You didn't list the tfl/tflite files, but I expect they also would be supported.
• To point 2.3: I would suggest that the update.zip contains a firmware.bin and a html folder with all WebUI files.

[jomjol]

• Good point with partitions.bin and bootloader.bin. If we agree, that we will name the firmware always firmware.bin I can limit the update to this and there should be no problem.
• tfl/tlite is considered, but not inside the update.zip - can be adopted, to upload new neural network files in one rush. But what I don't recommend is to update them in the config.ini automatically.
• I don't think we need a substructure in the update.zip because neither firmware.bin nor *.tfl/tflite should ever be part of the html directory. Therefore keep it simple.

[caco3]

• Good point with partitions.bin and bootloader.bin. If we agree, that we will name the firmware always firmware.bin I can limit the update to this and there should be no problem.

Yes, that sound good. One would need to enforce bricking on purpose.

* tfl/tlite is considered, but not inside the `update.zip` - can be adopted, to upload new neural network files in one rush. But what I don't recommend is to update them in the `config.ini` automatically.

What I wanted to day that you mist to list it as a supported file type. I also would not integrate it into the update.zip file.

* I don't think we need a substructure in the `update.zip` because neither `firmware.bin` nor `*.tfl/tflite` should ever be part of the html directory. Therefore keep it simple.

Its easier to validate and understand the zip file if there is a structure. Or is it extra work on implementation side?

[jomjol]

Done - even more: if we make a subdirectory in the zip file it is copied to the same on the sd-card. With this we can also update the /config if needed.
Only one restriction: flashing is only triggered automatically if the firmware.bin is in the root of the zip file.

[caco3]

I will update the build pipeline tomorrow to match your changes.

Need to go to bed now :)

[jomjol]

I suggest to create 4 files for the update

1. firmware.bin
2. html.zip
3. update.zip
   only containing firmware.bin (root directory) and html-files (in /html)
4. update_incl_cnn.zip
   being (3) + all tflite files in /config

(1) and (2) beeing the current update files to be compatible with the current update mechanism.
(3) beeing the new update way
(4) giving the option to upload the newest neural network files as well to the config, but still needed manual activation in the settings. As this might be a bigger file (some tflite are > 1 MB) and sometimes more of them are available, this might increase the update time significantly, therefore both versions.

Regarding the update mechanism itself, I suggest for the future to trigger the reboot automatically if necessary (firmware.bin).
I'm working on the new update page, but as I'm only limited available the next week, it will take some time.

@caco3: sprichst du Deutsch? Dann können wir gerne umstellen :-).

[caco3]

Hallo @jomjol Ja, Deutsch wäre auch ok. Manchmal ist Englisch aber auch einfacher :)

Können wir es so machen, dass nur der erste Teil des Dateinamens passen muss?
Ich würde gerne dahinter das Datum/Version anhängen, z.B. bei rolling: firmware__2022-09-10_21-54-50__add-update__(7c460f1).zip resp. (release) firmware_11.2.0.zip Das funktioniert jetzt schon so und wäre damit rückwärtskompatibel!
Ein Entwurf: https://github.com/caco3/AI-on-the-edge-device/actions/runs/3030944437

Die Frage ist, in den zip files (firmware__*.zip und update_*.zip), soll ich die Dateien darin ohne 2022-09-10_21-54-50__add-update__(7c460f1) machen oder auch mit? Ohne ist vermutlich einfacher, ausser bei der firmware, damit wir es nicht von hand umbenennen müssen.

Der Build-Prozess würde immer alle 4 zip files generieren, auch für die Firmware, da müsste der Anwender halt entpacken resp. wir machen das dann jeweils von Hand für das Release. (es wird immer gezipt, das kann ich nicht unterbinden). Du könntest aber auch firmware__*.zip unterstützen, falls es jemand vergisst.

Autoreboot finde ich auch gut.

Gerne kann ich den Umbau auf HTML-seite übernehmen, wenn Du möchtest.
Kannst Du mit JSON zurückgeben, was Du alles machen wirst (firmware update, reboot ja/nein) resp. wenn die Datei nicht untertstützt wird? Dann können wir das entsprechend im Web interface anzeigen.

[jomjol]

Der Name des Zip-File ist jetzt auch schon egal. Nur innerhalb des ZIP-Files muss die Datei dann firmware.bin heißen.
Mit dem neuen Algo müssen wir es dann auch nicht entpacken, da eine firmware.bin im Root-Verzeichnis einer ZIP erkannt wird.

P.S. bin jetzt bis Mittwoch nur eingeschränkt erreichbar, insbesondere ohne Testsystem