Added --elf-sha256-offset to ElfToBin cmd for Arduino framework

[paclema]

Bugfix for issue #972

[Jason2866]

Arduino IDE do not generate and use a SHA. Why should PlatformIO should behave here different?

[paclema]

I have found a way to integrate ESP-Insights ESP-IDF component into Arduino framework.

This component helps while monitoring firmware crashes or reporting variables and firmware metrics but in order to make it work, there are some things that need to be added into the binary, As it happens before in the #272 closed issue when the SHA256 hash was an important variable to perform the ESP OTA correctly, it is also important now to make this component work properly.

In this case, the ESP-Insights dashboard requires uploading a firmware package containing the same binary (among other things.. more info can be seen in this issue: espressif/esp-insights#14) running into the esp32 board, and the elf with the same hash stored in the binary.

The fix I propose in this PR: #973 is the solution that it was already added (for solving this #272 ) but removed afterward since it has been changed the way that the --elf-sha256-offset 0xb0 is passed to the elf2Bin command. This flag is now only added while building for IDF. I guess there was no need before while building for arduino-esp32 framework, but now this is the only way that esp-insights can be correctly configured.

You might also find that some other information like the project name and version (as appears also in the issue #272) need to be engraved into the firmware in order to make esp-insights works, and arduino-esp32 has a precompiled ESP-IDF version, containing the precompiled idf version as the version and arduino-esp32 as the project name I think. To solve that matter I have used this solution: How to engrave application name & version in esp32 partitions with arduino framework by @ankostis (which I guess that it could be also integrated in the future into arduino platform instead to stand in a separated script)

To summarize and test the ESP-Insights IDF component using the arduino-esp32 framework, I have created a ESP-Insights with arduino-esp32 framework example that has the next characteristics:

• Arduino-esp32 example code to connect the device to ESP-Insights platform.
• Those scripts to patch this project information, version, etc.
• The branch of the PR Added --elf-sha256-offset to ElfToBin cmd for Arduino framework #973 I propose as the platformio platform.
• Script to generate the zip package necessary to upload to ESP-Insights dashboard when building the firmware with platformio.
• Added my arduino-esp32 fork where I have added the ESP-Insights IDF component. It is built using the branch that I PR also to Include ESP-Insights component for esp32 target espressif/esp32-arduino-lib-builder#99

[paclema]

@Jason2866 Would you consider my feedback enough reason for adding this elf2Bin flag for arduino-esp32 framework too?

[Jason2866]

@paclema I am not a maintainer nor a member of Platformio.
Just did a few PRs here. I maintain a fork of platform-espressif32 for project Tasmota
Edit: Have you tried your change if uploading and debugging (OpenOCD) still works?
What happens when esptool.py is patching during flashing (since flash size detect a different flash size) is patching the magic header bytes?
That was the main reason for removing any SHAs when using Arduino. There is always the possibilty that firmware gets modified during flash. Arduino is here very different to the way it is done with IDF.
EDIT2: Your PR will brake the needed firmware patching with esptool.py Details here
This was extra added to make Arduino firmware work

[paclema]

@Jason2866 thanks a lot for your latest info! I have been testing the recent ESP-Insights now added into arduino-esp32 framework and also with the new way of generating the bootloader binary from the elf files.

I have enhanced the example that I mention in my previous post: ESP-Insights with arduino-esp32 framework example adding different board configurations and I have tested to debug it using an ESP-Prog (only debugging, uploading firmware using the board usb port) using this PR and using the default branch and I have the same successful results.

With this PR I can store the elf hash (mandatory to use ESP-Insights platform) and I can still debug and change the flash freq and mode with esptool

Could you also run the example that I provide? It also prints using the serial some of the flash configurations and the app hash, so you can verify that it works well with any board flash configuration and this PR. As example:

[     4][D][esp32-hal-cpu.c:244] setCpuFrequencyMhz(): PLL: 480 / 2 = 240 Mhz, APB: 80000000 Hz
[   451][I][esp32-hal-psram.c:96] psramInit(): PSRAM enabled
[   474][I][firmware_info.h:83] show_flash_info(): Chip out_id: 2113560
[   475][I][firmware_info.h:92] show_flash_info(): Chip unique out_id: 3676923104355062989
[   479][I][firmware_info.h:96] show_flash_info(): Flash mode: 00
[   486][I][firmware_info.h:97] show_flash_info(): CPU flash freq: 240 MHz, xTal: 40 MHz, APB: 80000000 Hz, Flash: 80000000 Hz
--+ Firmware info: +--
project_name: FirmwareNameTest
version: 1.2.3
time: 15:41:29+0100
date: 20 Dec 2022
idf_ver: v4.4.3
app_elf_sha256: 2ac5a7cbf44b14abf3254f3a7537a67e251bea1dd02b43040789c0a7cbc855df
[   664][I][firmware_info.h:25] print_sha256(): [shaELF] SHA-256 for current app running partition: : 9e971cffd23a51e3691d2f0ff24474e0c5abd4bfad2fca219d95d66a37f7ea07

On the other hand, I actually wonder now if adding this --elf-sha256-offset flag to ElfToBin cmd should be added here in this repo or into the platformio-build.py arduino-esp32 script... @me-no-dev what do you also know if adding this fix here would be better than adding it into arduino-esp32 build script?

[paclema]

Do you know what to do in this case @valeros? Thank you!

[Jason2866]

Yes, with the generating the bootloader out of the elf files, the problem with patching magic bytes needed with esptool.py is solved. I see no reason anymore for not implementing. Probably "cleaner" when done in Arduino code. Imho adding here is kind of patching.
The link for the example ends in 404

[paclema]

@Jason2866 Sorry the example was in a private repo but you can find it now here: ESP-Insights with arduino-esp32 framework example

[paclema]

This is already merged into arduino-esp32 on https://github.com/espressif/arduino-esp32/pull/7566/files#diff-c910a41dbc9d05e6c71b19a2d6f9723ce36a566cbfa761e1fa195a68f99bb82cR247, therefore, this PR is not longer necessary.