write_flash --erase-all option corrupting flash (ESPTOOL-562)

[git-n-pissed]

Operating System

Windows 10, WSL 1.0 (Ubuntu 22.04.1 LTS)

Esptool Version

4.4

Python Version

3.10.6

Chip Description

ESP32-S3-WROOM

Device Description

ESP32-S3-DevKitC-1

Hardware Configuration

No response

How is Esptool Run

bash

Full Esptool Command Line that Was Run

python3 -m esptool --baud 921600 --chip esp32s3 --port /dev/ttyS9 write_flash --erase-all --verify 0 /mnt/c/Users/git-n-pissed/Desktop/GENERIC_S3-20220618-v1.19.1.bin

Esptool Output

esptool.py v4.4
Serial port /dev/ttyS9
Connecting....
Chip is ESP32-S3 (revision v0.1)
Features: WiFi, BLE
Crystal is 40MHz
MAC: BE:EF:DE:AD:BE:EF
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 921600
Changed.
Configuring flash size...
Erasing flash (this may take a while)...
Chip erase completed successfully in 20.4s
Compressed 1387904 bytes to 916115...
Wrote 1387904 bytes (916115 compressed) at 0x00000000 in 19.8 seconds (effective 560.5 kbit/s)...
Hash of data verified.

Leaving...
Verifying just-written flash...
(This option is deprecated, flash contents are now always read back after flashing.)
Verifying 0x157670 (1406576) bytes @ 0x00000000 in flash against /mnt/c/Users/git-n-pissed/Desktop/GENERIC_S3-20220618-v1.19.1.bin...
-- verify FAILED (digest mismatch)

More Information

esptool v2.6, the write_flash --erase-all option worked as expected, erasing the flash before writing, all in one command. I just upgraded to esptool v4.4 and this option now causes the flash to be corrupted on write without any indication of failure until you REPL in. What's worse, if you also use the --verify option, you are told the --verify option is deprecated and that flash contents are always read back after flashing, but verification goes ahead anyway, and eventually fails. I'm seeing the same behavior on ESP8266, ESP32-WROOM32, and ESP32-S3-WROOM32, using various USB chips (FT232RQ, CH340, CP2102W) all using a stock downloaded build of MicroPython (v1.19.1).

Other Steps to Reproduce

No response

I Have Read the Troubleshooting Guide

• I confirm I have read the troubleshooting guide.

[radimkarnis]

Hello @git-n-pissed,
thanks a lot for reporting this!

I can confirm the --erase-all arg is indeed broken - I traced the regression to be introduced in commit in v4.3. Every release up to v4.2.1 works as intended.

I will fix this and add tests so this never goes under the radar again.

In the meantime, I advise everyone to run the write_flash and erase_flash commands separately (or use v4.2).

[git-n-pissed]

I can confirm the --erase-all arg is indeed broken - I traced the regression to be introduced in commit in v4.3. Every release up to v4.2.1 works as intended.

Was the behavior of the --verify option affected in the same commit, or should I make a separate issue for that?

[radimkarnis]

It's actually more complicated, but the --verify option behaved correctly - that's why when verify fails, you can also identify a failure when you REPL in. No need to open a separate issue.

The fix is in the works - you can try and verify it. Just add this code under this continue line:

            ...
            except (FatalError, struct.error, RuntimeError):
                continue
            finally:           # add this
                argfile.seek(0)  # add this
            if image.chip_id != esp.IMAGE_CHIP_ID:
            ...