ESP32: Can't Update to 13.x.x due To Flash Size

[AlexKusnezov]

Hey guys, I can't update my ESP32, everytime I try I get

Program flash size is larger than real flash size

Here's chip info:

ESP Chip Id | 401960 (ESP32-D0WDQ6 rev.1)
-- | --
0x1640D8 (DOUT)
4096 KB
4096 KB
1525 KB
1600 KB
147.4 KB (frag. 18%)
 
1600 KB
1600 KB (used 95%)
60 KB

FYI:
I also have this one, which also wasn't updating. But this one I managed to update by using tasmota32-bluetooth. The other one above didn't work with this workaround.

ESP Chip Id | 13778000 (ESP32-D0WD-V3 rev.3)
-- | --
0x16405E (DOUT)
4096 KB
4096 KB
1647 KB
1792 KB
81.9 KB (frag. 6%)
 
1792 KB (used 91%)
1792 KB
60 KB

Any idea how to update?

[sfromis]

You did neither tell what specific Tasmota binary you're already on, nor what exactly you do to update (including what binary URL/file), nor what your device is, nor whether you have the device in safeboot partitioning scheme.

[AlexKusnezov]

I'm on tasmota32.bin (12.5.0)
I always just downloaded the tsamota32.bin and udated my device manually, which is not working since 12.5.0.
Binary is http://ota.tasmota.com/tasmota32/release/tasmota32.bin Device is ESP32-DevkitC (NodeMCU)
I included chip specs above.

whether you have the device in safeboot partitioning scheme.

Do you inow where I can see that? Update worked always flawlessly until now! (13.0.0)

[sfromis]

With safeboot (which you likely do not have), the information page ends with settings like this:

With Tasmota 13.0, support for Matter is now included by default, which means that the Tasmota binaries now exceed the size of the two equal-sized partitions from before the safeboot partitioning scheme. Thus you need to first upgrade your setup to the safeboot layout. You can do this by running the Partition Wizard tool, without reinstalling from scratch.

If you instead do a fresh install of Tasmota, using the recommended Tasmota Web Installer tool, you'll also get a setup using the safeboot partitioning. That works by installing tasmota32.factory.bin at offset 0. If you do this, any existing Tasmota install is overwritten.

[AlexKusnezov]

cool, if I run the partition wizard will the config be wiped out/ can I restore it afterwards?

[sfromis]

Config will not be wiped if you use the Partition Wizard, that is only if you reinstall from scratch. Better have backup of anything you have in the file system.

[AlexKusnezov]

thanks, that's exactly waht I did and it worked right away!

[grahamsingle]

I've have 12.5.0 and when I try to upload "Partition Wizard.tapp" from "Manage File System" nothing happens. The file doesn't appear in the file system after I choose the file press "Start Upload".

I get the following from info:
Partition app0 1856 KB
Partition app1* 1856 KB (used 82%)
Partition fs 320 KB

Maybe there's not enough room for the Wizard, but no error is displayed and the file files are on the file system as before. Does the * suggest I'm running from app1?

Any suggestions?

[ArtjomE]

I had exactly the same issue with Shelly Plus 1PM (ESP32-U4WDH-S rev.3) "Program flash size is larger than real flash size" by upgrade from 12.5.0 to 13.0.0

Program Version 12.5.0(solo1)single-core
Build Date & Time 2023-04-17T08:07:06
Core/SDK Version | 2_0_7/4.4.4.20230218

I did a firmware upgrade with tasmota32solo1-safeboot.bin and now I have although new version 13.0.0 but just in SAFEBOOT

Program Version 13.0.0(safeboot)single-core
Build Date & Time 2023-06-26T10:42:56
Core/SDK Version 2_0_10/4.4.5.230614

and the problem with the firmware update with the tasmota32solo1.bin has continued and show me the same message as before "Program flash size is larger than real flash size ".

by upload the tasmota32solo1.factory.bin i got the message "Upload Failed" "Invalid file signature"

[sfromis]

To upgrade to version 13 you needed to do the repartitioning I already suggested. Not sure why you decided to try updating to the safeboot binary, instead of following the instructions, but this just made your situation worse, as the safeboot binary is not suitable for running the device.

You got yourself into some "additional trouble", but if you want to try to salvage it without reinstalling, you could try the command Restart 3 to get to the other regular partition, and then use the Partition Wizard to change partition scheme. Not 100% sure if this will work, but I think that the chances are not too bad. You may want to consider the risk of time spent on repairs being longer than reinstalling.

[ArtjomE]

this is just what happens when you try everything first and then google for solutions 🙈

the command Restart 3 does not work in console MQT: stat/TasmotaCarportOutside/RESULT = {"Command":"Error"}

how can I reinstall it, I do not care about the data ?

[sfromis]

Reinstall can be done via the externally available (no need for removing the cover) pins shown here:
https://templates.blakadder.com/shelly_plus_1PM.html

[ArtjomE]

is there no way to do this via OTA ?
ota with http://ota.tasmota.com/tasmota32/shelly/mgos32-to-tasmota32-Plus1PM.zip does not work

PS:
i got my shelly back to life with flash version 12.5.0 of file tasmota32solo1.bin

[sfromis]

Well, OTA is the normal way of upgrading, but when you screwed it up, any repair attempt after that becomes somewhat experimental, meaning that I'm not saying what will not work, or what will work. And experimental repair can be time-consuming, especially if trying to guide someone over the Internet, while wired flashing is a quick and easy process, when you have the hardware setup for it, and use the Tasmota Web Installer.

Anyway, great if you got it to work.

[AlexKusnezov]

this is just what happens when you try everything first and then google for solutions 🙈

the command Restart 3 does not work in console MQT: stat/TasmotaCarportOutside/RESULT = {"Command":"Error"}

how can I reinstall it, I do not care about the data ?

well, the thing is. What the most of users do is just hit the firmware update and expect it to work like it has woked the last 10 times before. In the changelog I couldn't see anything like "you should convert your esp32's partition with partition wizard in order to be able to update to v13"
Maybe I'm just dumb but all I've red was

ESP32 Tasmota SafeBoot with changed partition scheme allowing larger binaries

and thought, yeah cool so what?

Imho such guides, like usage of the partition wizard or at least to mention it should be included in the chnagelog.

Also thanks to @sfromis who pointed us, ignorant update button pushers, into the right direction

[sfromis]

ESP32 Binaries size increase of 300k due to Matter support may need Partition Wizard

Arguably, it could be called a "breaking" change to some users, but lots of users already on the safeboot partitioning scheme are not affected.

[AlexKusnezov]

@arendst really? I'm sorry, but I didn't noticed any Chnaged section under 13.0.0 only Added, I'm looking here: https://github.com/arendst/Tasmota/releases

Can you please point me to the right entry?

[arendst]

How about this https://github.com/arendst/Tasmota/releases/tag/v13.0.0

[AlexKusnezov]

that's perfect! Thanks you so much, never thought that descriptions from tags differ from releases.

[meyergru]

Since I fell for the same mistake, I can only say that this could have stood out a little more aggressively like under "breaking changes". There are devices that cannot be easily re-flashed via serial without relocating them first.

You remember the Monty Python clip: https://www.youtube.com/watch?v=1oyCXhiwulk ? "It should bear a large red label "Warning: lark's vomit!'"

[blacknell]

I found the upgrade to 13.0 from 12.4 wouldn't work even with safeboot 12.1 already installed. I think a more recent version of 12.x safeboot might be needed?

I found these in the logs

Jul  3 10:37:39 tasmota-BAE078-0120 ESP-Project tasmota - temphumid_loft Version 12.1.0(safeboot)-2_0_4_1(2022-08-18T10:33:31)
Jul  3 10:37:39 tasmota-BAE078-0120 ESP-WARNING This version does not support persistent settings

I executed Upgrade 2 from normal boot mode which updated the boot partition to 13.0 and then Upgrade 1 worked as normal.

I couldnt see anything that defined what is the oldest version of safeboot need for this to work

[arendst]

Yes. If safeboot fails to start the upgrade it's likely due to an old version (<12.4). Later versions use a different way of signalling an upgrade.

Your solution is the correct one.

[sfromis]

I've gotten into the habit of occasionally upgrading safeboot to current version, indeed using Upgrade 2. If wanting to switch out of safeboot (or to safeboot), the command Restart 3 can be useful, instead of doing some upgrade just to switch partitions.

[Noschvie]

How can the version of the safeboot partition be checked? Thanks

[sfromis]

By using the restart command I just described in the previous post here. That allows you to switch partition, and see what you have, without performing an upgrade. Arguably, having reason to check is also having reason to just upgrade 😁

[ReelHiJacker]

I've got several Shelly Plus 1 with 12.5.0. installed. I like to upgrade them, which doesn't work like you mention above. I'v tried the Partion Wizard Solution but i'v got this error in console after upload the file and restart:

TAP: Loaded Tasmota App 'Partition_Wizard.tapp'
BRY: failed to run compiled code 'io_error' - inconsistent number of builtin objects.

Partition Manager works, but not the Wizard... now i'm stuck... can you help me?
Tried it on different Shelly Plus 1 ... everytime the same error...

[ReelHiJacker]

This works... Thank you! :-)

[sfromis]

Looks like Partition Wizard needs another fix, @s-hadinger

[s-hadinger]

I need to fix some stuff. Please be patient

[s-hadinger]

This is now fixed. Please use the latest from the repo:
https://github.com/arendst/Tasmota/blob/development/tasmota/berry/modules/Partition_Wizard.tapp

[cschoell]

I get a "UPL: error (network_error) GET returned 302" whenever I try to use the Partition Wizard - (it looks like the safeboot url is redirected from http to https)
... turns out I blocked the tasmota device from internet access via my router - it's working fine.

[foobar26]

Hi,
I wanted to upgrade my Shelly Plus 2PM from 12.5.0 to 13 via the Partition manager approach.
The first steps went fine, it seems.
But now I'm stuck in Safeboot mode.
I tried to switch witch Restart 3, but still end up in Safeboot mode.
Restart 1 same thing -> safeboot mode.
I also tried to upgrade FW, same result.
Information page shows at the end:

Partition safeboot*	832 KB (used 94%)
Partition app0	2688 KB (used 69%)
Partition fs	448 KB

I'm not sure why the device doesn't want to switch to the app0 partition. Any way to force it?

[sfromis]

Which binary did you try to upgrade to? If not tasmota32solo1.bin, try doing that.

[foobar26]

tried that. again ending up in safeboot

[foobar26]

used also upgrade 1 or 2. no difference.

[foobar26]

I ended up flashing the device via serial :-\

[fir3drag0n]

Unfortunately the problem still exists

[Jason2866]

@foobar26 The reason for the Shelly failing to convert successfully was an updated more secure Shelly firmware. It has nothing to do with the Safeboot convert in general. Safeboot convert works well.
The actual Shelly firmwares uses a locked bootloader. When this bootloader is not replaced the device gets stuck on the firmware which is at address 0x10000. No way out. Just serial flashing can solve.
Since 19.Sept 2023 we have an updated Shelly Convert to Tasmota online.
Shelly converting is a cat and mice game. Let's see how long it will last this time.

[foobar26]

@Jason2866 thx for your explanation and adding the extra warning to the migration process for mgos32-to-tasmota32!

[Jason2866]

@peepshow-21 Wrong Berry App. You have Partition Manager running. For converting PartitionWizard is needed.

[peepshow-21]

sorry, realised that, and deleted my silly question before i saw your reply!! :)
thanks you

[oywino]

Maybe not exactly the same problem - but very similar;
I have many SonOff switches and bridges, all flashed with Tasmota 13.2.0 but this seems to be the end of it ?
Going forward I no longer can flash even the Tasmota-minimal.bin as there just isn't enough memory!
So what then?
Are we stuck at 13.2.0 ?

[arendst]

Totally offtopic. This is about ESP32 based evices.

Yours are ESP8266 based and there is no issue upgrading these to latest versions other than bad power supplies not able to support latest features

[sfromis]

For ESP8266, as indicated on https://ota.tasmota.com/tasmota/release/ the .bin files are only "initial firmware" for wired flashing, while upgrading should always be with the .bin.gz files.

When doing upgrades OTA, do not download first, and do not use -minimal.bin.gz as this will be automatically handled by the upgrade process.

[oywino]

OMG - Thank you so much for that information. I've been using Tasmotized devices for years, and not once have I been able to successfully complete an OTA upgrade. So I guess I (wrongly) concluded long ago that OTA just doesn't work.
Based on your reply above, I now decided to give it another go - since my year-long faithful manual file based method had come to a halt. In fact, I didn't have much choice.
It's a nerve wrecking experience. I first checked the URL and then closed my eyes and pressed the button. Not much feedback indicating what was going on or if anything at all, but after a while it came back online with the minimal FW and requested another upgrade. So far, so good.
Then pressing upgrade again - nothing happens, waiting--- and it was gone 404. Ok that's it I thought. another bricked device. I went for a cup of coffee. And guess what! Upon my return som 15 minutes later, it had miraculously recovered all by itself and now with Tasmota 13.3.0 Joy to the people! 😄❤️👍
Thank you so much.
But two questions remain in my mind:

1. How come I haven't found that vital bit of information anywhere before;
   When doing upgrades OTA, do not download first, and do not use -minimal.bin.gz as this will be automatically handled by the upgrade process.
   Honestly - it isn't very obvious that the minimal.bin.gz has been provided so we should have to use it - eh? I guess it serves some other mysterious purpose beyond being used.

2. I wish the upgrade process would provide some sort of visual on screen progress feedback. I almost unplugged the device thinking it was bricked. Thank God for the coffee break 😊

[Jason2866]

Point 1) Documented here https://tasmota.github.io/docs/Upgrading/
Point 2) Not possible during upgrade

[forestial]

I'm looking for the correct upgrade path for a SwitchBot device that I have. This uses an ESP32-C3. I have been able to get Tasmota 11.1.0 on it using OTA and the 'switchbota' tool, but it will not upgrade past that; I presume because of the partition change. The case of the device is 'welded' and I would really like to avoid cutting it open, so I am looking for a way to do this OTA.

From reading the replies above I think I need to

1. Load the Partition Manager TAPP file
2. Power cycle the device.
   .... I'm not clear what I would do next. Do I need to issue one of the 'restart' commands? Will tasmota still be functional at this point so that I can access its web page interface? Can I just attempt OTA upgrade again, and will it now install the latest v13.x?

[Jason2866]

This one should be small enough https://github.com/Jason2866/Tasmota-specials/releases/download/2024-01-30.d.95b03592dc0b753dfcfa30c1cc17d8e1f2a547f9/tasmota32c3_2M.bin

[barbudor]

AH thanks @Jason2866

[barbudor]

@forestial So upgrade to the one pointed by Jason
This will allow you to run the PartitionWizard (you may need to re-upload it)
Once you have migrated to the new safeboot structure, you can then upgrade to the standard tasmota

[forestial]

Thanks guys! I used Jason's '2M' version, then ran the Partition Wizard, then upgraded to the regular tasmota32.bin. (Had to repeat that a couple of times, for some reason it stayed with the safeboot initially). Now I see

[barbudor]

Congrats