Important: Bootloop exception after uploading tasmota-minimal 14.2.0.4 (on 9 from 13 devices!)

[TPDomo]

PROBLEM DESCRIPTION

Yesterday i pushed 14.2.0.4 firmware using Tasmoadmin to all my devices simultaneously.
Full pushed update version info: pre-compiled 14.2.0.4(27611a1-minimal)-2_7_7(2024-09-18T17:05:46)

By-yesterday working version on all those devices was 14.1.0.1.

Tasmoadmin does mass update automatically - first it sends tasmota-minimal.bin.gz to all selected devices and next it sends selected full firmware .gz file. This is working for me for few years already, and had no issues with the process ever, so describing it just to give full picture (mass-update).

From 13 devices i tried to update, only 4 survived (2 step update finished fully).

On 9 devices minimal firmware was uploaded but full firmware didn't succeed.
So, started investigating and it occurs each of those devices is now having boot loop - it resets 1-3 seconds after connecting to my standard wifi network.

Devices hw impacted by this:
1 x Sonoff 4CH Pro rev2 (ESP8285N08)
1 x Sonoff T1 2CH (ESP8285N08)
1 x Sonoff Pow R2 (ESP8266EX)
5 x Shelly 2.5 (ESP8266EX)
1 x Shelly 1

Devices which "survived":
1 x Shelly 1PM
1 x Shelly 1
1 x Sonoff T1 2CH
1 x Sonoff T1 1CH

Been trying to send any command using Web Console but it never succeeded (tried on couple devices).

Tried also to click OTA from URL update using button from web interface or:
/cm?cmnd=Upgrade%201
but none of those worked also.

All devices configured similarly, with MQTT connection to Domoticz and HA.
Mostly simple setup, without special scripts, rules or other tweaks.

After each reboot console log usually ends after "wifi Connected" / "Web server active..."
Sometimes it manages to show MQTT connection also (rarely) - providing a screen below with one such catch.

Seem, i now have 9 in-wall/rack devices requiring disassembly and serial programming :(

Hope it will warn others and maybe save somebody from similar failure...!

REQUESTED INFORMATION

Make sure your have performed every step and checked the applicable boxes before submitting your issue. Thank you!

• Read the Contributing Guide and Policy and the Code of Conduct
• Searched the problem in issues
• Searched the problem in discussions
• Searched the problem in the docs
• Searched the problem in the chat
• Device used (e.g., Sonoff Basic): as listed above
• Tasmota binary firmware version number used: during upgrade from 14.1.0.1 to 14.2.0.4
  • Pre-compiled
  • Self-compiled
• Flashing tools used: Tasmoadmin (patched for support of .gz files)
• Provide the output of command: Backlog Template; Module; GPIO 255: not able to issue any command

  Configuration output here:

• If using rules, provide the output of this command: Backlog Rule1; Rule2; Rule3: not using rules, not able to list them

  Rules output here:

• Provide the output of this command: Status 0: not able to issue any command

  STATUS 0 output here:

• Set weblog to 4 and then, when you experience your issue, provide the output of the Console log: not able

  Console output here:

TO REPRODUCE

Steps to reproduce the behavior:
upgrade initiated, minimal firmware uploaded

EXPECTED BEHAVIOUR

A clear and concise description of what you expected to happen.
working minimal fw allowing to upload of full one

SCREENSHOTS

If applicable, add screenshots to help explain your problem.

ADDITIONAL CONTEXT

Add any other context about the problem here.

(Please, remember to close the issue when the problem has been addressed)

[sfromis]

IDK what happened in your case, but I'm having many devices upgraded to 14.2.0.4 without any trouble.

[Airey001]

So i can confim i just had this same think happen when I tried to update 4 devices. I always test in small groups before i do everything. So i did the the lights in my office. All of them where done with the WebUI. 3 of the 4 devices are just rebooting contantly. I can't get it it to do anyting not even commands in the console webpage. They updated to the Minimal and then started acting this way. I tried via the web in the time before it reboot to update to an older version of the minimal file but that errors out about the file being to big before restarting.

[Airey001]

here is what i see in the consle

00:00:00.002 HDW: ESP8285N08
00:00:00.047 CFG: Loaded from flash at FB, Count 4013
00:00:00.053 Project tasmota - SB-Ceiling-Bed-Bulb Version 14.2.0.4(27611a1-minimal)-2_7_7(2024-09-18T17:05:46)
00:00:00.054 WARNING This version does not support persistent settings
00:00:01.167 WIF: Connecting to AP1 Remus_IOT Channel 6 BSSId 02:92:BF:57:68:1A in mode 11n as tasmota-A64DC4-3524...
00:00:10.431 WIF: Connect failed as AP cannot be reached
00:00:11.438 WIF: Connecting to AP1 Remus_IOT in mode 11n as tasmota-A64DC4-3524...
00:00:23.772 WIF: Connect failed with AP timeout
00:00:25.780 WIF: Connecting to AP1 Remus_IOT in mode 11n as tasmota-A64DC4-3524...
00:00:31.753 WIF: Connected
00:00:32.004 HTP: Web server active on tasmota-A64DC4-3524 with IP address 172.16.3.122
00:00:38.702 MQT: Attempting connection...
00:00:38.728 MQT: Connected
00:00:38.731 MQT: tele/tasmota_A64DC4/LWT = Online (retained)
00:00:38.733 MQT: cmnd/tasmota_A64DC4/POWER =
00:00:38.738 MQT: tele/tasmota_A64DC4/INFO1 = {"Info1":{"Module":"FE-OM60-15K-AG","Version":"14.2.0.4(27611a1-minimal)","FallbackTopic":"cmnd/DVES_A64DC4_fb/","GroupTopic":"cmnd/tasmotas/"}}
00:00:38.749 MQT: tele/tasmota_A64DC4/INFO2 = {"Info2":{"WebServerMode":"Admin","Hostname":"tasmota-A64DC4-3524","IPAddress":"172.16.3.122"}}
00:00:38.763 MQT: tele/tasmota_A64DC4/INFO3 = {"Info3":{"RestartReason":"Exception","BootCount":198}}

here is pinging results

[TPDomo]

Hi again,
Replaced one of Shellys 2.5 with a new one so got malfunctioning on my table. Connected with serial and there is a looping result repeating every few seconds:

`

ets Jan  8 2013,rst cause:2, boot mode:(3,6)

load 0x4010f000, len 3584, room 16
tail 0
chksum 0xb0
csum 0xb0
v3969889e
~ld

00:00:00.001 HDW: ESP8266EX
00:00:00.047 CFG: Loaded from flash at FB, Count 25581
00:00:00.053 Project tasmota - Shelly25-2_02 Kxxxxxxxxy Version 14.2.0.4(27611a1-minimal)-2_7_7(2024-09-18T17:05:46)
00:00:00.054 WARNING This version does not support persistent settings
00:00:01.001 WIF: Connecting to AP1 Mxxxxxxxxo Channel 6 BSSId 6xxxxxxxxA in mode 11n as Dxxxxxxxxy-2266...
00:00:05.513 WIF: Connected
00:00:05.764 HTP: Web server active on Dxxxxxxxxy-2266 with IP address 1xxxxxxxx9

--------------- CUT HERE FOR EXCEPTION DECODER ---------------

Exception (3):
epc1=0x4000bf64 epc2=0x00000000 epc3=0x00000000 excvaddr=0x4025003f depc=0x00000000

>>>stack>>>

ctx: cont
sp: 3ffffa70 end: 3fffffc0 offset: 0190
3ffffc00:  00000000 4bc6a7f0 00001d6b 00000000
3ffffc10:  4025003f 000000d2 00000000 40206d25
3ffffc20:  00000017 000000d2 3ffeffc4 402022cf
3ffffc30:  00000000 00000001 00fffd91 3ffffcf8
3ffffc40:  00000000 3ffffd81 3ffeffc4 4020dc9e
3ffffc50:  00000000 3ffffd81 00000020 40100b1e
3ffffc60:  646e6d63 4556442f 36375f53 41443843
3ffffc70:  2f62665f 3fff0a00 00000000 40100b65
3ffffc80:  40217c74 3ffffd40 3ffffcbd 402177a1
3ffffc90:  00000032 0000000a 3ffffcbc 40217a79
3ffffca0:  00000001 3fff327c 00000000 00000000
3ffffcb0:  3fff327c 00000010 3ffffd9c 4021880e
3ffffcc0:  0000000b 00000002 00000009 40218d44
3ffffcd0:  00000009 00000000 00000020 3ffffde0
3ffffce0:  00002710 000004e2 000004e2 4010085f
3ffffcf0:  0000000d 00000003 4c620000 0000676f
3ffffd00:  00700067 00000000 00000000 00000000
3ffffd10:  00002430 00000486 00000486 4010085f
3ffffd20:  3ffffda8 4021c9c0 3fff0000 3ffffde0
3ffffd30:  00000001 3ffffe04 00000020 40100ae7
3ffffd40:  00000001 00000001 00000000 3ffe86b2
3ffffd50:  00000000 00000000 00000001 3ffffd96
3ffffd60:  00000001 3ffffe04 3ffffd88 3ffffda0
3ffffd70:  3ffffda0 3ffffd96 3ffffe0a 4020deec
3ffffd80:  00003020 3fff2e44 43626557 616d6d6f
3ffffd90:  4154532f 00535554 00000000 4020de2c
3ffffda0:  3ffffe04 0000005f 0000002f 3ffffd90
3ffffdb0:  3fff2e0c 3fff2e0c 00000008 3ffffde0
3ffffdc0:  00000001 3ffffe04 00000008 4020df24
3ffffdd0:  00000001 3ffeead0 0000000c 4020f5f0
3ffffde0:  646e6d00 3fff0000 00000020 40100ae7
3ffffdf0:  00000000 3fff10d8 3fff10b4 402130cd
3ffffe00:  00000006 74617453 30207375 08000000
3ffffe10:  3fff2be8 3ffffe60 00000000 40218f1c
3ffffe20:  74736f00 00000003 3ffffe60 402070c8
3ffffe30:  00000001 00000001 3fff2ba4 4021be82
3ffffe40:  00000001 4021bee0 3fff2ba4 401000e1
3ffffe50:  3fff10f4 3fff10f4 3fff2ba4 40207100
3ffffe60:  006d6300 3fff10f4 00ff10b4 40218a69
3ffffe70:  3fff2ba4 3fff10f4 3fff10b4 4020a7fa
3ffffe80:  006d632f 3fffc6fc 03000001 74736f00
3ffffe90:  00007400 00002f2a 00001d64 3fff0398
3ffffea0:  3fff10f4 00000000 4bc6a7f0 00000001
3ffffeb0:  00000001 00000001 40100302 5fbe76c8
3ffffec0:  00000000 3fff2e44 3fff10b4 3fff0398
3ffffed0:  00000001 3fff10d8 3fff10b4 4020a91b
3ffffee0:  4021c828 00000000 00001388 4021c499
3ffffef0:  00000000 3fff2e44 3ffee5d0 4021608a
3fffff00:  00000017 00000007 00000000 4020a9ba
3fffff10:  00000000 4bc6a7f0 00001d64 4020c2c0
3fffff20:  00000000 00000000 4bc6a7f0 00000000
3fffff30:  00000000 4bc6a7f0 00001d64 3fff0398
3fffff40:  00000017 00000007 00000000 4020afe3
3fffff50:  3ffee9d0 3ffeead0 40100302 3fff0398
3fffff60:  3fffdad0 00000000 00001d64 40210f18
3fffff70:  3fffdad0 00000000 00001d64 40210fd0
3fffff80:  00000000 00000000 00000001 40100214
3fffff90:  3fffdad0 00000000 3fff0358 3fff0398
3fffffa0:  3fffdad0 00000000 3fff0358 40219b20
3fffffb0:  feefeffe feefeffe 3ffe8538 40100de1
<<<stack<<<

--------------- CUT HERE FOR EXCEPTION DECODER ---------------

`

[thecem]

@Airey001 i have same hardare and problem with SP111

[e9hack]

Try the following fix:

--- a/tasmota/tasmota_support/support_command.ino
+++ b/tasmota/tasmota_support/support_command.ino
@@ -468,7 +468,7 @@ void CommandHandler(char* topicBuf, char* dataBuf, uint32_t data_len) {
 #endif  // USE_SCRIPT_SUB_COMMAND
   }

-  if (ResponseStartsWith(S_JSON_COMMAND_ERROR)) {
+  if (ResponseStartsWith("{\"" D_JSON_COMMAND "\":\"" D_JSON_ERROR "\"")) {
     // No calls to Response_P performed if got here so it's either Command Error or Unknown
     TasmotaGlobal.no_mqtt_response = false;  // Make sure to report commands starting with underline
     if (!strlen(type)) {

[arendst]

You're hot.

UPDATE: Yes that's the problem.

I'll fix it now although I cannot explain why it fails in minimal but works fine in normal binaries.

[thecem]

will this fix work via ota?

[arendst]

Yes but only if you:

• download latest tasmota-minimal.bin from either dev or release branch
• go to the GUI and use option Upgrade by file upload - select your downloaded file
  This will install a functional minimal.bin. Then continue to your desired binary by:
• go to the GUI and use Option Upgrade by web server - check correct OTA Url

[e9hack]

If tasmoadmin is used, it sends some request which forces the issue at the minimal binary during the update process. . If tasmoadmin isn't used, OTA works at buggy minimal image.

[thecem]

how to prevent the boot loop? no minimal.bin.gz is working!

[e9hack]

The normal binary fails too if you hit in the web interface in some menus the 'save' button.

[arendst]

Should be fixed now in latest dev branch (and was always fine in latest release v14.2.0)

[thecem]

[thecem]

no solution for this?

[TPDomo]

isolate it from anything in your network (at least that worked for me) and send new (or 14.2.0 ) tasmota-minimal.bin.gz first !

[TPDomo]

Wow, great it helped to identify bug and rescue some of us!
I owe some update from my troubleshooting by night...
First i tried to isolate board from MQTT and it's been "alive" a bit longer.
So, I blocked the board from connecting to my WiFi network ( Ubqt).
Surprisingly it was not catching exception by itself, just been trying to connect to defined wifi.
Exception was only appearing immediately if i tried to send (CTRL+Enter) any letter or digit from serial console.
So, i created isolated AP with same SSID and board connected to it, then was staying alive, just trying to connect to MQTT.
Again, sending any letter or digit was causing an immediate exception and reboot.
Wow, i thought - exceptions were triggered by "something" in my normal network....
And as @e9hack mentioned the "guilty" one was my Tasmoadmin instance (as seem sending some messages to boards all the time).
After i disabled Tasmoadmin container i've been able to update this and other 8 modules, flashing them manually from web interface...

Thank you all for engagement and fantastic support!

[CBSchilling]

MOSCHE liebe Community,

hier ein Workaround, welcher bei mir 100% funktioniert hat:

1. Einstellungen / Geräte & Dienste: Tasmota-Integration deaktivieren.
   TasmoAdmin kann aktiv bleiben um die IP des Gerätes direkt anzuklicken.

2. Download der zwei benötigten Dateien (in diesem Fall für ESP8266! (ESP32 abweichend!)):
   https://github.com/arendst/Tasmota-firmware/blob/firmware/release-firmware/tasmota/tasmota-minimal.bin.gz
   und:
   die finale FW (ist gerätespezifisch!) https://github.com/arendst/Tasmota-firmware/blob/firmware/release-firmware/tasmota/tasmota.bin.gz
   unter:

   • für ESP8266: https://github.com/arendst/Tasmota-firmware/tree/firmware/release-firmware/tasmota/
   • für ESP32: https://github.com/arendst/Tasmota-firmware/tree/firmware/release-firmware/tasmota32/

3. Über die lokale IP-Adresse (z.B. 192.168.2.171 o.ä.) auf das Device zugreifen und dann
   das Device rebooten.

4. Kurz abwarten, bis das Device wieder verbunden ist.

5. Unter Firmwareupgrade die heruntergeladene Datei "*-minimal.bin.gz" auswählen und updaten
   !!!! Bei mir hat das 1 bis 3 Durchläufe benötigt !!! - Einfach ein paar mal wiederholen, wenn es nicht funzt, meine Freunde! :-)

6. Nach erfolgreichem Upload und Reconnect die lokal heruntergeladene "finale" Firmware (gerätespezifisch!) hochladen.

7. Gerät über das Web-Frontend rebooten.

8. Bei allen Geräten wiederholen, die sich im Bootloop befinden.

9. Wenn ALLE Geräte wieder laufen unter:
   Einstellungen / Geräte & Dienste: Tasmota-Integration wieder aktivieren!

Nachtrag: Das OTA-Update funzt nicht, da die Dateien anscheinend fehlerhaft sind (per aktuellem Stand).

Viel Erfolg, liebe Tasmo-Jünger - bei mir hat es ganauso funktioniert!

#Chris :-)
#The@ ;-)

[Airey001]

Thanks worked perfectly for me. I think the key is making sure the HomeAssistant Intigration is off before you upload.

[e9hack]

To use simple strings with string class strings, they must be cast to string class:

--- a/tasmota/tasmota_support/support.ino
+++ b/tasmota/tasmota_support/support.ino
@@ -1325,8 +1325,8 @@ void ResponseReplace(const char* rold, const char* rnew) {
   TasmotaGlobal.mqtt_data.replace(rold, rnew);
 }
 
-bool ResponseStartsWith(const char* start_with) {
-  return TasmotaGlobal.mqtt_data.startsWith(start_with);
+bool ResponseStartsWith_P(const char* start_with) {
+  return TasmotaGlobal.mqtt_data.startsWith(reinterpret_cast<const __FlashStringHelper *>(start_with));
 }
 
 void ResponseJsonStart(void) {

This is mandatory for static flash strings. Inline flash strings are placed in a different section. The string class can handle them like normal (writeable) strings.

[arendst]

Thx. I rewrote that part today and got rid of that function.