OTA crash with UDP server and CORE debug

[luffykesh]

Basic Infos

• [x ] This issue complies with the issue POLICY doc.
• [x ] I have read the documentation at readthedocs and the issue is not addressed there.
• [x ] I have tested that the issue is present in current master branch (aka latest git).
• [ x] I have searched the issue tracker for a similar issue.
• [ ] If there is a stack dump, I have decoded it.
• [x ] I have filled out all fields below.

Platform

• Hardware: ESP-12
• Core Version: git | last commit I have 2528013c5dc1ce7703bf211b41a3aefcd6afe0e5
• Development Env: Arduino IDE
• Operating System: MacOS

Settings in IDE

• Module: Generic ESP8266 Module
• Flash Mode: qio
• Flash Size: 4MB
• lwip Variant: v2 Higher Bandwidth
• Reset Method: ck
• Flash Frequency: 40Mhz
• CPU Frequency: 80Mhz
• Upload Using: OTA
• Debug Port: Serial
• Debug Level: Core
• Espressif FW: 2.2.1

Problem Description

Detailed problem description goes here.

OTA updates crash with Fatal exception 3(LoadStoreErrorCause) when there is a UDP server running AND CORE debug is enabled.
If CORE debug is disabled, it works just fine.

Haven't been able to decode the stack, because with the the git version the ESPExceptionDecoder is not able to locate the file xtensa-lx106-elf-addr2line:ERROR: xtensa-lx106-elf-addr2line not found!

MCVE Sketch

#include <ESP8266WiFi.h>
#include <WiFiClient.h>
#include <ESP8266WebServer.h>
#include <ESP8266HTTPUpdateServer.h>
#include <WiFiUDP.h>

unsigned int localPort = 8888;

#ifndef STASSID
#define STASSID "ssid"
#define STAPSK  "password"
#endif

const char* ssid = STASSID;
const char* password = STAPSK;

ESP8266WebServer httpServer(80);
ESP8266HTTPUpdateServer httpUpdater;
WiFiUDP Udp;
char packetBuffer[UDP_TX_PACKET_MAX_SIZE];

void setup(void) {
  Serial.begin(9600);
  Serial.println();
  Serial.println("Booting Sketch...");
  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);

  while (WiFi.waitForConnectResult() != WL_CONNECTED) {
    WiFi.begin(ssid, password);
    Serial.println("WiFi failed, retrying.");
  }
  Serial.print("IP: ");
  Serial.println(WiFi.localIP());
  httpUpdater.setup(&httpServer);
  httpServer.begin();
  Udp.begin(localPort);

}
void loop(void) {
  httpServer.handleClient();
    // if there's data available, read a packet
  int packetSize = Udp.parsePacket();
  if (packetSize) {
    Serial.print("Received packet of size ");
    Serial.println(packetSize);
    Serial.print("From ");
    IPAddress remote = Udp.remoteIP();
    for (int i = 0; i < 4; i++) {
      Serial.print(remote[i], DEC);
      if (i < 3) {
        Serial.print(".");
      }
    }
    Serial.print(", port ");
    Serial.println(Udp.remotePort());

    // read the packet into packetBufffer
    Udp.read(packetBuffer, UDP_TX_PACKET_MAX_SIZE);
    Serial.println("Contents:");
    Serial.println(packetBuffer);
  }
	delay(10);
}

[luffykesh]

I converted the ESPExceptionDecoder to CLI version and successfully decoded the stack.
Here is the gist which contains stack trace along with the the decoding.

Decode Success
Exception 3: LoadStoreError: Processor internal physical address or data error during load or store
PC: 0x4000228b
EXCVADDR: 0x4024549b

Decoding stack results
0x40208dd0: ets_printf_P(char const*, ...) at /Users/rushikesh/Library/Arduino15/packages/esp8266/hardware/esp8266/2.5.0/cores/esp8266/ core_esp8266_postmortem.cpp  line 80
0x40208dc4: ets_printf_P(char const*, ...) at /Users/rushikesh/Library/Arduino15/packages/esp8266/hardware/esp8266/2.5.0/cores/esp8266/ core_esp8266_postmortem.cpp  line 80
0x40100290: vPortFree(void*, char const*, int) at /Users/rushikesh/Library/Arduino15/packages/esp8266/hardware/esp8266/2.5.0/cores/esp8266/ heap.cpp  line 61
0x40208e1e: print_stack(uint32_t, uint32_t) at /Users/rushikesh/Library/Arduino15/packages/esp8266/hardware/esp8266/2.5.0/cores/esp8266/ core_esp8266_postmortem.cpp  line 197
0x40208fc4: __wrap_system_restart_local() at /Users/rushikesh/Library/Arduino15/packages/esp8266/hardware/esp8266/2.5.0/cores/esp8266/ core_esp8266_postmortem.cpp  line 175
0x4020aa9e: umm_realloc(void*, size_t) at /Users/rushikesh/Library/Arduino15/packages/esp8266/hardware/esp8266/2.5.0/cores/esp8266/umm_malloc/ umm_malloc.cpp  line 1745
0x4020973c: realloc_loc(void*, size_t, char const*, int) at /Users/rushikesh/Library/Arduino15/packages/esp8266/hardware/esp8266/2.5.0/cores/esp8266/ heap.cpp  line 142
0x4020aa9e: umm_realloc(void*, size_t) at /Users/rushikesh/Library/Arduino15/packages/esp8266/hardware/esp8266/2.5.0/cores/esp8266/umm_malloc/ umm_malloc.cpp  line 1745
0x4020973c: realloc_loc(void*, size_t, char const*, int) at /Users/rushikesh/Library/Arduino15/packages/esp8266/hardware/esp8266/2.5.0/cores/esp8266/ heap.cpp  line 142
0x40202cd4: WiFiUDP::stopAll() at /Users/rushikesh/Library/Arduino15/packages/esp8266/hardware/esp8266/2.5.0/libraries/ESP8266WiFi/src/ WiFiUdp.cpp  line 270
0x40206113: std::_Function_handler<void(), ESP8266HTTPUpdateServer::setup(ESP8266WebServer*, const String&, const String&, const String&)::__lambda2>::_M_invoke(const std::_Any_data &) at /Users/rushikesh/Library/Arduino15/packages/esp8266/hardware/esp8266/2.5.0/libraries/ESP8266HTTPUpdateServer/src/ ESP8266HTTPUpdateServer.cpp  line 72
0x402079b0: String::~String() at /Users/rushikesh/Library/Arduino15/packages/esp8266/hardware/esp8266/2.5.0/cores/esp8266/ WString.cpp  line 125
0x402044e2: std::function<void ()>::operator()() const at /Users/rushikesh/Library/Arduino15/packages/esp8266/hardware/esp8266/2.5.0/tools/xtensa-lx106-elf/xtensa-lx106-elf/include/c++/4.8.2/ functional  line 2465
0x40202d34: FunctionRequestHandler::canUpload(String) at /Users/rushikesh/Library/Arduino15/packages/esp8266/hardware/esp8266/2.5.0/libraries/ESP8266WebServer/src/detail/ RequestHandlersImpl.h  line 30
0x4020455d: FunctionRequestHandler::upload(ESP8266WebServer&, String, HTTPUpload&) at /Users/rushikesh/Library/Arduino15/packages/esp8266/hardware/esp8266/2.5.0/libraries/ESP8266WebServer/src/detail/ RequestHandlersImpl.h  line 51
0x40207ce0: String::String(String const&) at /Users/rushikesh/Library/Arduino15/packages/esp8266/hardware/esp8266/2.5.0/cores/esp8266/ WString.cpp  line 41
0x4020522b: ESP8266WebServer::_parseForm(WiFiClient&, String const&, unsigned int) at /Users/rushikesh/Library/Arduino15/packages/esp8266/hardware/esp8266/2.5.0/libraries/ESP8266WebServer/src/ Parsing.cpp  line 479
0x40204b31: ESP8266WebServer::_parseArguments(String const&) at /Users/rushikesh/Library/Arduino15/packages/esp8266/hardware/esp8266/2.5.0/libraries/ESP8266WebServer/src/ Parsing.cpp  line 299
0x40204cd0: std::_Function_handler<void (String&, String&, String const&, int, int, int, int), storeArgHandler>::_M_invoke(std::_Any_data const&, String&, String&, String const&, int, int, int, int) at /Users/rushikesh/Library/Arduino15/packages/esp8266/hardware/esp8266/2.5.0/tools/xtensa-lx106-elf/xtensa-lx106-elf/include/c++/4.8.2/ functional  line 2069
0x40204530: FunctionRequestHandler::upload(ESP8266WebServer&, String, HTTPUpload&) at /Users/rushikesh/Library/Arduino15/packages/esp8266/hardware/esp8266/2.5.0/libraries/ESP8266WebServer/src/detail/ RequestHandlersImpl.h  line 46
0x40202d34: FunctionRequestHandler::canUpload(String) at /Users/rushikesh/Library/Arduino15/packages/esp8266/hardware/esp8266/2.5.0/libraries/ESP8266WebServer/src/detail/ RequestHandlersImpl.h  line 30
0x40205ba9: ESP8266WebServer::_parseRequest(WiFiClient&) at /Users/rushikesh/Library/Arduino15/packages/esp8266/hardware/esp8266/2.5.0/libraries/ESP8266WebServer/src/ Parsing.cpp  line 210
0x40202e88: FunctionRequestHandler::canHandle(HTTPMethod, String) at /Users/rushikesh/Library/Arduino15/packages/esp8266/hardware/esp8266/2.5.0/libraries/ESP8266WebServer/src/detail/ RequestHandlersImpl.h  line 20
0x4020478f: ESP8266WebServer::handleClient() at /Users/rushikesh/Library/Arduino15/packages/esp8266/hardware/esp8266/2.5.0/libraries/ESP8266WebServer/src/ ESP8266WebServer.cpp  line 303
0x4020911f: delay(unsigned long) at /Users/rushikesh/Library/Arduino15/packages/esp8266/hardware/esp8266/2.5.0/cores/esp8266/ core_esp8266_wiring.cpp  line 54
0x40201143: loop() at /Users/rushikesh/Documents/Arduino/sketch_apr30a/ sketch_apr30a.ino  line 46
0x40208bf8: loop_wrapper() at /Users/rushikesh/Library/Arduino15/packages/esp8266/hardware/esp8266/2.5.0/cores/esp8266/ core_esp8266_main.cpp  line 125

[hreintke]

Sorry for using this thread to get other info.
@luffykesh : would also like to have a CLI version of ESPExceptionDecoder .
Do you have it somewhere available or can you show how it can be converted ?

[luffykesh]

@hreintke https://github.com/luffykesh/EspExceptionDecoder-CLI

[d-a-v]

@luffykesh

Can you try commenting this line:

Arduino/libraries/ESP8266WiFi/src/WiFiUdp.cpp

Line 269 in 192aaa4

DEBUGV("%s %p %p\n", __func__, it, _s_first);

In my build, this %s is at address 0x40244cbb which is not even.
objdump shows the right string (stopAll).

[luffykesh]

@d-a-v
Yes, It works when

Arduino/libraries/ESP8266WiFi/src/WiFiUdp.cpp

Line 269 in 192aaa4

DEBUGV("%s %p %p\n", __func__, it, _s_first);

is commented.
The __func__ macro is what is causing the problem. I tried printing it elsewhere in the sketch too.

[luffykesh]

@d-a-v I have 3 devices, 2 of which do not boot after OTA is successful (after removing debug print in WiFiUdp::stopAll), with CORE debug on.
Just simply stops here.

 ets Jan  8 2013,rst cause:2, boot mode:(3,6)

load 0x4010f000, len 1384, room 16 
tail 8
chksum 0x2d
csum 0x2d
v2528013c
~ld
e:
 ets Jan  8 2013,rst cause:3, boot mode:(3,6)

ets_main.c 

[d-a-v]

It's a different issue and OP is about to be closed-fixed.
Since several firmware versions are now around, can you try and flash erase those two devices, reflash them then OTA-flash again and report ? If this new issue persists, please file a new bug.