OTA (HTTP client mode) running out of memory and crashing when using TLS (https) #2499
Comments
|
I had a slightly more involved sketch (fetches time via NTP and uses NeoPixel library to drive an array of WS LEDs with the number of days until Christmas. |
|
FWIW, my systems are running reasonably stable after switching to a custom implementation that fetches only a cryptographically secure checksum over HTTPS and performs the actual update over HTTP. This isn't optimal for privacy, but still provides protection against firmware modification. |
|
I've got the same problem as you. But with a very small original sketch (the sketch making the upgrade), it work to HTTPS OTA update. As you said before, it's probably a memory issue. |
|
@silbe is this issue still valid with latest git? |
devyte
added
the
waiting for feedback
|
I'm using last rc (2.4.0-rc2) and this issue is still present. |
|
I think there should be enough RAM at this point for OTA-over-HTTPS to be possible, as axTLS memory usage was trimmed down by several kB since 2.3.0. However the issue likely is that there isn't a single contiguous block of 16kB at the time it is needed. I have a work-in-progress implementation of memory pool for axTLS, which should help with this, but it's not ready yet and i will try to finish it once 2.4.0 is released. |
|
@igrr A crazy thought that keeps nagging at me: instead of trying to hold the entire 16KB buffer in RAM, do you think it would be possible to mimic a 16KB buffer in a manner similar to a computer swap file? I mean have the data on flash or SPIFFS, and hold in RAM only e.g.: 4KB at a time. You could put an accessor in front to make it look like a buffer. When you to access beyond the 4KB in RAM, you swap the 4KB block with the corresponding block in the flash. |
|
In my mind usage would look something like this: |
|
It's possible, i think. The question is how many flash sectors you allocate for swap space. Too few, and you kill your flash pretty fast. Too many, and your filesystem can no longer fit the files you need. Also this is going to be very slow. Assuming a 16kB TLS fragment buffer, that's 4 write-erase cycles to fill it with received encrypted data, and another 4 write-erase cycles to decrypt it. For those who find this issue a showstopper, deploying a Haproxy or nginx instance can be solution. It would be configured to support TLS fragment size negotiation extension. Then ESP8266 can tell that it wants to use 4kB TLS fragments, and 16kB buffer would not be needed at all. |
|
BearSSL is merged in #4273 , with alternate BearSSL::WiFi* classes. Although axtls-based classes are still available and even the default, they are planned for deprecation and then retirement, hence won't be fixed. Any issues with BearSSL-based classes should be reported in new issues. |
Basic Infos
Hardware
Hardware: ESP-12
Core Version: git master (commit 4897e00)
Description
When modifying the ESP8266httpUpdate example "httpUpdate" to use an https URI, the ESP8266 crashes with a null pointer dereference (Exception 29 = StoreProhibitedCause, excvaddr=0). Debugging suggests we're running out of DRAM, even though the sketch is not doing anything other than OTA. The server uses a 2048-bit RSA key with a certificate issued by letsencrypt.org.
The null pointer dereference is already tracked as #2201; this ticket is about running out of memory in the first place. The failure is actually very similar, but in #2201 the server used a 4096-bit key (which could be considered excessive), whereas we're using the absolute minimum currently permitted by most CAs and major browsers. If it doesn't even work with a 2048-bit key and an otherwise empty sketch, it's pretty much useless for practical purposes.
Build system
Build system: makeEspArduino (commit 18d85bc5)
Module: Generic ESP8266 Module (BOARD=generic)
Flash Size: 4MB/1MB (FLASH_DEF=4M1M)
CPU Frequency: unmodified
Flash Mode: unmodified
Flash Frequency: unmodified
Upload Using: SERIAL
Reset Method: manual (power switch)
Sketch
Makefile
Debug Messages
Serial output without debugging options (which can increase IRAM/DRAM usage):
Decoding addresses contained in the above output:
After enabling
-DDEBUG_ESP_HTTP_UPDATE -DDEBUG_ESP_PORT=Serial, I get the following on the serial port:Decoded addresses:
The text was updated successfully, but these errors were encountered: