Skip to content

OTA update not working over HTTPS #5818

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
6 tasks done
dzmpr opened this issue Feb 24, 2019 · 3 comments
Closed
6 tasks done

OTA update not working over HTTPS #5818

dzmpr opened this issue Feb 24, 2019 · 3 comments
Labels
waiting for feedback Waiting on additional info. If it's not received, the issue may be closed.

Comments

@dzmpr
Copy link

dzmpr commented Feb 24, 2019

Basic Infos

  • This issue complies with the issue POLICY doc.
  • I have read the documentation at readthedocs and the issue is not addressed there.
  • I have tested that the issue is present in current master branch (aka latest git).
  • I have searched the issue tracker for a similar issue.
  • If there is a stack dump, I have decoded it.
  • I have filled out all fields below.

Platform

  • Hardware: ESP-12E
  • Core Version: 2.5.0
  • Development Env: Arduino IDE/vscode w/ extension
  • Operating System: Windows

Settings in IDE

  • Module: NodeMCU 1.0 (ESP-12E Module)
  • Flash Mode: didn't find, through CH340
  • Flash Size: 4MB
  • lwip Variant: v2 Lower Memory
  • Reset Method: nodemcu
  • Flash Frequency: 40Mhz
  • CPU Frequency: 80Mhz
  • Upload Using: SERIAL
  • Upload Speed: 115200

Problem Description

I trying update ESP by OTA from my server over HTTPS. Server is Apache/2.4.6 with PHP 7.3 and Let's Encrypt Authority X3 certificate. Server side script taken from documentation. Basically nothing happens at all (Debug Messages and Log from server). In the server logs package size rarely is 32768 instead 16384. Once I catched wdt reset - decoded stack below.

Strange fact - I get successfully updated ESP only when enabled debug on level "CORE+WIFI+HTTP_UPDATE+UPDATER+OTA+OOM", not from first time, it takes several attempts. But it is updates every time I try! On other debug levels it still not updating.

I tried update without server script, directly pointing .bin location (not working). Often I gets "Read Timeout" error and tried increase timeout for 10 sec, but it isn't helped.

MCVE Sketch

#include <Arduino.h>
#include <ESP8266WiFi.h>
#include <ESP8266HTTPClient.h>
#include <ESP8266httpUpdate.h>
#include <WiFiClientSecureBearSSL.h>

const char *fingerprint = "69:D6:81:6C:C9:B3:2F:78:C4:66:44:36:A5:79:49:99:DC:A9:DD:1F";

void setup() {

  Serial.begin(115200);
  WiFi.begin(APSSID, APPSK);
    Serial.print("\nConnecting to wifi..\n");
    while (WiFi.status() != WL_CONNECTED) {
        Serial.print(".");
        delay(500);
    }
    Serial.println("WiFi Connected!");
}

void loop() {
    std::unique_ptr<BearSSL::WiFiClientSecure>client(new BearSSL::WiFiClientSecure);
    client->setFingerprint(fingerprint);
    ESPhttpUpdate.setLedPin(LED_BUILTIN, LOW);
    t_httpUpdate_return ret = ESPhttpUpdate.update(*client, "https://host/update.php", "1.1.1");
    switch (ret) {
      case HTTP_UPDATE_FAILED:
        Serial.printf("HTTP_UPDATE_FAILD Error (%d): %s\n", ESPhttpUpdate.getLastError(), ESPhttpUpdate.getLastErrorString().c_str());
        break;

      case HTTP_UPDATE_NO_UPDATES:
        Serial.println("HTTP_UPDATE_NO_UPDATES");
        break;

      case HTTP_UPDATE_OK:
        Serial.println("HTTP_UPDATE_OK");
        break;
    }
}

Debug Messages

SDK:3.0.0-dev(c0f7b44)/Core:2.5.0=20500000/lwIP:STABLE-2_1_2_RELEASE/glue:1.1/BearSSL:6778687

Connecting to wifi..
.scandone
state: 0 -> 2 (b0)
state: 2 -> 3 (0)
state: 3 -> 5 (10)
add 0
aid 2
cnt 

connected with Nodule, channel 7
dhcp client start...
ip:192.168.0.104,mask:255.255.255.0,gw:192.168.0.1
WiFi Connected!
pm open,type:2 0

Log from server

[u][24/Feb/2019:22:40:53 +0300] 0.050 8.100 200  GET /update.php HTTP/1.0 "ESP8266-http-Update" "-" 16384 
[u][24/Feb/2019:22:41:02 +0300] 0.050 8.100 200  GET /update.php HTTP/1.0 "ESP8266-http-Update" "-" 16384 
[u][24/Feb/2019:22:41:33 +0300] 0.000 30.000 200  GET /update.php HTTP/1.0 "ESP8266-http-Update" "-" 16384 
[u][24/Feb/2019:22:45:59 +0300] 0.000 30.000 200  GET /update.php HTTP/1.0 "ESP8266-http-Update" "-" 16384 

Decoded stack dump

0x402121e8: memp_malloc at core/memp.c line 356
0x402137fd: sys_timeout_abs at core/timeouts.c line 189
0x402138f8: tcpip_tcp_timer at core/timeouts.c line 145
0x402048eb: BearSSL::WiFiClientSecure::flush(unsigned int) at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\libraries\ESP8266WiFi\src\WiFiClientSecureBearSSL.cpp line 204
0x40203091: WiFiClient::stop(unsigned int) at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\libraries\ESP8266WiFi\src\WiFiClient.cpp line 326
0x402047b7: BearSSL::WiFiClientSecure::stop(unsigned int) at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\libraries\ESP8266WiFi\src\WiFiClientSecureBearSSL.cpp line 191
0x40203934: WiFiClient::stopAll() at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\libraries\ESP8266WiFi\src\WiFiClient.cpp line 386
0x40202ae0: ESP8266WiFiGenericClass::_eventCallback(void*) at c:\users\User\appdata\local\arduino15\packages\esp8266\tools\xtensa-lx106-elf-gcc\2.5.0-3-20ed2b9\xtensa-lx106-elf\include\c++\4.8.2\bits/range_access.h line 48
0x4021e152: br_sha2small_round at src/hash/sha2small.c line 98
0x4021dd38: br_sha2small_round at src/hash/sha2small.c line 85
0x4021daa0: br_sha384_update at src/hash/sha2big.c line 165
0x4021e30b: sha2small_out at src/hash/sha2small.c line 249
0x4022aa7d: br_multihash_update at src/hash/multihash.c line 141
0x40211ae5: glue2esp_linkoutput at glue-esp/lwip-esp.c line 299
0x40204825: BearSSL::WiFiClientSecure::_run_until(unsigned int, bool) at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\libraries\ESP8266WiFi\src\WiFiClientSecureBearSSL.cpp line 432
0x40204813: BearSSL::WiFiClientSecure::_run_until(unsigned int, bool) at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\libraries\ESP8266WiFi\src\WiFiClientSecureBearSSL.cpp line 427
0x402170e6: tcp_output at core/tcp_out.c line 1319
0x40100628: _umm_free at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\cores\esp8266\umm_malloc\umm_malloc.c line 1300
0x40100d7c: free at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\cores\esp8266\umm_malloc\umm_malloc.c line 1760
0x4021ad64: mem_free at core/mem.c line 237
0x40207e98: esp_yield() at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\cores\esp8266\core_esp8266_main.cpp line 91
0x40207f76: __yield() at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\cores\esp8266\core_esp8266_main.cpp line 100
0x40207fb9: optimistic_yield(uint32_t) at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\cores\esp8266\core_esp8266_main.cpp line 115
0x40202ea7: WiFiClient::available() at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\libraries\ESP8266WiFi\src\WiFiClient.cpp line 263
0x4020486e: BearSSL::WiFiClientSecure::_run_until(unsigned int, bool) at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\libraries\ESP8266WiFi\src\WiFiClientSecureBearSSL.cpp line 489
0x40220148: br_ssl_hs_client_run at src/ssl/ssl_hs_client.c line 958
0x402200f0: br_ssl_hs_client_init_main at /home/earle/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/xtensa-lx106-elf/include/sys/pgmspace.h line 67
0x4021fbfe: br_ssl_engine_hs_reset at src/ssl/ssl_engine.c line 1305
0x40204aa0: BearSSL::WiFiClientSecure::_wait_for_handshake() at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\libraries\ESP8266WiFi\src\WiFiClientSecureBearSSL.cpp line 526
0x40204c83: BearSSL::WiFiClientSecure::_connectSSL(char const*) at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\libraries\ESP8266WiFi\src\WiFiClientSecureBearSSL.cpp line 935
0x40207e98: esp_yield() at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\cores\esp8266\core_esp8266_main.cpp line 91
0x40201627: delay at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\cores\esp8266\core_esp8266_wiring.c line 51
0x402038c1: WiFiClient::connect(IPAddress const&, unsigned short) at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\libraries\ESP8266WiFi\src/include/ClientContext.h line 136
0x40204d4a: BearSSL::WiFiClientSecure::connect(char const*, unsigned short) at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\libraries\ESP8266WiFi\src\WiFiClientSecureBearSSL.cpp line 223
0x40209984: HTTPClient::connected() at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\libraries\ESP8266HTTPClient\src\ESP8266HTTPClient.cpp line 448
0x402088b4: HTTPClient::connect() at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\libraries\ESP8266HTTPClient\src\ESP8266HTTPClient.cpp line 1055
0x402074fb: String::reserve(unsigned int) at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\cores\esp8266\WString.cpp line 146
0x40208e20: HTTPClient::sendRequest(char const*, unsigned char*, unsigned int) at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\libraries\ESP8266HTTPClient\src\ESP8266HTTPClient.cpp line 593
0x4020769d: String::operator=(char const*) at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\cores\esp8266\WString.cpp line 243
0x40211988: operator new[](unsigned int) at ../../../../../dl/gcc-xtensa/libstdc++-v3/libsupc++/new_opv.cc line 33
0x40208767: HTTPClient::collectHeaders(char const**, unsigned int) at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\libraries\ESP8266HTTPClient\src\ESP8266HTTPClient.cpp line 981
0x40208ece: HTTPClient::GET() at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\libraries\ESP8266HTTPClient\src\ESP8266HTTPClient.cpp line 526
0x40205e51: ESP8266HTTPUpdate::handleUpdate(HTTPClient&, String const&, bool) at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\libraries\ESP8266httpUpdate\src\ESP8266httpUpdate.cpp line 290
0x40100d7c: free at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\cores\esp8266\umm_malloc\umm_malloc.c line 1760
0x4020746c: String::invalidate() at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\cores\esp8266\WString.cpp line 141
0x40206039: ESP8266HTTPUpdate::update(WiFiClient&, String const&, String const&) at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\libraries\ESP8266httpUpdate\src\ESP8266httpUpdate.cpp line 94
0x40207400: UpdaterClass::printError(Print&) at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\cores\esp8266\Updater.cpp line 512
0x4020757a: String::String(char const*) at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\cores\esp8266\WString.cpp line 36
0x4020287c: loop() at C:\Rep\Dev\Arduino\Abug/Abug.ino line 94
0x4020280c: setup() at C:\Rep\Dev\Arduino\Abug/Abug.ino line 88
0x40207f44: loop_wrapper() at C:\Users\User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.5.0\cores\esp8266\core_esp8266_main.cpp line 125
@dzmpr dzmpr mentioned this issue Apr 19, 2019
6 tasks
@earlephilhower earlephilhower added the waiting for feedback Waiting on additional info. If it's not received, the issue may be closed. label May 1, 2019
@earlephilhower
Copy link
Collaborator

I'm unable to reproduce any problem with this on my own testing. I've pointed the app (seems like a copy of the example?) directly to a private https server (and updated the fingerprint, obviously) and had it pull file.bin directly. It works every time and then reboots into the new sketch.

Please past the full decode, including the header lines (i.e. where it actually crashed, whether there were OOM errors, etc.) on another run with GIT head. The current issue didn't list where it actually crashed.

Anything to the pattern (like, it tries and fails to find a new file 3 times but then on the 4rd itry it gets it and crashes or something)?

@dzmpr
Copy link
Author

dzmpr commented Jun 4, 2019

I'm reverted to 2.4.2 to fix this problem. Now I use 2.5.2, where updates (seems) works fine.
But there is some problem from 2.5.0. When I send GET requests periodically - responses doesn't update. I.e. I get response "1", then I change data on server ("1" -> "2"), but responses still go with "1" payload. When I reboot board responses came with new data "2", but when I change again - responses still "2".
And when HTTP reuse set to false odd responses come with 200 code, even responses with -2 code (Can't send header), and both come without payload.
And also I'd like to note that on 2.4.2 requests works properly, so problem caused not by server.

#include <Arduino.h>
#include <ESP8266WiFi.h>
#include <ESP8266HTTPClient.h>

#define SSID 
#define PASS 

HTTPClient https;
BearSSL::WiFiClientSecure client;

void setup() {
    Serial.begin(115200);
    WiFi.begin(SSID,PASS);
    while(WiFi.status() != WL_CONNECTED) {
        Serial.print(".");
        delay(500);
    }
    Serial.println("\nWifi connected!");

    uint8_t cert[20] = {184,120,14,91,203,255,205,98,26,130,214,144,95,34,236,197,210,208,31,245};
    client.setFingerprint(cert);
    https.setReuse(true);
    String url = "https://apilink";
    https.begin(client,url);
    Serial.println(https.GET());
    Serial.println(https.connected());
    for(uint8_t i = 0; i < 10; i++) {
        Serial.print(String(i)+" : ");
        Serial.println(https.GET());
        Serial.println(https.getString());
        client.flush();
        delay(1000);
    }
    
}

void loop() {
}

@earlephilhower
Copy link
Collaborator

Closing this for now. OTA reported working fine in the latest release, and the only outstanding bit is the http.GET method which is still borked (but with a PR in-progress). We'll track that problem in the consolidated issue referenced above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
waiting for feedback Waiting on additional info. If it's not received, the issue may be closed.
Projects
None yet
Development

No branches or pull requests

2 participants