Fatal Exception (29) cycling Client Secure (both SHA1 and CACert validation)

[Mark8869]

Basic Infos

• This issue complies with the issue POLICY doc.
• I have read the documentation at readthedocs and the issue is not addressed there.
• I have tested that the issue is present in current master branch (aka latest git).
• I have searched the issue tracker for a similar issue.
• If there is a stack dump, I have decoded it.
• I have filled out all fields below.

Platform

• Hardware: [ESP-12E]
• Core Version: [2.4.1]
• Development Env: [Arduino IDE]
• Operating System: [Windows]

Settings in IDE

• Module: [Nodemcu]
• Flash Mode: [dio]
• Flash Size: [4MB]
• lwip Variant: [v2 Lower Memory]
• Reset Method: [nodemcu]
• Flash Frequency: [40Mhz]
• CPU Frequency: [80Mhz]
• Upload Using: [SERIAL]
• Upload Speed: [115200] (serial upload only)

Problem Description

Hello,
I get an error with the "client secure" library (both with SHA1 and CACert validation).
In practice, in my sketch I have to use the secure client to communicate with two different hosts, so I have to use two different root certificates.
I have compiled the sketch as attached, but after 4 loops I get an error on the serial monitor (Fatal Exception 29), that seems to be memory issue.
I have tried to modify the sketch using CACert for first host and SHA1 fingerprint for second host (avoiding the repeating of certificates loading), and in this way it reaches 28 loops, then again the same error.

UPDATE: see next post.

MCVE Sketch

#include <WiFiClientSecure.h>

WiFiClientSecure client;

String apiKey = "OMITTED"; 
const char* host2 = "api.thingspeak.com"; 
const char* host1 = "maker.ifttt.com";
const int httpsPort = 443;

const unsigned char caCert1[] PROGMEM = {OMITTED};
const unsigned int caCertLen1 = 1028;

const unsigned char caCert2[] PROGMEM = {OMITTED};
const unsigned int caCertLen2 = 969;

extern const unsigned char caCert1[] PROGMEM;
extern const unsigned int caCertLen1;

extern const unsigned char caCert2[] PROGMEM;
extern const unsigned int caCertLen2;

void setup() {
  Serial.begin(115200);
  while (!Serial) {   //ATTESA APERTURA PORTA SERIALE
    delay(10); 
  }

  configTime(0, 0, "1.it.pool.ntp.org", "2.it.pool.ntp.org", "3.it.pool.ntp.org"); //CONFIGURAZIONE CLIENT NTP
  while (time(nullptr) < 50000) {
    Serial.println("UPDATING SNTP");
    delay(1000);
  }
}

void loop() {
  webhooks("test");
  thingspeak("a","b","c","d","e","f","g","h");
  delay(15000);
}

//************************************************************************************************************
void webhooks(String eventname) {

  time_t now = myTZ.toLocal(time(nullptr), &tcr);
  struct tm * timeinfo = gmtime(&now);

  // Load root certificate in DER format into WiFiClientSecure object
  bool res = client.setCACert_P(caCert1, caCertLen1);
  if (!res) {
    Serial.println("Failed to load root CA certificate!");
    while (true) {
      yield();
    }
  }
  
// Connect to remote server
  Serial.print("connecting to ");
  Serial.println(host1);
  if (!client.connect(host1, httpsPort)) {
    Serial.println("connection failed");
    return;
  }

  // Verify validity of server's certificate
  if (client.verifyCertChain(host1)) {
    Serial.println("Server certificate verified");
  } else {
    Serial.println("ERROR: certificate verification failed!");
    return;
  }

  String url = "/trigger/" + eventname + "/with/key/OMITTED"; //IFTTT MAKER KEY
  Serial.print("requesting URL: ");
  Serial.println(host1 + url);

  client.print(String("GET ") + url + " HTTP/1.1\r\n" +
               "Host: " + host1 + "\r\n" +
               "User-Agent: BuildFailureDetectorESP8266\r\n" +
               "Connection: close\r\n\r\n");

  Serial.println("request sent");
  while (client.connected()) {
    String line = client.readStringUntil('\n');
    if (line == "\r") {
      Serial.println("headers received");
      break;
    }
  }

  client.stop();

}

//************************************************************************************************************
void thingspeak(String field1,String field2,String field3,String field4,
                String field5,String field6,String field7,String field8) {


  time_t now = myTZ.toLocal(time(nullptr), &tcr);
  struct tm * timeinfo = gmtime(&now);

  bool res = client.setCACert_P(caCert2, caCertLen2);
  if (!res) {
    Serial.println("Failed to load root CA certificate!");
    while (true) {
      yield();
    }
  }
  
  Serial.print("connecting to ");
  Serial.println(host2);
  if (!client.connect(host2, httpsPort)) {
    Serial.println("connection failed");
    return;
  }

  if (client.verifyCertChain(host2)) {
    Serial.println("Server certificate verified");
  } else {
    Serial.println("ERROR: certificate verification failed!");
    return;
  }

  String postStr = apiKey;
  postStr +="&field1=";
  postStr += String(field1); //hh
  postStr +="&field2=";
  postStr += String(field2); //mm
  postStr +="&field3=";
  postStr += String(field3); //t_int
  postStr +="&field4=";
  postStr += String(field4); //t_ext
  postStr +="&field5=";
  postStr += String(field5); //t_rad
  postStr +="&field6=";
  postStr += String(field6); //heat_lev
  postStr +="&field7=";
  postStr += String(field7); //heat_sys
  postStr +="&field8=";
  postStr += String(field8); //as_usage     

  client.print("POST /update HTTP/1.1\n");
  client.print("Host: api.thingspeak.com\n");
  client.print("Connection: close\n");
  client.print("X-THINGSPEAKAPIKEY: "+apiKey+"\n");
  client.print("Content-Type: application/x-www-form-urlencoded\n");
  client.print("Content-Length: ");
  client.print(postStr.length());
  client.print("\n\n");
  client.print(postStr);
  Serial.println("%. Send to Thingspeak.");
               
  client.stop();
  
}

Debug Messages

connecting to maker.ifttt.com

Exception (29):
epc1=0x4020c7ea epc2=0x00000000 epc3=0x00000000 excvaddr=0x00000004 depc=0x00000000

ctx: cont 
sp: 3fff04a0 end: 3fff0920 offset: 01a0

>>>stack>>>
3fff0640:  00820000 00000082 3fff3b44 402199f2  
3fff0650:  3ffef8ac 3fff6264 3fff3b44 40219bec  
3fff0660:  3fff4ef4 00000208 3fff37a4 40100670  
3fff0670:  00000104 00002960 0000052c 00000042  
3fff0680:  00000040 00000082 3fff3abc 40219f05  
3fff0690:  00800000 00000080 00000000 00000041  
3fff06a0:  00000040 0000003f 3fff899c 00000041  
3fff06b0:  3fff4adc 3fff3abc 3fff899c 3fff896c  
3fff06c0:  3fff4ff0 ffffffff 3fff6264 40204ee4  
3fff06d0:  00000040 3fff6264 3fff3abc 3fff6264  
3fff06e0:  00000040 3fff6264 3fff3abc 4021aa3f  
3fff06f0:  3fff8924 0000003f c57631a8 00000001  
3fff0700:  3fff6264 3fff89b4 3fff3abc 00000001  
3fff0710:  00000010 3fff6264 0000000f 4021ac94  
3fff0720:  3fff899c 3fff8984 0000000f 00000001  
3fff0730:  00000001 3fff8954 3fff80e4 401004f4  
3fff0740:  3fff701c 3fff80e4 3fff7034 3fff624c  
3fff0750:  3fff701c 3fff80e4 3fff6264 4021ad97  
3fff0760:  3fff8954 3fff7034 3fff6264 40219a98  
3fff0770:  00000100 3fff90dc 3fff66f4 00000000  
3fff0780:  00000100 3fff90dc 3fff80e4 402177ff  
3fff0790:  5adeecba 0009de85 3fff1f54 40219d6e  
3fff07a0:  3fff7bb4 3fff9b7c 3fff8144 3fff7b6c  
3fff07b0:  3fff0850 3fff1664 00000877 3fff7034  
3fff07c0:  3fff701c 00000000 3fff51fc 40100690  
3fff07d0:  00000100 3fff724c 3fff1f54 3fff51fc  
3fff07e0:  00000100 3fff724c 3fff1f54 402178a4  
3fff07f0:  5adeecba 000709ba 00000013 3fff65c4  
3fff0800:  3fff7bb4 3fff90dc 3fff1fcc 3fff9b24  
3fff0810:  3fff0850 3fff1664 40204e2c 3fff7034  
3fff0820:  3fff701c 00003a98 00003a98 4020136f  
3fff0830:  3ffef5c8 00000000 3fff6bf4 3ffef5c8  
3fff0840:  3ffe851c 3fff271c 3fff15f4 402157d7  
3fff0850:  00000001 3ffef5c8 3ffe8e78 3ffef5c8  
3fff0860:  000001bb 3ffef5c8 3fff15f4 40203990  
3fff0870:  3ffe8e78 56c5cc12 40107180 56c5cc12  
3fff0880:  3ffe851c 3ffef5c8 3ffef8cc 402022d5  
3fff0890:  00000010 3fff08f0 3fff08f0 4020498f  
3fff08a0:  3ffe8d38 00000004 3fff08f0 402049db  
3fff08b0:  3ffeef80 5adf08d8 3fff08f0 40204a0d  
3fff08c0:  3fff08f0 00000000 3fff08f0 40204a5a  
3fff08d0:  3ffe8d38 00001388 00001388 3ffef8f8  
3fff08e0:  3fffdad0 00000000 3ffef8f0 4020240e  
3fff08f0:  3fff1dbc 0000000f 00000004 40204e4d  
3fff0900:  3fffdad0 00000000 3ffef8f0 40204e78  
3fff0910:  feefeffe feefeffe 3ffef900 40100710  
<<<stack<<<

 ets Jan  8 2013,rst cause:2, boot mode:(1,6)


 ets Jan  8 2013,rst cause:4, boot mode:(1,6)

wdt reset

[Mark8869]

UPDATE:
I tried to monitor free heap size, and as I was suspecting, every loop it decreases, so I think that this the cause of fatal exception 29.
But... I'm not able to understand why happens this, and above all how the problem can be fixed.

Thank you in advance all those who want to get interested.

EDIT:
I noticed that are the functions client.connect() and client.verifyCertChain() to reduce heap size every loop.
Removing client.verifyCertChain() from loop and moving it to setup then the client.print() still works (but not sure about connection encryption...), but removing also client.connect() then it can't work...

EDIT2:
I tried to monitor free heap size looping with normal WiFi.Client (not secure) and I noted that heap size start to decrease from about 45000 byte to about 26000 byte, then it stops to decrease, remaining almost costant every loop.
I don't know why but maybe it could be significant for troubleshooting.

[devyte]

I understand that there was a memleak fixed with WiFiCLient|Secure after 2.4.1 release, which could be relevant.
You have marked in the issue checklist above that you tested with latest master branch (aka latest git), and that the issue is still present. Is that really true?

[Mark8869]

I'm using latest release 2.4.1.

[devyte]

BearSSL is merged in #4273 , with alternate BearSSL::WiFi* classes. Although axtls-based classes are still available and even the default, they are planned for deprecation and then retirement, hence won't be fixed. Any issues with BearSSL-based classes should be reported in new issues.
Also, there have been several bug fixes related to connection stability merged post 2.4.1. There is a reason why the issue template has a checkbox that says:

I have tested that the issue is present in current master branch (aka latest git).

which you have ticked.
Closing.