Generation of Self signed certificate do the core dump

[robertgregor]

Hi,
I wanted to generate Self Signed Certificate, but it seems do always the core dump even if the key size is so small i.e. 1024:

I have ESP32 1.0.4. My code is based from the example:

        LogTool::logSketch('V', PSTR("RemoteHome::processWebServerHandler"), PSTR("Access allowed for /gencert"));
        SSLCert * cert = new SSLCert();
        int createCertResult = createSelfSignedCert(
            *cert,
            KEYSIZE_1024,
            "CN=BobTestCa,O=FancyCompany,C=DE",
            "20200101000000",
            "20300101000000"
        );
        preferences.putBytes("certdata", cert->getCertData(),cert->getCertLength());
        preferences.putBytes("pkeydata", cert->getPKData(),cert->getPKLength());

But it do core dump:

2021-01-14 17:24:26 I (42592) HttpServer: Request: GET /gencert?=1610640995561 (FID=57)
2021-01-14 17:24:27 I (42862) RemoteHome::processWebServerHandler: Calling /gencert?=1610640995561
Guru Meditation Error: Core 1 panic'ed (Unhandled debug exception)
Debug exception reason: Stack canary watchpoint triggered (loopTask)
Core 1 register dump:
PC : 0x400fd520 PS : 0x00060036 A0 : 0x800fe7e7 A1 : 0x3ffca460
A2 : 0x00000006 A3 : 0x00000020 A4 : 0x000000cd A5 : 0x00000100
A6 : 0x3ffca5cc A7 : 0x0000000a A8 : 0x000000ca A9 : 0x00000000
A10 : 0x0000002a A11 : 0x3ffca47c A12 : 0x3ffca478 A13 : 0x3ffca4ec
A14 : 0x00000040 A15 : 0x00000006 SAR : 0x00000016 EXCCAUSE: 0x00000001
EXCVADDR: 0x00000000 LBEG : 0x4000c46c LEND : 0x4000c477 LCOUNT : 0x00000000

Backtrace: 0x400fd520:0x3ffca460 0x400fe7e4:0x3ffca4a0 0x400fe8bd:0x3ffca4c0 0x4010d7ab:0x3ffca4f0 0x4010c8bf:0x3ffca520 0x40109944:0x3ffca550 0x400e633d:0x3ffcb1f0 0x400e657a:0x3ffcb580 0x400e0b06:0x3ffcb8a0 0x400e1176:0x3ffcbf10 0x401aacea:0x3ffcc140 0x400e2582:0x3ffcc160 0x400e30f2:0x3ffcc190 0x400e316e:0x3ffcc1b0 0x400e34c1:0x3ffcc210 0x400e30f2:0x3ffcc250 0x400e3c81:0x3ffcc270 0x400e5432:0x3ffcc470 0x400e1ff8:0x3ffcc4a0 0x400d51f5:0x3ffcc4d0 0x400ede35:0x3ffcc510 0x4008b921:0x3ffcc530

Exception decoder is saying:

PC: 0x400fd520: rsa_rsassa_pkcs1_v15_encode at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/mbedtls/mbedtls/library/rsa.c line 1699
EXCVADDR: 0x00000000

Decoding stack results
0x400fd520: rsa_rsassa_pkcs1_v15_encode at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/mbedtls/mbedtls/library/rsa.c line 1699
0x400fe7e4: mbedtls_rsa_rsassa_pkcs1_v15_sign at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/mbedtls/mbedtls/library/rsa.c line 1773
0x400fe8bd: mbedtls_rsa_pkcs1_sign at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/mbedtls/mbedtls/library/rsa.c line 1839
0x4010d7ab: rsa_sign_wrap at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/mbedtls/mbedtls/library/pk_wrap.c line 118
0x4010c8bf: mbedtls_pk_sign at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/mbedtls/mbedtls/library/pk.c line 265
0x40109944: mbedtls_x509write_crt_der at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/mbedtls/mbedtls/library/x509write_crt.c line 425
0x400e633d: httpsserver::cert_write(httpsserver::SSLCert&, std::__cxx11::string, std::__cxx11::string, std::__cxx11::string) at C:\RH\Sources\RH\RemoteHomeArduinoSketches\WiFi_ESP32\libraries\esp32_https_server\src\SSLCert.cpp line 247
0x400e657a: httpsserver::createSelfSignedCert(httpsserver::SSLCert&, httpsserver::SSLKeySize, std::__cxx11::basic_string , std::allocator >, std::__cxx11::basic_string , std::allocator >, std::__cxx11::basic_string , std::allocator >) at C:\RH\Sources\RH\RemoteHomeArduinoSketches\WiFi_ESP32\libraries\esp32_https_server\src\SSLCert.cpp line 297
0x400e0b06: RemoteHome::processWebServerHandler(httpsserver::HTTPRequest*, httpsserver::HTTPResponse*) at C:\RH\Sources\RH\RemoteHomeArduinoSketches\WiFi_ESP32\libraries\RemoteHome\src\RemoteHome.cpp line 740
0x400e1176: RemoteHome::handleSPIFFS(httpsserver::HTTPRequest*, httpsserver::HTTPResponse*) at C:\RH\Sources\RH\RemoteHomeArduinoSketches\WiFi_ESP32\libraries\RemoteHome\src\RemoteHome.cpp line 445
0x401aacea: std::_Function_handler (RemoteHome*, std::_Placeholder1>, std::_Placeholder2>)> >::_M_invoke(std::_Any_data const&, httpsserver::HTTPRequest*&&, httpsserver::HTTPResponse*&&) at c:\users\gregorro\appdata\local\arduino15\packages\esp32\tools\xtensa-esp32-elf-gcc\1.22.0-80-g6c4433a-5.2.0\xtensa-esp32-elf\include\c++\5.2.0/functional line 600
0x400e2582: std::_Function_handler (httpsserver::HTTPRequest*, httpsserver::HTTPResponse*)> >::_M_invoke(std::_Any_data const&) at c:\users\gregorro\appdata\local\arduino15\packages\esp32\tools\xtensa-esp32-elf-gcc\1.22.0-80-g6c4433a-5.2.0\xtensa-esp32-elf\include\c++\5.2.0/functional line 2271
0x400e30f2: std::function ::operator()() const at c:\users\gregorro\appdata\local\arduino15\packages\esp32\tools\xtensa-esp32-elf-gcc\1.22.0-80-g6c4433a-5.2.0\xtensa-esp32-elf\include\c++\5.2.0/functional line 2271
0x400e316e: httpsserver::validationMiddleware(httpsserver::HTTPRequest*, httpsserver::HTTPResponse*, std::function ) at C:\RH\Sources\RH\RemoteHomeArduinoSketches\WiFi_ESP32\libraries\esp32_https_server\src\HTTPConnection.cpp line 641
0x400e34c1: std::_Function_handler ))(httpsserver::HTTPRequest*, httpsserver::HTTPResponse*, std::function )> >::_M_invoke(std::_Any_data const&) at c:\users\gregorro\appdata\local\arduino15\packages\esp32\tools\xtensa-esp32-elf-gcc\1.22.0-80-g6c4433a-5.2.0\xtensa-esp32-elf\include\c++\5.2.0/functional line 1074
0x400e30f2: std::function ::operator()() const at c:\users\gregorro\appdata\local\arduino15\packages\esp32\tools\xtensa-esp32-elf-gcc\1.22.0-80-g6c4433a-5.2.0\xtensa-esp32-elf\include\c++\5.2.0/functional line 2271
0x400e3c81: httpsserver::HTTPConnection::loop() at C:\RH\Sources\RH\RemoteHomeArduinoSketches\WiFi_ESP32\libraries\esp32_https_server\src\HTTPConnection.cpp line 510
0x400e5432: httpsserver::HTTPServer::loop() at C:\RH\Sources\RH\RemoteHomeArduinoSketches\WiFi_ESP32\libraries\esp32_https_server\src\HTTPServer.cpp line 122
0x400e1ff8: RemoteHome::loop() at C:\RH\Sources\RH\RemoteHomeArduinoSketches\WiFi_ESP32\libraries\RemoteHome\src\RemoteHome.cpp line 1630
0x400d51f5: loop() at C:\RH\Sources\RH\RemoteHomeArduinoSketches\WiFi_ESP32\Sensor/Sensor.ino line 700
0x400ede35: loopTask(void*) at C:\Users\gregorro\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4\cores\esp32\main.cpp line 19
0x4008b921: vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c line 143

[robertgregor]

I see, what is the problem. The call has to be done from static context - like from the Sketch directly. Then it works. If it is called from the class, which is instantiated (like I am doing), then there is this issue.
Seems like, the problem is in IDF, not in your library.
BTW, your library is working much better that the build in Web server. It is faster and more stable and it supports multiple sockets at the same time. You must discuss with the people from official arduino esp32 project to include it to the main arduino esp32 toolchain 👍

[proddy]

BTW, your library is working much better that the build in Web server. It is faster and more stable and it supports multiple sockets at the same time. You must discuss with the people from official arduino esp32 project to include it to the main arduino esp32 toolchain 👍

I'm thinking of porting one of my projects from AsyncWebServer to esp32_https_server. Has anyone done a comparison (other than the self-cert HTTPS)?

[robertgregor]

Hi, yes, I already did that. This library is much faster and mainly supports several request at the same time. So if you are loading in the webpage i.e. css, js, pictures and so on, it was failing to load with the asyncwebserver. And perfectly works with this library...

[fhessel]

The memory issue during certificate generation has popped up multiple times by now (cf. #48 which also has a workaround by using a separate task with a huge stack). I couldn't find a single root cause for it though, because debugging through the OpenSSL interface and mbedtls code is still hard, and addressing issues there even more so (as those are modules of the esp-idf, which on its own is only a part of the Arduino Core). There are also two limits which could be exceeded: on the stack and the heap. And for internal idf module code, I cannot chose where things get allocated.

Anyhow, it would be at least good to a point where this function fails gracefully and returns some "out of memory" error instead of just panic'ing.

BTW, your library is working much better that the build in Web server. It is faster and more stable and it supports multiple sockets at the same time. You must discuss with the people from official arduino esp32 project to include it to the main arduino esp32 toolchain 👍

Thanks for the kind feedback. I'm not sure though if it makes sense to approach the developers of the Arduino Core regarding integration. While I focused on multi-socket operation, TLS integration etc., that also comes with a larger sketch size and memory footprint. Also there is this std::string / String issue, not being able to provide WiFiClient objects for the connection, usage of pointers and dynamic allocation and so on, all of which contradicts the very beginner-friendly Arduino styleguide (intentionally, though). From my personal view, I'd prefer a smaller core and more functionality in libraries, because that allows you to tune your application better and keep unused features away from it. An with the lib being listed in Arduino's and PlatformIO's library registry, it should be fairly easy to integrate.

Has anyone done a comparison (other than the self-cert HTTPS)?

There was an ESP32 Arduino web server performance comparison topic on reddit, doesn't include this server, but maybe you can adapt their scripts if you want to get some numbers for your own. In case you do it, I'd be very interested in the results, too :)