[esp_http_client]: underlying transport errors are not captured (IDFGH-15609)

[bryghtlabs-richard]

Answers checklist.

• I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
• I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
• I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.

v6.0-dev-774-g68b583f30c

Espressif SoC revision.

ESP32-S3 (QFN56) (revision v0.2)

Operating System used.

Windows

How did you build your project?

Command line with idf.py

If you are using Windows, please specify command line type.

CMD

Development Kit.

esp32-s3-wroom-1 n8r8

Power Supply used.

USB

What is the expected behavior?

Transport errors resulting in HTTP_EVENT_DISCONNECTED should bubble up through the protocol stack

What is the actual behavior?

Instead, esp_tls_get_and_clear_last_error() and errno sometimes both return zero, so the disconnect is then nearly undebuggable.

Steps to reproduce.

1. Setup a Linux-based router
2. Setup iptables to TCP-RST periodically. I used iptables -F; iptables -A FORWARD -p tcp --dport 443 -m statis tic --mode nth --every 11 --packet 1 -j REJECT --reject-with tcp-reset
3. Monitor console output of examples/protocols/esp_http_client - may need to run it a few times to drop TCP mid-request

Often mid-request TCP connection loss looks like this:

E (48961) transport_base: poll_write select error 104, errno = Connection reset by peer, fd = 55
W (48961) transport_base: Poll timeout or error, errno=Success, fd=55, timeout_ms=5000
E (48971) HTTP_CLIENT: Error write request
I (48971) HTTP_CLIENT: HTTP_EVENT_DISCONNECTED
^ esp_tls_get_and_clear_last_error() returns zero - no way to get 104 error code.
I (48981) HTTP_CLIENT: errno: 0x0
^ I also printed errno - also zero - no way to get 104 error code.
E (48981) HTTP_CLIENT: HTTP request failed: ESP_ERR_HTTP_WRITE_DATA
E (32856) HTTP_CLIENT: HTTP request failed: ESP_ERR_HTTP_WRITE_DATA

Debug Logs.

I (551) example_connect: Waiting for IP(s)
I (3041) wifi:new:<1,0>, old:<1,0>, ap:<255,255>, sta:<1,0>, prof:1, snd_ch_cfg:0x0
I (3041) wifi:state: init -> auth (0xb0)
I (3051) wifi:state: auth -> assoc (0x0)
I (3051) wifi:state: assoc -> run (0x10)
I (3071) wifi:connected with BryghtTest, aid = 2, channel 1, BW20, bssid = c8:d3:a3:5a:33:c7
I (3071) wifi:security: WPA2-PSK, phy: bgn, rssi: -27
I (3071) wifi:pm start, type: 1

I (3071) wifi:dp: 1, bi: 102400, li: 3, scale listen interval from 307200 us to 307200 us
I (3081) wifi:set rx beacon pti, rx_bcn_pti: 0, bcn_timeout: 25000, mt_pti: 0, mt_time: 10000
I (3091) wifi:dp: 2, bi: 102400, li: 4, scale listen interval from 307200 us to 409600 us
I (3101) wifi:AP's beacon interval = 102400 us, DTIM period = 2
I (4121) esp_netif_handlers: example_netif_sta ip: 192.168.5.100, mask: 255.255.255.0, gw: 192.168.5.1
I (4121) example_connect: Got IPv4 event: Interface "example_netif_sta" address: 192.168.5.100
I (4411) example_connect: Got IPv6 event: Interface "example_netif_sta" address: fe80:0000:0000:0000:deda:0cff:fe29:d584, type: ESP_IP6_ADDR_IS_LINK_LOCAL
I (4411) example_common: Connected to example_netif_sta
I (4411) example_common: - IPv4 address: 192.168.5.100,
I (4421) example_common: - IPv6 address: fe80:0000:0000:0000:deda:0cff:fe29:d584, type: ESP_IP6_ADDR_IS_LINK_LOCAL
I (4431) HTTP_CLIENT: Connected to AP, begin http example
I (4431) HTTP_CLIENT: HTTP request with url =>
I (4441) main_task: Returned from app_main()
I (4441) wifi:<ba-add>idx:0 (ifx:0, c8:d3:a3:5a:33:c7), tid:0, ssn:1, winSize:64
I (5751) HTTP_CLIENT: HTTP GET Status = 200, content_length = 263
I (5751) HTTP_CLIENT: 7b 0a 20 20 22 61 72 67 73 22 3a 20 7b 0a 20 20
I (5751) HTTP_CLIENT: 20 20 22 65 73 70 22 3a 20 22 22 0a 20 20 7d 2c
I (5751) HTTP_CLIENT: 20 0a 20 20 22 68 65 61 64 65 72 73 22 3a 20 7b
I (5761) HTTP_CLIENT: 0a 20 20 20 20 22 48 6f 73 74 22 3a 20 22 68 74
I (5771) HTTP_CLIENT: 74 70 62 69 6e 2e 6f 72 67 22 2c 20 0a 20 20 20
I (5771) HTTP_CLIENT: 20 22 55 73 65 72 2d 41 67 65 6e 74 22 3a 20 22
I (5781) HTTP_CLIENT: 45 53 50 33 32 20 48 54 54 50 20 43 6c 69 65 6e
I (5781) HTTP_CLIENT: 74 2f 31 2e 30 22 2c 20 0a 20 20 20 20 22 58 2d
I (5791) HTTP_CLIENT: 41 6d 7a 6e 2d 54 72 61 63 65 2d 49 64 22 3a 20
I (5801) HTTP_CLIENT: 22 52 6f 6f 74 3d 31 2d 36 38 36 35 34 36 36 36
I (5801) HTTP_CLIENT: 2d 30 62 38 39 63 66 38 39 30 30 61 33 30 34 36
I (5811) HTTP_CLIENT: 35 32 37 32 38 30 66 30 63 22 0a 20 20 7d 2c 20
I (5821) HTTP_CLIENT: 0a 20 20 22 6f 72 69 67 69 6e 22 3a 20 22 31 33
I (5821) HTTP_CLIENT: 36 2e 33 37 2e 31 36 38 2e 32 32 22 2c 20 0a 20
I (5831) HTTP_CLIENT: 20 22 75 72 6c 22 3a 20 22 68 74 74 70 3a 2f 2f
I (5831) HTTP_CLIENT: 68 74 74 70 62 69 6e 2e 6f 72 67 2f 67 65 74 3f
I (5841) HTTP_CLIENT: 65 73 70 22 0a 7d 0a
I (6151) HTTP_CLIENT: HTTP POST Status = 200, content_length = 423
I (7411) example_connect: Got IPv6 event: Interface "example_netif_sta" address: fd0a:3515:e5b1:0000:deda:0cff:fe29:d584, type: ESP_IP6_ADDR_IS_UNIQUE_LOCAL
I (7791) HTTP_CLIENT: HTTP PUT Status = 502, content_length = 122
I (10871) HTTP_CLIENT: HTTP PATCH Status = 200, content_length = 335
I (12501) HTTP_CLIENT: HTTP DELETE Status = 200, content_length = 308
I (14141) HTTP_CLIENT: HTTP HEAD Status = 200, content_length = 242
I (14141) HTTP_CLIENT: HTTP_EVENT_DISCONNECTED
I (14141) HTTP_CLIENT: errno: 0x0
I (14141) HTTP_CLIENT: HTTP request with hostname and path =>
I (14551) HTTP_CLIENT: HTTP GET Status = 200, content_length = 242
I (16601) HTTP_CLIENT: HTTP POST Status = 200, content_length = 444
I (17011) HTTP_CLIENT: HTTP PUT Status = 200, content_length = 443
I (19671) HTTP_CLIENT: HTTP PATCH Status = 200, content_length = 335
I (21311) HTTP_CLIENT: HTTP DELETE Status = 200, content_length = 308
I (21721) HTTP_CLIENT: HTTP HEAD Status = 502, content_length = 122
I (21721) HTTP_CLIENT: HTTP_EVENT_DISCONNECTED
I (21721) HTTP_CLIENT: errno: 0x0
I (21721) HTTP_CLIENT: HTTP GET request with encoded query =>
I (21861) HTTP_CLIENT: HTTP GET Status = 200, content_length = 295
I (21861) HTTP_CLIENT: HTTP Relative path redirect request =>
I (28681) HTTP_CLIENT: HTTP Relative path redirect Status = 200, content_length = 242
I (28681) HTTP_CLIENT: HTTP_EVENT_DISCONNECTED
I (28681) HTTP_CLIENT: errno: 0x0
I (28691) HTTP_CLIENT: HTTP Absolute path redirect request =>
I (31961) HTTP_CLIENT: HTTP Absolute path redirect Status = 200, content_length = 242
I (31961) HTTP_CLIENT: HTTP_EVENT_DISCONNECTED
I (31961) HTTP_CLIENT: errno: 0x0
I (31971) HTTP_CLIENT: HTTP Absolute path redirect (manual) request =>
I (39331) HTTP_CLIENT: HTTP Absolute path redirect (manual) Status = 200, content_length = 303
I (39331) HTTP_CLIENT: HTTP_EVENT_DISCONNECTED
I (39341) HTTP_CLIENT: errno: 0x0
I (39341) HTTP_CLIENT: HTTPS request with url =>
E (39381) esp-tls-mbedtls: mbedtls_ssl_handshake returned -0x0050
I (39381) esp-tls-mbedtls: Certificate verified.
E (39381) esp-tls: Failed to open new connection
E (39381) transport_base: Failed to open a new connection
E (39391) HTTP_CLIENT: Connection failed, sock < 0
E (39391) HTTP_CLIENT: Error perform http request ESP_ERR_HTTP_CONNECT
I (39401) HTTP_CLIENT: HTTP_EVENT_DISCONNECTED
I (39401) HTTP_CLIENT: Last esp error code: 0x801a
I (39411) HTTP_CLIENT: Last mbedtls failure: 0x50
I (39411) HTTP_CLIENT: errno: 0x80
I (39421) HTTP_CLIENT: HTTPS request with hostname and path =>
E (40221) esp-tls-mbedtls: read error :-0x0050
E (40221) transport_base: esp_tls_conn_read error, errno=Connection reset by peer
I (40221) HTTP_CLIENT: HTTP_EVENT_DISCONNECTED
I (40221) HTTP_CLIENT: errno: 0x68
I (40231) HTTP_CLIENT: HTTPS Status = 200, content_length = 12333
I (40231) HTTP_CLIENT: HTTP_EVENT_DISCONNECTED
I (40241) HTTP_CLIENT: errno: 0x80
I (40241) HTTP_CLIENT: HTTP redirect to HTTPS request =>
I (40561) HTTP_CLIENT: HTTP_EVENT_DISCONNECTED
I (40561) HTTP_CLIENT: errno: 0x0
I (40561) HTTP_CLIENT: HTTP_EVENT_DISCONNECTED
I (40561) HTTP_CLIENT: errno: 0x0
I (41301) HTTP_CLIENT: HTTP_EVENT_DISCONNECTED
I (41301) HTTP_CLIENT: errno: 0x0
I (41301) HTTP_CLIENT: HTTP redirect to HTTPS Status = 200, content_length = 12333
I (41301) HTTP_CLIENT: HTTP_EVENT_DISCONNECTED
I (41311) HTTP_CLIENT: errno: 0x80
I (41311) HTTP_CLIENT: HTTP chunk encoding request =>
I (42661) HTTP_CLIENT: HTTP chunk encoding Status = 200, content_length = -1
I (42661) HTTP_CLIENT: HTTP_EVENT_DISCONNECTED
I (42661) HTTP_CLIENT: errno: 0x0
I (42661) HTTP_CLIENT: HTTP Stream reader request =>
I (43431) HTTP_CLIENT: HTTP Stream reader Status = 200, content_length = 242
I (43431) HTTP_CLIENT: HTTPS async requests =>
I (44291) HTTP_CLIENT: HTTPS Status = 200, content_length = 1107
I (44291) HTTP_CLIENT: HTTP_EVENT_DISCONNECTED
I (44291) HTTP_CLIENT: errno: 0x0
I (44471) esp-x509-crt-bundle: Certificate validated
I (46711) HTTP_CLIENT: HTTPS Status = 200, content_length = 243
I (46711) HTTP_CLIENT: HTTP_EVENT_DISCONNECTED
I (46711) HTTP_CLIENT: errno: 0x0
I (46711) HTTP_CLIENT: HTTPS request with invalid url =>
E (46721) esp-tls: couldn't get hostname for :not.existent.url: getaddrinfo() returns 202, addrinfo=0x0
E (46721) esp-tls: Failed to open new connection
E (46731) transport_base: Failed to open a new connection
E (46731) HTTP_CLIENT: Connection failed, sock < 0
E (46741) HTTP_CLIENT: Error perform http request ESP_ERR_HTTP_CONNECT
I (46741) HTTP_CLIENT: HTTP_EVENT_DISCONNECTED
I (46751) HTTP_CLIENT: Last esp error code: 0x8001
I (46751) HTTP_CLIENT: Last mbedtls failure: 0x0
I (46761) HTTP_CLIENT: errno: 0x0
I (46761) HTTP_CLIENT: HTTP native request =>
I (47121) HTTP_CLIENT: HTTP GET Status = 200, content_length = 242
I (47121) HTTP_CLIENT: 7b 0a 20 20 22 61 72 67 73 22 3a 20 7b 7d 2c 20
I (47121) HTTP_CLIENT: 0a 20 20 22 68 65 61 64 65 72 73 22 3a 20 7b 0a
I (47121) HTTP_CLIENT: 20 20 20 20 22 48 6f 73 74 22 3a 20 22 68 74 74
I (47131) HTTP_CLIENT: 70 62 69 6e 2e 6f 72 67 22 2c 20 0a 20 20 20 20
I (47141) HTTP_CLIENT: 22 55 73 65 72 2d 41 67 65 6e 74 22 3a 20 22 45
I (47141) HTTP_CLIENT: 53 50 33 32 20 48 54 54 50 20 43 6c 69 65 6e 74
I (47151) HTTP_CLIENT: 2f 31 2e 30 22 2c 20 0a 20 20 20 20 22 58 2d 41
I (47151) HTTP_CLIENT: 6d 7a 6e 2d 54 72 61 63 65 2d 49 64 22 3a 20 22
I (47161) HTTP_CLIENT: 52 6f 6f 74 3d 31 2d 36 38 36 35 34 36 39 30 2d
I (47171) HTTP_CLIENT: 32 38 37 36 31 37 65 34 33 37 65 62 63 65 65 36
I (47171) HTTP_CLIENT: 33 34 38 62 32 33 32 39 22 0a 20 20 7d 2c 20 0a
I (47181) HTTP_CLIENT: 20 20 22 6f 72 69 67 69 6e 22 3a 20 22 31 33 36
I (47191) HTTP_CLIENT: 2e 33 37 2e 31 36 38 2e 32 32 22 2c 20 0a 20 20
I (47191) HTTP_CLIENT: 22 75 72 6c 22 3a 20 22 68 74 74 70 3a 2f 2f 68
I (47201) HTTP_CLIENT: 74 74 70 62 69 6e 2e 6f 72 67 2f 67 65 74 22 0a
I (47201) HTTP_CLIENT: 7d 0a
I (48351) HTTP_CLIENT: HTTP POST Status = 200, content_length = 423
I (48351) HTTP_CLIENT: 7b 0a 20 20 22 61 72 67 73 22 3a 20 7b 7d 2c 20
I (48351) HTTP_CLIENT: 0a 20 20 22 64 61 74 61 22 3a 20 22 7b 5c 22 66
I (48351) HTTP_CLIENT: 69 65 6c 64 31 5c 22 3a 5c 22 76 61 6c 75 65 31
I (48361) HTTP_CLIENT: 5c 22 7d 22 2c 20 0a 20 20 22 66 69 6c 65 73 22
I (48371) HTTP_CLIENT: 3a 20 7b 7d 2c 20 0a 20 20 22 66 6f 72 6d 22 3a
I (48371) HTTP_CLIENT: 20 7b 7d 2c 20 0a 20 20 22 68 65 61 64 65 72 73
I (48381) HTTP_CLIENT: 22 3a 20 7b 0a 20 20 20 20 22 43 6f 6e 74 65 6e
I (48381) HTTP_CLIENT: 74 2d 4c 65 6e 67 74 68 22 3a 20 22 31 39 22 2c
I (48391) HTTP_CLIENT: 20 0a 20 20 20 20 22 43 6f 6e 74 65 6e 74 2d 54
I (48401) HTTP_CLIENT: 79 70 65 22 3a 20 22 61 70 70 6c 69 63 61 74 69
I (48401) HTTP_CLIENT: 6f 6e 2f 6a 73 6f 6e 22 2c 20 0a 20 20 20 20 22
I (48411) HTTP_CLIENT: 48 6f 73 74 22 3a 20 22 68 74 74 70 62 69 6e 2e
I (48421) HTTP_CLIENT: 6f 72 67 22 2c 20 0a 20 20 20 20 22 55 73 65 72
I (48421) HTTP_CLIENT: 2d 41 67 65 6e 74 22 3a 20 22 45 53 50 33 32 20
I (48431) HTTP_CLIENT: 48 54 54 50 20 43 6c 69 65 6e 74 2f 31 2e 30 22
I (48431) HTTP_CLIENT: 2c 20 0a 20 20 20 20 22 58 2d 41 6d 7a 6e 2d 54
I (48441) HTTP_CLIENT: 72 61 63 65 2d 49 64 22 3a 20 22 52 6f 6f 74 3d
I (48451) HTTP_CLIENT: 31 2d 36 38 36 35 34 36 39 31 2d 35 61 61 64 32
I (48451) HTTP_CLIENT: 38 31 65 32 30 31 34 35 38 38 37 34 30 32 37 35
I (48461) HTTP_CLIENT: 66 34 66 22 0a 20 20 7d 2c 20 0a 20 20 22 6a 73
I (48471) HTTP_CLIENT: 6f 6e 22 3a 20 7b 0a 20 20 20 20 22 66 69 65 6c
I (48471) HTTP_CLIENT: 64 31 22 3a 20 22 76 61 6c 75 65 31 22 0a 20 20
I (48481) HTTP_CLIENT: 7d 2c 20 0a 20 20 22 6f 72 69 67 69 6e 22 3a 20
I (48481) HTTP_CLIENT: 22 31 33 36 2e 33 37 2e 31 36 38 2e 32 32 22 2c
I (48491) HTTP_CLIENT: 20 0a 20 20 22 75 72 6c 22 3a 20 22 68 74 74 70
I (48501) HTTP_CLIENT: 3a 2f 2f 68 74 74 70 62 69 6e 2e 6f 72 67 2f 70
I (48501) HTTP_CLIENT: 6f 73 74 22 0a 7d 0a
I (48511) HTTP_CLIENT: HTTP partial download =>
I (48651) esp-x509-crt-bundle: Certificate validated
I (48961) HTTP_CLIENT: HTTP Status = 206, content_length = 69
E (48961) transport_base: poll_write select error 104, errno = Connection reset by peer, fd = 55
W (48961) transport_base: Poll timeout or error, errno=Success, fd=55, timeout_ms=5000
E (48971) HTTP_CLIENT: Error write request
I (48971) HTTP_CLIENT: HTTP_EVENT_DISCONNECTED
I (48981) HTTP_CLIENT: errno: 0x0
E (48981) HTTP_CLIENT: HTTP request failed: ESP_ERR_HTTP_WRITE_DATA
I (49061) esp-x509-crt-bundle: Certificate validated
E (49351) esp-tls-mbedtls: read error :-0x0050
E (49351) transport_base: esp_tls_conn_read error, errno=Connection reset by peer
E (49351) HTTP_CLIENT: HTTP request failed: ESP_ERR_HTTP_FETCH_HEADER
I (49351) HTTP_CLIENT: HTTP_EVENT_DISCONNECTED
I (49361) HTTP_CLIENT: errno: 0x68
I (49361) HTTP_CLIENT: Finish http example

Diagnostic report archive.

idf-diag-a52f1a5b-0208-4f32-9721-659575abe72f.zip

More Information.

Also impacts 5.1.2, 5.1.6, 5.5

[nileshkale123]

Hello @bryghtlabs-richard ,

I have tested and verified the issue on my end based on your suggested reproduction steps.

Setup:

1. Modified iptables to send a TCP reset for every 5th packet using the command: sudo iptables -A OUTPUT -p tcp --dport 443 -m statistic --mode nth --every 5 --packet 1 -j REJECT --reject-with tcp-reset
2. Executed esp_http_client_example.c’s http_partial_download() test in a loop (20 iterations) to cover all possible scenarios.
3. Added errno print just after read/write error in esp_tls_mbedtls.c

Observation:
I consistently observed errno = 104 (Connection reset by peer) whenever the corresponding error was triggered.

read error logs
E (1292067324) esp-tls-mbedtls: read error :-0x0050 E (1292067324) esp-tls-mbedtls: in esp_mbedtls_read errno: 104 (Connection reset by peer) E (1292067324) transport_base: esp_tls_conn_read error, errno=Connection reset by peer

write error logs
E (1292067186) esp-tls-mbedtls: write error :-0x0050 E (1292067186) esp-tls-mbedtls: in esp_mbedtls_write errno: 104 (Connection reset by peer) E (1292067186) transport_base: esp_tls_conn_write error, errno=Connection reset by peer

Recommendation:
I suggest you try this approach on your side to re-verify the behavior. I believe it will address your query.

[bryghtlabs-richard]

Observation: I consistently observed errno = 104 (Connection reset by peer) whenever the corresponding error was triggered.

I see that in the logs, but I think I want esp_tls_get_and_clear_last_error() to return the error. I'll attach my stripped down example code.

When I reset all TCP with iptables -A FORWARD -p tcp --dport 443 -j REJECT --reject-with tcp-reset, then attempt to connect:

I (7435) HTTP_CLIENT: HTTPS async requests =>
I (7445) main_task: Returned from app_main()
I (7465) wifi:<ba-add>idx:0 (ifx:0, c8:d3:a3:5a:33:c7), tid:0, ssn:1, winSize:64
E (7485) esp-tls-mbedtls: mbedtls_ssl_handshake returned -0x004E
I (7485) esp-tls-mbedtls: Certificate verified.
^^^Separately, this is a very strange thing to say when it could not download the certificate.
E (7485) HTTP_CLIENT: Connection failed
I (7485) HTTP_CLIENT: HTTP_EVENT_ERROR
E (7495) HTTP_CLIENT: Error perform http request ESP_ERR_HTTP_CONNECT
I (7495) HTTP_CLIENT: HTTP_EVENT_DISCONNECTED
I (7505) HTTP_CLIENT: esp_tls_get_and_clear_last_error() returned ESP_OK
^^^Why did it not return the underlying cause of the connection error?
I (7505) HTTP_CLIENT: Finish http example
I (11415) example_connect: Got IPv6 event: Interface "example_netif_sta" address: fd0a:3515:e5b1:0000:3e84:27ff:fec7:127c, type: ESP_IP6_ADDR_IS_UNIQUE_LOCAL

When I connect to HTTPS server and begin download, then once download begins kill all TCP with iptables -A FORWARD -p tcp --dport 443 -j REJECT --reject-with tcp-reset:

I (18765) HTTP_CLIENT: HTTP_EVENT_ON_DATA, len=512
I (18775) HTTP_CLIENT: HTTP_EVENT_ON_DATA, len=512
I (18785) HTTP_CLIENT: HTTP_EVENT_ON_DATA, len=512
E (18795) esp-tls-mbedtls: read error :-0x0050
E (18795) transport_base: esp_tls_conn_read error, errno=Connection reset by peer
I (18795) HTTP_CLIENT: HTTP_EVENT_ON_FINISH
^^^Why did it send finish-event? This download did not finish.
I (18795) HTTP_CLIENT: HTTPS Status = 200, content_length = 3901456384
I (18805) HTTP_CLIENT: HTTP_EVENT_DISCONNECTED
I (18805) HTTP_CLIENT: esp_tls_get_and_clear_last_error() returned ESP_OK
^^^Why did it not return the underlying cause of the connection error?
I (18815) HTTP_CLIENT: Finish http example

I also tried adding #16194 , but spurious HTTP_EVENT_ON_FINISH still occurs and esp_tls_get_and_clear_last_error() still returns ESP_OK when TCP-RST in the middle of a response download. Should I make another ticket for spurious HTTP_EVENT_ON_FINISH ?

[bryghtlabs-richard]

Attach example client with long-running HTTPS download esp_http_client_example.c.txt

[nileshkale123]

Hello @bryghtlabs-richard,

I (7505) HTTP_CLIENT: esp_tls_get_and_clear_last_error() returned ESP_OK

I got your point. We have raised a fix to add a missing event tracker during failure in the mbedtls read operation. With this, you will get an error captured after the call to esp_tls_get_and_clear_last_error().
patch_issue_16239.txt

^^^Separately, this is a very strange thing to say when it could not download the certificate.

Also, as we will remove the confusing and unnecessary log of certificate verification, because this API got called at line to check if there is an error in certificate verification, as per the comment. Also, I have checked the Wireshark trace, and as per this, I was able to see the client hello and server hello phase completed successfully, which meant the client has a certificate at its end. I request you to confirm if this is the same happening at your end.

[bryghtlabs-richard]

We have raised a fix to add a missing event tracker during failure in the mbedtls read operation. With this, you will get an error captured after the call to esp_tls_get_and_clear_last_error().

Thanks Nilesh!

[Richard] ^^^Separately, this is a very strange thing to say when it could not download the certificate.

[Nilesh] Also, as we will remove the confusing and unnecessary log of certificate verification, because this API got called at line to check if there is an error in certificate verification, as per the comment. Also, I have checked the Wireshark trace, and as per this, I was able to see the client hello and server hello phase completed successfully, which meant the client has a certificate at its end. I request you to confirm if this is the same happening at your end.

Sorry, I don't have access to those code servers. But in my case, this log was printed even with router rejecting all TCP:443 connections, iptables -A FORWARD -p tcp --dport 443 -j REJECT --reject-with tcp-reset. (you may need to adjust your chain) I have confirmed with wireshark that a TCP connection is not established under these conditions, so client-hello and server-hello do not occur either, but "Certificate verified" is still printed.

[nileshkale123]

Hello @bryghtlabs-richard ,

I have reverified once again.

Setup:

1. Target - linux
2. Blocking all packets using sudo iptables -A OUTPUT -p tcp -m tcp --dport 443 -j REJECT --reject-with tcp-reset
3. Executing subtask http_partial_download() in esp_http_client_example.c

Observation:

1. With this, I am not getting any log for 'certificate verified'.

NOTE:

Also, as we will remove the confusing and unnecessary log of certificate verification, because this API got called at line to check if there is an error in certificate verification, as per the comment. Also, I have checked the Wireshark trace, and as per this, I was able to see the client hello and server hello phase completed successfully, which meant the client has a certificate at its end. I request you to confirm if this is the same happening at your end.

Also, I am attaching patch to remove this unnecessary print. Please verify if you still getting these logs even with these changes.

patch_print_remove.txt

If the issue is still persists, please tell me your testing setup information in detail.

[bryghtlabs-richard]

I think this may be a deeper problem than an extra printout.

Also, I am attaching patch to remove this unnecessary print. Please verify if you still getting these logs even with these changes.

With that change I do not see "Certificate verified.", but I don't see "Failed to verify peer certificate" either! Please consider:

void esp_mbedtls_verify_certificate(esp_tls_t *tls)
{
    int flags;

    if ((flags = mbedtls_ssl_get_verify_result(&tls->ssl)) != 0) {
        ESP_LOGI(TAG, "Failed to verify peer certificate!");
        ESP_INT_EVENT_TRACKER_CAPTURE(tls->error_handle, ESP_TLS_ERR_TYPE_MBEDTLS_CERT_FLAGS, flags);
#if (CONFIG_LOG_DEFAULT_LEVEL_DEBUG || CONFIG_LOG_DEFAULT_LEVEL_VERBOSE)
        char buf[100];
        bzero(buf, sizeof(buf));
        mbedtls_x509_crt_verify_info(buf, sizeof(buf), "  ! ", flags);
        ESP_LOGD(TAG, "verification info: %s", buf);
#endif
    } else {
        ESP_LOGI(TAG, "Certificate verified.");
    }
}

If mbedtls_ssl_get_verify_result(&tls->ssl)) is returning 0 without downloading a server certificate, I believe this is wrong, and may be risky in other situations too. Instead, I believe it should print "Failed to verify peer certificate!". TLS security should not fail open.

Perhaps we need to invalidate something in mbedtls before the connection starts?
Perhaps we should not call esp_mbedtls_verify_certificate without download a certificate?
But if we call esp_mbedtls_verify_certificate() without downloading server-certificate, it must not succeed.

If the issue is still persists, please tell me your testing setup information in detail.

1. Target=ESP32-S3, WiFi router running Linux
2. TCP blocked at access point with iptables -A FORWARD -p tcp --dport 443 -j REJECT --reject-with tcp-reset
3. Executing subtask http_partial_download() in esp_http_client_example.c

[nileshkale123]

Hello @bryghtlabs-richard ,

I have reviewed the code flow and agree that the issue you mentioned is valid. A good way to confirm whether this is a certificate-related problem would be to check if the client is able to fetch the server certificate. What do you think?

I have updated the patch accordingly. Please take a look and let us know if you have any suggestions. Thanks for your patience and support.

patch_issue_16239.txt

[bryghtlabs-richard]

This patch looks good to me. Thank you for your patience and for taking my concerns seriously @nileshkale123 !