No result from github https query

[glennswest]

Describe the bug

Doing a query to a github api endpoint (Publically available, and works from curl), and get no response.

Steps to Reproduce

Run app(i2cgateha) on m5stack esp32 (Which has working mqttasync and webserver), query returns no result.
App github: https://github.com/glennswest/i2cgateha
Debug Text: https://raw.githubusercontent.com/glennswest/i2cgateha/main/problem.txt

Steps to reproduce the behavior. Including the MRE sketches
Run above program

Expected behavior

Returns:
Print something like:
[
{
"name": "index.html",
"path": "contents/index.html",
"sha": "9e52837f62962c18c7e214a7e1a0d70882a1c726",
"size": 11,
"url": "https://api.github.com/repos/glennswest/i2cgateha/contents/contents/index.html?ref=main",
"html_url": "https://github.com/glennswest/i2cgateha/blob/main/contents/index.html",
"git_url": "https://api.github.com/repos/glennswest/i2cgateha/git/blobs/9e52837f62962c18c7e214a7e1a0d70882a1c726",
"download_url": "https://raw.githubusercontent.com/glennswest/i2cgateha/main/contents/index.html",
"type": "file",
"_links": {
"self": "https://api.github.com/repos/glennswest/i2cgateha/contents/contents/index.html?ref=main",
"git": "https://api.github.com/repos/glennswest/i2cgateha/git/blobs/9e52837f62962c18c7e214a7e1a0d70882a1c726",
"html": "https://github.com/glennswest/i2cgateha/blob/main/contents/index.html"
}
}
]

Actual behavior

Returns no data

Debug and AT-command log (if applicable)

See serial log:
https://raw.githubusercontent.com/glennswest/i2cgateha/main/problem.txt

Information

Please ensure to specify the following:
Arduino 1.18.16
Hardware: M5Stack-Paper ESP32

• Contextual information (e.g. what you were trying to achieve)
  Pull a list of files on the directory on github
• Simplest possible steps to reproduce
  Use the supplied url to test with your example code

[glennswest]

Similar result to: "https://worldtimeapi.org/api/timezone/Europe/London.txt"

[glennswest]

When I do the London, example: I see [ATCP] _recv: tot_len = 1211
In the log, indicating a decent respond.

[glennswest]

Looks like data comes back on the github api call:
ATCP] _handle_async_event: Task Name = async_tcp_ssl
[ATCP] _handle_async_event: LWIP_TCP_SENT = 0x3F848928
[ATCP] _sent: len = 303
[ATCP] _handle_async_event: Task Name = async_tcp_ssl
[ATCP] _handle_async_event: LWIP_TCP_RECV = 0x3F848928
[ATCP] _recv: tot_len = 1436
[ATCP] _handle_async_event: Task Name = async_tcp_ssl
[ATCP] _handle_async_event: LWIP_TCP_RECV = 0x3F848928
[ATCP] _recv: tot_len = 1253

[glennswest]

Here's something to make it even more interesting:
https://raw.githubusercontent.com/glennswest/i2cgateha/main/contents/.version
This returns "1" as expected.

[khoih-prog]

Hi @glennswest

Thanks for your interest and usage of the library.

I've just tested and found no issue at all with the libraries, using the AsyncHTTPSRequest_ESP example. Check the following Debug Terminal Output for both

• "https://worldtimeapi.org/api/timezone/America/Toronto.txt"
• "https://worldtimeapi.org/api/timezone/Europe/London.txt"

I suggest you restart rewriting your code, slowly and methodologically, from the above example, then move forward gradually to identify when and where the code is broken.

I'm sorry I certainly don't have all the time in the world to read, test and debug your code. I also don't have the same hardware and/or similar settings (network, ISP, etc.).

As this is not a bug of the library, I'm closing the issue now, and only reopen when you prove this is a bug of this library.

Good Luck,

---

Debug Terminal Outputs

Starting AsyncHTTPSRequest_ESP using ESP32_DEV
AsyncTCP_SSL v1.2.0
AsyncHTTPSRequest_Generic v2.0.0
Connecting to WiFi SSID: HueNet1
........
AsyncHTTPSRequest @ IP : 192.168.2.93

**************************************
abbreviation: EDT
client_ip: 45.72.131.83
datetime: 2022-03-24T13:06:02.807503-04:00
day_of_week: 4
day_of_year: 83
dst: true
dst_from: 2022-03-13T07:00:00+00:00
dst_offset: 3600
dst_until: 2022-11-06T06:00:00+00:00
raw_offset: -18000
timezone: America/Toronto
unixtime: 1648141562
utc_datetime: 2022-03-24T17:06:02.807503+00:00
utc_offset: -04:00
week_number: 12
**************************************
HHH

Starting AsyncHTTPSRequest_ESP using ESP32_DEV
AsyncTCP_SSL v1.2.0
AsyncHTTPSRequest_Generic v2.0.0
Connecting to WiFi SSID: HueNet1
.......
AsyncHTTPSRequest @ IP : 192.168.2.93

**************************************
abbreviation: GMT
client_ip: 45.72.131.83
datetime: 2022-03-24T17:07:16.182664+00:00
day_of_week: 4
day_of_year: 83
dst: false
dst_from: 
dst_offset: 0
dst_until: 
raw_offset: 0
timezone: Europe/London
unixtime: 1648141636
utc_datetime: 2022-03-24T17:07:16.182664+00:00
utc_offset: +00:00
week_number: 12
**************************************

[glennswest]

Actually, the probem is "timing".
request.setTimeout(60);
Solved the problem.
Adding that made everything work.
Suggest you increase the default timeout.

[glennswest]

I've ran it multiple times, and every url I use works now, with debug off.
So Im really happy.
I need to look how to do "chunks", so I can download arbitrary files to sd card.

[khoih-prog]

Thanks for your feedback and I'm glad you found the elusive clue from debug messages

[AHTTPS] _onPoll timeout

Will increase the DEFAULT_RX_TIMEOUT from 3s to reasonable 30s to cope with some slow network.

[glennswest]

Great. Happy I could help.
Im working on a auto update of flash and web content using your library.
Push to github, and device can update itself.
(SD card required of course)

[khoih-prog]

Hi @glennswest

The new AsyncHTTPSRequest_Generic releases v2.0.1 has just been published. Your contribution is noted in Contributions and Thanks

Best Regards,

---

Releases v2.0.1

1. Increase DEFAULT_RX_TIMEOUT to 30s from 3s for slower networks. Check No result from github https query #8

[glennswest]

Wow, thanks.
Made my day.
I love opensource.