assert failed: tcp_update_rcv_ann_wnd (needs help to test fix)

[DaeMonSxy]

It would be good to have an internal crash prevention for asynctcp library as well.
I would appreciate any hints to solve such crashes-and have stable tcp lib also.

assert failed: tcp_update_rcv_ann_wnd IDF/components/lwip/lwip/src/core/tcp.c:951 (new_rcv_ann_wnd <= 0xffff)


Backtrace: 0x40083d65:0x3ffb34d0 0x4008ca75:0x3ffb34f0 0x4009209d:0x3ffb3510 0x40127c06:0x3ffb3640 0x40127cb4:0x3ffb3660 0x400f4662:0x3ffb3680 0x4012498c:0x3ffb36a0
  #0  0x40083d65 in panic_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/panic.c:408
  #1  0x4008ca75 in esp_system_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/esp_system.c:137
  #2  0x4009209d in __assert_func at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/newlib/assert.c:85
  #3  0x40127c06 in tcp_update_rcv_ann_wnd at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/core/tcp.c:951
      (inlined by) tcp_update_rcv_ann_wnd at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/core/tcp.c:931
  #4  0x40127cb4 in tcp_recved at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/core/tcp.c:991
  #5  0x400f4662 in _tcp_recved_api(tcpip_api_call_data*) at /Users/daemons/platformio_libdeps/ESP32/Async TCP/src/AsyncTCP.cpp:444
  #6  0x4012498c in tcpip_thread_handle_msg at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/api/tcpip.c:172
      (inlined by) tcpip_thread at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/api/tcpip.c:154

[mathieucarbou]

Hello,
Please have a look at https://github.com/mathieucarbou/AsyncTCP?tab=readme-ov-file#important-recommendations
And let me know also which board / pio config you are using.

[DaeMonSxy]

theoretically I have it in platformio,

build_flags =
-D CONFIG_ASYNC_TCP_QUEUE_SIZE=128      ; Async TCP queue size
-D CONFIG_ASYNC_TCP_RUNNING_CORE=1      ; Async TCP async task core
-D CONFIG_ASYNC_TCP_STACK_SIZE=8096     ; Async TCP async stac ksize
-D WS_MAX_QUEUED_MESSAGES=64            ; WebSocket queue size
;-D CONFIG_ASYNC_TCP_PRIORITY=10         ; ask priority
;-D CONFIG_ASYNC_TCP_MAX_ACK_TIME=3000   ;

[mathieucarbou]

Seems related to the queue size ?
Ref: me-no-dev#88

It's hard to help you with so little information. How the application is built, uses ressources and how many request / concurrent request you have can all affect that.

If you can pinpoint a bit more the issue, let me know.

[DaeMonSxy]

Well, hard to tell the exact crash location in lib.

Having several modules built in top of asynctcp including webserver.All of those are using asyncTCP,
which part makes asynctcp to crash is a good question, however, I would expect asyncTCP to handle the receive window - heap or stack , or any error may accur -so somehow i would epect the lib internally to be more crash-proof.

[mathieucarbou]

FYI I was able to reproduce today. The issue happens when I am doing some operation in the post request handler which executes on core 1, and is updating some states in ESPAsyncWebserver (websocket in my case).

Typically, I was in a code path, calling websocket to push some data (WebSerial), within some code called from the POST handler of ESPAsyncWebserver.

[mikeg0]

FWIW - I'm also getting the same error assert failed: tcp_update_rcv_ann_wnd IDF/components/lwip/lwip/src/core/tcp.c:951 (new_rcv_ann_wnd <= 0xffff)

I was able to replicate almost 100% of the time with a delay(500) in the server.on event listener ...

    // Route for root / web page
    server.on("/", HTTP_GET, [](AsyncWebServerRequest *request)
    {
        digitalWrite(networkLedPin, HIGH);

        request->send(SPIFFS, "/index.html", "text/html", false);

        delay(500);
        digitalWrite(networkLedPin, LOW);
    });

Basically, I wanted to turn on the LED during a page request and the tcp_update_rcv_ann_wnd error kept showing up.

[mathieucarbou]

@mikeg0 :

• what flags are you using to compile (see end of readme) ?
• your design is not good because besides executing long running code you are blocking the async_tcp task with a delay
• SPIFFS is deprecated in favor of Little FS, and ESPAsyncWebServer has a feature to serve static content with caching to avoid subsequent requests.

In async mode, never ever bock or execute long-running code in a callback coming from another task.
This applies whatever the library used.

Keep the states, run the code from main loop or another loop (except if you know that the code will execute really fast).

[mathieucarbou]

I am reopening because I saw it today. This is really random - got it twice in several months.

Ref also: espressif/arduino-esp32#7895

assert failed: tcp_update_rcv_ann_wnd /IDF/components/lwip/lwip/src/core/tcp.c:957 (new_rcv_ann_wnd <= 0xffff)


Backtrace: 0x40083f01:0x3ffda560 0x4008ed25:0x3ffda580 0x40094622:0x3ffda5a0 0x40134001:0x3ffda6d0 0x4013409a:0x3ffda6f0 0x401c1a9a:0x3ffda710 0x40130778:0x3ffda730 0x40091576:0x3ffda760
  #0  0x40083f01 in panic_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/panic.c:466
  #1  0x4008ed25 in esp_system_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/port/esp_system_chip.c:84
  #2  0x40094622 in __assert_func at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/newlib/assert.c:81
  #3  0x40134001 in tcp_update_rcv_ann_wnd at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/core/tcp.c:957 (discriminator 1)
  #4  0x4013409a in tcp_recved at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/core/tcp.c:997
  #5  0x401c1a9a in _tcp_recved_api(tcpip_api_call_data*) at .pio/libdeps/pro-esp32-debug/AsyncTCP/src/AsyncTCP.cpp:450
  #6  0x40130778 in tcpip_thread_handle_msg at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/api/tcpip.c:166
      (inlined by) tcpip_thread at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/api/tcpip.c:148
  #7  0x40091576 in vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/FreeRTOS-Kernel/portable/xtensa/port.c:162

[mathieucarbou]

@mikeg0 : if you can try this fix ?

branch: https://github.com/mathieucarbou/AsyncTCP/tree/issue-14
pio dep: https://github.com/mathieucarbou/AsyncTCP#issue-14

[mathieucarbou]

I was able to reproduce in a reliable way and then tested the fix and it works, I will merge and issue a new release.

[mathieucarbou]

FYI @DaeMonSxy @mikeg0 @notixor : #24

If you happen to see the issue again with the new upcoming releases, please let me know.

[arifainchtein]

hello,
I am getting this issue, I cloned the repo and test it and still getting the issue. I am using an ESP32 custom board and running under Arduino environment

[mathieucarbou]

hello, I am getting this issue, I cloned the repo and test it and still getting the issue. I am using an ESP32 custom board and running under Arduino environment

Hello,
You need to give more details.
I cannot help you with just that ;-)

The issue was fixed (see this long discussion with 2 people testing )tbnobody/OpenDTU#2326)

So please double check your environment, lib versions, if you have duplicates, etc.
Also, which arduino version, board, etc.

[arifainchtein]

Sorry about that.
The board is a custom pcb using the ESP32-WROOM as microcontroller. I am using Arduino 1.8 . I have an index.html that gets downloaded, in that page i make request to download things like jquery and bootstrap. The crash seems to have at random places, ie sometimes it downloads the first 3 files as requested by the index.html, other times it crashes after 5 files have been downloaded. Will look over the link you provided and try to get a more reproducible scenario, will get back shortly.
I have insured to only have your library in the build path..
Thanks

[mathieucarbou]

Sorry about that. The board is a custom pcb using the ESP32-WROOM as microcontroller. I am using Arduino 1.8 . I have an index.html that gets downloaded, in that page i make request to download things like jquery and bootstrap. The crash seems to have at random places, ie sometimes it downloads the first 3 files as requested by the index.html, other times it crashes after 5 files have been downloaded. Will look over the link you provided and try to get a more reproducible scenario, will get back shortly. I have insured to only have your library in the build path.. Thanks

Ok. Please try to work on a reproductible use case in an .ino file.
And make sure you use the latest software.
Right now, you seem to be the first one seeing this issue again and being able to reproduce. That's what I find weird.