Problem runing tasks while BLE connected (IDFGH-7456)

[jrHernandezSio]

Hi,

ENVIRONMENT:
espressif32@3.3.2
ESP-IDF 4.3

While the ESP32 is connected via BLE to smartphone I have problem ringbuf module.
While connected several tasks are created and deleted severl times.

Sometimes I get the following crash:

--assertion "pxRingbuffer" failed: file "C:\Users\jr.hernandez\.platformio\packages\framework-espidf\components\esp_ringbuf\ringbuf.c", line 1306, function: xRingbufferGetMaxItemSize

abort() was called at PC 0x401a2e93 on core 0

Backtrace:0x4009235f:0x3fffbc60 0x40092ab1:0x3fffbc80 0x40098a36:0x3fffbca0 0x401a2e93:0x3fffbd10 0x40092155:0x3fffbd40 0x4012e28e:0x3fffbd60 0x4012ec4a:0x3fffbda0 0x400dc5c0:0x3fffbdd0 0x400ded53:0x3fffbdf0 0x400dee55:0x3fffbe30 0x400dee8d:0x3fffbe50 0x400deeaf:0x3fffbe80 0x400deabf:0x3fffbea0 0x400d6195:0x3fffbec0 0x400d62f5:0x3fffbee0 0x40095bb5:0x3fffbf00


ELF file SHA256: da3abc92960c90c0

Rebooting...

All tasks are run in core 0. If I use core 1 for BLE I have other ringbuf error when creating the other tasks by a second time. See error here: "assertion "pxRingbuffer" failed: file "C:\Users\jr.hernandez.platformio\packages\framework-espidf\components\esp_ringbuf\ringbuf.c", line 1100, function: xRingbufferReceive"

Regards, Ramon.

[igrr]

Hi @jrHernandezSio,

Could you please decode the backtrace into a readable format?

Most likely the issue is that some code is passing a NULL pointer to xRingbufferGetMaxItemSize function. The backtrace will reveal which code that is.

[jrHernandezSio]

Not sure how to decode the backtrace. Shall I run idf.py monitor while running the device? or can I decode post-mortem?

Heap summary for capabilities 0x00000004:
  At 0x3ff80518 len 6872 free 144 allocated 5104 min_free 144
    largest_free_block 0 alloc_blocks 35 free_blocks 0 total_blocks 35    
  At 0x3ffb6388 len 7288 free 88 allocated 5576 min_free 64
    largest_free_block 0 alloc_blocks 21 free_blocks 0 total_blocks 21    
  At 0x3ffb9a20 len 16648 free 292 allocated 14732 min_free 292
    largest_free_block 0 alloc_blocks 72 free_blocks 0 total_blocks 72    
  At 0x3ffcfd48 len 66232 free 1536 allocated 63072 min_free 1404
    largest_free_block 32 alloc_blocks 349 free_blocks 5 total_blocks 354 
  At 0x3ffe0440 len 129984 free 18844 allocated 109516 min_free 18148     
    largest_free_block 16384 alloc_blocks 23 free_blocks 2 total_blocks 25
  Totals:
    free 20904 allocated 198000 min_free 20052 largest_free_block 16384   
Heap summary for capabilities 0x00000004:
  At 0x3ff80518 len 6872 free 144 allocated 5104 min_free 144
    largest_free_block 0 alloc_blocks 35 free_blocks 0 total_blocks 35   
  At 0x3ffb6388 len 7288 free 88 allocated 5576 min_free 64
    largest_free_block 0 alloc_blocks 21 free_blocks 0 total_blocks 21
  At 0x3ffb9a20 len 16648 free 548 allocated 14476 min_free 292
    largest_free_block 256 alloc_blocks 70 free_blocks 1 total_blocks 71
  At 0x3ffcfd48 len 66232 free 1512 allocated 63096 min_free 1404
    largest_free_block 32 alloc_blocks 350 free_blocks 4 total_blocks 354
  At 0x3ffe0440 len 129984 free 15692 allocated 112668 min_free 2704
    largest_free_block 4096 alloc_blocks 36 free_blocks 4 total_blocks 40
  Totals:
    free 17984 allocated 200920 min_free 4608 largest_free_block 4096
assertion "pxRingbuffer" failed: file "C:\Users\jr.hernandez\.platformio\packages\framework-espidf\components\esp_ringbuf\ringbuf.c", line 1306, function: xRingbufferGetMaxItemSize

abort() was called at PC 0x401a1b8b on core 0

Backtrace:0x4009235f:0x3fffbcc0 0x40092ab1:0x3fffbce0 0x40098a36:0x3fffbd00 0x401a1b8b:0x3fffbd70 0x40092155:0x3fffbda0 0x4012cfbe:0x3fffbdc0 0x4012d97a:0x3fffbe00 0x400db344:0x3fffbe30 0x400ddad7:0x3fffbe50 0x400ddbd9:0x3fffbe90 0x400ddc11:0x3fffbeb0 0x400ddc33:0x3fffbee0 0x400dd843:0x3fffbf00 0x400d4f15:0x3fffbf20 0x400d5075:0x3fffbf40 0x40095bb5:0x3fffbf60


ELF file SHA256: 52c2e222c5b26b1b

Rebooting...

[igrr]

You can decode by running idf.py monitor. As you are using PlatformIO, i think you can check their documentation, there is also some feature to decode backtraces, IIRC. After the fact, you can use xtensa-esp32-elf-addr2line -pfiaC -e path/to/your/program.elf and then enter the addresses which start with 0x4 (0x4009235f, 0x40092ab1, 0x40098a36, etc).

[igrr]

Here's the relevant link for PlatformIO address decoding: platformio/platform-espressif32#105 (comment)

[jrHernandezSio]

Thanks for the tip :) Really helpful

Here you are. I hope you understand sth cause I do not :)

It seems to fail the allocation twice (non-BLE tasks are using 2048 bytes) and it crash at the second fail.

heap_caps_malloc was called but failed to allocate 2048 bytes with 0x1800 capabilities.
-DEBUG-: [Thu May 26 10:51:28 2022] Expecting ini...
-DEBUG-: [Thu May 26 10:51:28 2022] Dev detected -> Model Id: 0xB4
-DEBUG-: [Thu May 26 10:51:28 2022] Dev detected -> Model Id: 0xB4
-DEBUG-: [Thu May 26 10:51:28 2022] Expecting ini. buff (len=51): VC0703 1.00
Ctrl infr exist
OV7720
525
Init end
heap_caps_malloc was called but failed to allocate 2048 bytes with 0x1800 capabilities.
assertion "pxRingbuffer" failed: file "C:\Users\jr.hernandez\.platformio\packages\framework-espidf\components\esp_ringbuf\ringbuf.c", line 1306, function: xRingbufferGetMaxItemSize

abort() was called at PC 0x401a1bbf on core 0

Backtrace:0x4009235f:0x3fffbe30 0x40092ab1:0x3fffbe50 0x40098a36:0x3fffbe70 0x401a1bbf:0x3fffbee0 0x40092155:0x3fffbf10 0x4012cfda:0x3fffbf30 0x4012d996:0x3fffbf70 0x400db35c:0x3fffbfa0 0x400ddaef:0x3fffbfc0 0x400ddbf1:0x3fffc000 0x400ddc29:0x3fffc020 0x400ddc4b:0x3fffc050 0x400dd85b:0x3fffc070 0x400d4f35:0x3fffc090 0x400d5095:0x3fffc0b0 0x40095bb5:0x3fffc0d0
  #0  0x4009235f:0x3fffbe30 in panic_abort at C:\Users\jr.hernandez\.platformio\packages\framework-espidf\components\esp_system/panic.c:356
  #1  0x40092ab1:0x3fffbe50 in esp_system_abort at C:\Users\jr.hernandez\.platformio\packages\framework-espidf\components\esp_system/system_api.c:112
  #2  0x40098a36:0x3fffbe70 in abort at C:\Users\jr.hernandez\.platformio\packages\framework-espidf\components\newlib/abort.c:46
  #3  0x401a1bbf:0x3fffbee0 in __assert_func at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/newlib/newlib/libc/stdlib/assert.c:62 (discriminator 8)
  #4  0x40092155:0x3fffbf10 in xRingbufferGetMaxItemSize at C:\Users\jr.hernandez\.platformio\packages\framework-espidf\components\esp_ringbuf/ringbuf.c:1306 (discriminator 1)
  #5  0x4012cfda:0x3fffbf30 in uart_tx_all at C:\Users\jr.hernandez\.platformio\packages\framework-espidf\components\driver/uart.c:1106
  #6  0x4012d996:0x3fffbf70 in uart_write_bytes at C:\Users\jr.hernandez\.platformio\packages\framework-espidf\components\driver/uart.c:1163 (discriminator 2)
  #7  0x400db35c:0x3fffbfa0 in SI_send at lib/SERIAL_INTERFACE/serial_interface.c:580
  #8  0x400ddaef:0x3fffbfc0 in sendCommand at lib_prj/CAMERA/Adafruit_VC0706.c:647
  #9  0x400ddbf1:0x3fffc000 in runCommand at lib_prj/CAMERA/Adafruit_VC0706.c:614
  #10 0x400ddc29:0x3fffc020 in reset at lib_prj/CAMERA/Adafruit_VC0706.c:121
  #11 0x400ddc4b:0x3fffc050 in Adafruit_VC0706_begin at lib_prj/CAMERA/Adafruit_VC0706.c:75
  #12 0x400dd85b:0x3fffc070 in CAM_begin at lib_prj/CAMERA/camera.c:32
  #13 0x400d4f35:0x3fffc090 in CAMERA_routine at src/main.c:3431
  #14 0x400d5095:0x3fffc0b0 in TASK_cam_routine at src/main.c:4921
  #15 0x40095bb5:0x3fffc0d0 in vPortTaskWrapper at C:\Users\jr.hernandez\.platformio\packages\framework-espidf\components\freertos\port\xtensa/port.c:168


ELF file SHA256: 09a5cb63b65f9023

Rebooting...

`Boot!...

[igrr]

The issue is that uart_driver_install function in v4.3 release was missing a check for successful allocation of tx_ring_buf ringbuffer. If the ringbuffer wasn't allocated correctly, tx_ring_buf member of uart object was NULL but the uart_driver_install function returned successfully. Subsequent access to tx_ring_buf in uart_tx_all results in a failed assertion.

This was fixed in commit 73c5a68, available in v4.3.2 bugfix release. Now uart_driver_install should return with an error and print "UART driver malloc error".

If you upgrade to IDF 4.3.2, the failed assertion issue will be fixed. However uart_driver_install will still not succeed because there's simply not enough heap RAM left to allocate the ringbuffer.

[jrHernandezSio]

Thanks for the support.

I will optimize my RAM so as to provide enough heap to the tasks.

Cheers.

[igrr]

No problem! If you think this and the other issue you have opened are resolved, would you consider closing them?

If you have trouble optimizing the memory consumption of the bluetooth stack, feel free to open another issue on that topic, instead.

We also have a couple of articles which may help you:

• https://blog.espressif.com/esp32-programmers-memory-model-259444d89387
• https://blog.espressif.com/esp32-memory-analysis-case-study-eacc75fe5431

[jrHernandezSio]

Thanks.

[jrHernandezSio]

Hi,

After RAM was optimized everything worked ok, but now we have detected some units presenting memory problems, which are not reproduced in other units of ESP32.

We have similar problems described in issue #11393

Not sure if I should reopen this or keep on #11393

I am wondering how RAM gets unaccesible