'Memory exhausted' error while using example 'audio mixer tone' (AUD-5884)

[GCTechnology]

Environment

• Audio development kit: [ESP32-S3-WROOM-1-N16R8（ATK board） and ESP32-S3-WROOM-1U-N8]
• [Required] Module or chip used: [ESP32-S3-WROOM-1 and ESP32-S3-WROOM-1U]
• [Required] IDF version: v5.3.1
• [Required] ADF version: v2.7-6-g14c79a00
• Build system: [idf.py(Using vscode ESP-IDF extension)]
• [Required] Running log: All logs from power-on to problem recurrence

esp32> pmixer https://dl.espressif.com/dl/audio/ff-16b-2c-44100hz.mp3 1

I (22233) ESP_DECODER: esp_decoder_init, stack size is 10240
I (22236) MIXER_PIPE: One pipeline has been created, uri:https://dl.espressif.com/dl/audio/ff-16b-2c-44100hz.mp3, pipe:0x3fcd3e8c, mixer:0x3fca8fe8
W (22249) AUDIO_THREAD: Make sure selected the `CONFIG_SPIRAM_BOOT_INIT` and `CONFIG_SPIRAM_ALLOW_STACK_EXTERNAL_MEMORY` by `make menuconfig`
I (22262) AUDIO_THREAD: The mixer_manager task allocate stack on internal memory
I (22274) MIXER_PIPE: Mixer pipeline:0x3fca8fe8
W (22276) AUDIO_THREAD: Make sure selected the `CONFIG_SPIRAM_BOOT_INIT` and `CONFIG_SPIRAM_ALLOW_STACK_EXTERNAL_MEMORY` 
by `make menuconfig`
I (22289) AUDIO_THREAD: The mixing_in task allocate stack on internal memory
I (22301) AUDIO_ELEMENT: [mixing_in-0x3fca638c] Element task created
W (22302) AUDIO_THREAD: Make sure selected the `CONFIG_SPIRAM_BOOT_INIT` and `CONFIG_SPIRAM_ALLOW_STACK_EXTERNAL_MEMORY` by `make menuconfig`
I (22315) AUDIO_THREAD: The mixing_dec task allocate stack on internal memory
I (22327) AUDIO_ELEMENT: [mixing_dec-0x3fca67dc] Element task created
W (22329) AUDIO_THREAD: Make sure selected the `CONFIG_SPIRAM_BOOT_INIT` and `CONFIG_SPIRAM_ALLOW_STACK_EXTERNAL_MEMORY` by `make menuconfig`
I (22342) AUDIO_THREAD: The mixing_filter task allocate stack on internal memory
I (22354) AUDIO_ELEMENT: [mixing_filter-0x3fcd3b30] Element task created
I (22365) AUDIO_ELEMENT: [mixing_raw-0x3fca6b38] Element task created
I (22367) AUDIO_ELEMENT: [mixing_in] AEL_MSG_CMD_RESUME,state:1
I (22379) AUDIO_ELEMENT: [mixing_dec] AEL_MSG_CMD_RESUME,state:1
I (22383) ESP_DECODER: Ready to do audio type check, pos:0, (line 105)
I (22385) AUDIO_ELEMENT: [mixing_filter] AEL_MSG_CMD_RESUME,state:1
I (22398) MIXER_PIPE: [CMD] START Mixer ch[1] mixer[0x3fca8fe8]
I (22400) MIXER_PIPE: Mixer manager REC CMD:5,dat:0 src:0x3fca8fe8, Mixer:0x3fca8fe8
I (22415) MIXER_PIPE: Mixer manager REC CMD:8,dat:12 src:0x3fca6b38, Mixer:0x3fca8fe8
I (22422) MIXER_PIPE: Mixer manager REC CMD:8,dat:12 src:0x3fca6b38, Mixer:0x3fca8fe8
I (22496) RSP_FILTER: sample rate of source data : 44100, channel of source data : 2, sample rate of destination data : 48000, channel of destination data : 2
I (22510) MIXER_PIPE: Mixer manager REC CMD:8,dat:12 src:0x3fcd3b30, Mixer:0x3fca8fe8
I (23897) HTTP_STREAM: total_bytes=0
I (24491) wifi:<ba-add>idx:0 (ifx:0, 72:5e:f7:dd:1f:f9), tid:0, ssn:20, winSize:64
I (24638) HTTP_STREAM: total_bytes=2994349
I (24640) MIXER_PIPE: Mixer manager REC CMD:10,dat:0 src:0x3fca638c, Mixer:0x3fca8fe8
I (24645) MIXER_PIPE: Mixer manager REC CMD:8,dat:12 src:0x3fca638c, Mixer:0x3fca8fe8
I (24642) ESP_DECODER: Detect audio type is MP3
I (24658) MP3_DECODER: MP3 opened
I (24659) CODEC_ELEMENT_HELPER: The element is 0x3fca67dc. The reserve data 2 is 0x0.
E (24676) MP3_DE_LIB: pvmp3_framedecoder.cpp:698 (pvmp3_InitDecoder): Memory exhausted
E (24678) MP3_DECODER: Allocate buffer failed. (line 407)
E (24689) AUDIO_ELEMENT: [mixing_dec] AEL_STATUS_ERROR_OPEN,-1
W (24691) AUDIO_ELEMENT: [mixing_dec] audio_element_on_cmd_error,7
I (24692) MIXER_PIPE: [STATUS] Mixer pipeline is error, el:mixing_dec, Mixer:0x3fca8fe8
I (24702) MP3_DECODER: Closed
I (24708) MIXER_PIPE: Mixer manager REC CMD:8,dat:1 src:0x3fca67dc, Mixer:0x3fca8fe8
I (24720) MIXER_PIPE: Mixer manager REC CMD:8,dat:14 src:0x3fca6b38, Mixer:0x3fca8fe8
I (24732) MIXER_PIPE: Mixer manager REC CMD:11,dat:1070486100 src:0x3fca67dc, Mixer:0x3fca8fe8
I (24734) MIXER_PIPE: [STATUS] Mixer pipeline is stopped or finished, el:14, Mixer:0x3fca8fe8
W (24747) AUDIO_ELEMENT: [mixing_dec] Element already stopped
W (24749) AUDIO_ELEMENT: [mixing_raw] Element already stopped
W (24719) AUDIO_ELEMENT: IN-[mixing_filter] AEL_IO_ABORT
W (24763) AUDIO_ELEMENT: [0x3fcd3b30-mixing_filter] is already in the AEL_STATE_INIT state
W (24775) AUDIO_ELEMENT: [0x3fcd3b30-mixing_filter] is already in the AEL_STATE_INIT state
W (24812) HTTP_STREAM: No output due to stopping
W (24814) AUDIO_ELEMENT: [0x3fca638c-mixing_in] is already in the AEL_STATE_INIT state
W (24816) AUDIO_ELEMENT: [0x3fca638c-mixing_in] is already in the AEL_STATE_INIT state
E (29816) task_wdt: Task watchdog got triggered. The following tasks/users did not reset the watchdog in time:
E (29816) task_wdt:  - IDLE0 (CPU 0)
E (29816) task_wdt: Tasks currently running:
E (29816) task_wdt: CPU 0: mixing_filter
E (29816) task_wdt: CPU 1: mixing_in
E (29816) task_wdt: Print CPU 0 (current core) backtrace

• Compiler version: (crosstool-NG esp-13.2.0_20240530) 13.2.0
• Operating system: [Windows]
• Using an IDE?: [Yes (Vscode)]
• Power supply: [USB and external 5V]

Problem Description

Steps to Reproduce

1. Flash and restart.
2. Type 'pmixer https://dl.espressif.com/dl/audio/ff-16b-2c-44100hz.mp3 1' in esp console. (WIFI has been connected)
3. 'Memory exhausted' error got triggered and audio does not output.
   and unfortunately the pvmp3_framedecoder.cpp is included only as a library so there is no opportunity for me to examine the code and see what might be going on (unless I decompile it).

Code to Reproduce This Issue

audio mixer tone

[TempoTian]

When decode MP3, it uses more memory, So you need turn on PSRAM support.
You can enable it in idf.py menuconfig, log already have the tips:
Make sure selected the CONFIG_SPIRAM_BOOT_INIT and CONFIG_SPIRAM_ALLOW_STACK_EXTERNAL_MEMORY
by make menuconfig

[GCTechnology]

When decode MP3, it uses more memory, So you need turn on PSRAM support. You can enable it in idf.py menuconfig, log already have the tips: Make sure selected the CONFIG_SPIRAM_BOOT_INIT and CONFIG_SPIRAM_ALLOW_STACK_EXTERNAL_MEMORY by make menuconfig

But for those who do not have PSRAM, like 'ESP32-S3-WROOM-1U-N8'(8MB flash and NO PSRAM), how to solve this problem?

[TempoTian]

Maybe it not suitable to run audio related project.
If need run, you can try to use https://github.com/espressif/esp-adf-libs/blob/master/esp_audio_codec/README.md#decoder-1 it consume less memory for mp3 about 28k, you can use it to decode data and write to I2s directly.

[GCTechnology]

Maybe it not suitable to run audio related project. If need run, you can try to use https://github.com/espressif/esp-adf-libs/blob/master/esp_audio_codec/README.md#decoder-1 it consume less memory for mp3 about 28k, you can use it to decode data and write to I2s directly.

Okay I run the command 'pmixer' successfully on ESP32-S3-WROOM-1-N16R8 which has a PSRAM. You mean this example 'audio_mixer_tone' do not support a chip without PSRAM?

In our current project, we wanna to mix audio in and mp3 file together in real time. Maybe this won't cost much memory? Because it is no need to support all the formats.

[GCTechnology]

Maybe it not suitable to run audio related project. If need run, you can try to use https://github.com/espressif/esp-adf-libs/blob/master/esp_audio_codec/README.md#decoder-1 it consume less memory for mp3 about 28k, you can use it to decode data and write to I2s directly.

Okay I run the command 'pmixer' successfully on ESP32-S3-WROOM-1-N16R8 which has a PSRAM. You mean this example 'audio_mixer_tone' do not support a chip without PSRAM?

In our current project, we wanna to mix audio in and mp3 file together in real time. Maybe this won't cost much memory? Because it is no need to support all the formats.

And I would like to know how to modify this example to use only mp3 decoder, since mp3 may cost less memory.

[TempoTian]

If you have more than 100K mem left, I think run decoder mp3 and mixer together is OK. You can buildup your APP from simple case to your full APP. Decoder mp3 only (can use simple decoder) --> Decode in extra tasks --> Mixer 2 streams --> decode in tasks, decode write to ringfifo， mixer stream read 2 streams。 create task if needed in stead of use audio pipeline can decrease memory usage.