Examples don't work with ESP32-S3 Box 3? (AIS-1455)

[mapigit]

Hi there,

I've tried for several days now to get the Wake-word example working on my new ESP32-S3 Box 3. Different IDF versions gave the same reult: The example compiles without problems (yes, I've set the target to ESP32S3, and in menuconfig the Esp32-S3 Box), but there is no reaction to any wakeword. Same for the en speech recognition. However, the factory demo works fine: I've built and flashed the demo v1.2.4 with IDF 5.1.2; wakeword and english commands are recognized.

Is it possible that the Skainet examples don't work with the Box 3 Hardware version yet? BTW, the LOG of the wakeword example shows MIC1-MIC4, which I find strange, whereas the factory demo correctly prints only MIC1, MIC2.

Can you please help?
Thanks, cheers,
Martin

Here's the monitor output; after this the box doesn't react to wakewords Hi ESP or Alexa (both selected in menuconfig):

SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce3818,len:0x514
load:0x403c9700,len:0x4
load:0x403c9704,len:0xb58
load:0x403cc700,len:0x2b64
entry 0x403c98fc
I (24) cpu_start: Multicore app
I (25) octal_psram: vendor id : 0x0d (AP)
I (26) octal_psram: dev id : 0x03 (generation 4)
I (27) octal_psram: density : 0x05 (128 Mbit)
I (28) octal_psram: good-die : 0x01 (Pass)
I (29) octal_psram: Latency : 0x01 (Fixed)
I (30) octal_psram: VCC : 0x00 (1.8V)
I (31) octal_psram: SRF : 0x01 (Fast Refresh)
I (33) octal_psram: BurstType : 0x01 (Hybrid Wrap)
I (35) octal_psram: BurstLen : 0x01 (32 Byte)
I (36) octal_psram: Readlatency : 0x02 (10 cycles@Fixed)
I (38) octal_psram: DriveStrength: 0x00 (1/1)
I (39) MSPI Timing: PSRAM timing tuning index: 5
I (40) esp_psram: Found 16MB PSRAM device
I (41) esp_psram: Speed: 80MHz
I (42) cpu_start: Pro cpu up.
I (43) cpu_start: Starting app cpu, entry point is 0x403793f8
0x403793f8: call_start_cpu1 at C:/Users/Martin/esp/esp-idf/components/esp_system/port/cpu_start.c:157

I (0) cpu_start: App cpu up.
I (625) esp_psram: SPI SRAM memory test OK
I (634) cpu_start: Pro cpu start user code
I (634) cpu_start: cpu freq: 240000000 Hz
I (635) cpu_start: Application information:
I (636) cpu_start: Project name: wake_word_detection
I (637) cpu_start: App version: v0.3.0-304-gba3106e
I (638) cpu_start: Compile time: Jan 1 2024 20:37:53
I (640) cpu_start: ELF file SHA256: 8c70aa641326e0b5...
I (642) cpu_start: ESP-IDF: v5.1.2
I (643) cpu_start: Min chip rev: v0.0
I (643) cpu_start: Max chip rev: v0.99
I (644) cpu_start: Chip rev: v0.2
I (645) heap_init: Initializing. RAM available for dynamic allocation:
I (647) heap_init: At 3FC9CF70 len 0004C7A0 (305 KiB): DRAM
I (649) heap_init: At 3FCE9710 len 00005724 (21 KiB): STACK/DRAM
I (650) esp_psram: Adding pool of 16384K of PSRAM memory to heap allocator
I (652) spi_flash: detected chip: gd
I (653) spi_flash: flash io: qio
I (653) sleep: Configure to isolate all GPIO pins in sleep state
I (655) sleep: Enable automatic switching of GPIO sleep configuration
I (657) app_start: Starting scheduler on CPU0
I (658) app_start: Starting scheduler on CPU1
I (658) esp_psram: Reserving pool of 16K of internal memory for DMA/internal allocations
I (668) ES7210: Work in Slave mode
I (668) ES7210: Enable ES7210_INPUT_MIC1
I (668) ES7210: Enable ES7210_INPUT_MIC2
I (668) ES7210: Enable ES7210_INPUT_MIC3
I (668) ES7210: Enable ES7210_INPUT_MIC4
I (668) ES7210: Enable TDM mode
I (668) I2S_IF: channel mode 0 bits:32/32 channel:2 mask:3
I (668) I2S_IF: STD Mode 0 bits:32/32 channel:2 sample_rate:16000 mask:3
I (678) ES7210: Bits 16
I (678) ES7210: Enable ES7210_INPUT_MIC1
I (678) ES7210: Enable ES7210_INPUT_MIC2
I (678) ES7210: Enable ES7210_INPUT_MIC3
I (678) ES7210: Enable ES7210_INPUT_MIC4
I (678) ES7210: Enable TDM mode
I (688) ES7210: Unmuted
I (688) Adev_Codec: Open codec device OK
I (688) ES8311: Work in Slave mode
I (688) gpio: GPIO[46]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0
I (688) I2S_IF: channel mode 0 bits:32/32 channel:2 mask:3
I (688) I2S_IF: STD Mode 1 bits:32/32 channel:2 sample_rate:16000 mask:3
I (698) Adev_Codec: Open codec device OK
I (698) MODEL_LOADER: The storage free size is 15936 KB
I (698) MODEL_LOADER: The partition size is 5168 KB
I (698) MODEL_LOADER: Successfully map model partition
The first wakenet model: wn9_alexa
The second wakenet model: wn9_hiesp
I (698) AFE_SR: afe interface for speech recognition

I (698) AFE_SR: AFE version: SR_V220727

I (698) AFE_SR: Initial auido front-end, total channel: 3, mic num: 2, ref num: 1

I (708) AFE_SR: aec_init: 0, se_init: 1, vad_init: 1

I (708) AFE_SR: wakenet_init: 1

MC Quantized wakenet9: wakenet9l_v3h24_alexa_3_0.625_0.645, tigger:v3, mode:2, p:0, (Aug 24 2023 11:56:47)
MC Quantized wakenet9: wakeNet9_v1h24_hiesp_3_0.63_0.635, tigger:v3, mode:2, p:0, (Aug 24 2023 11:56:47)
I (928) AFE_SR: wake num: 3, mode: 0, (Aug 24 2023 11:56:48)

I (928) main_task: Returned from app_main()
------------detect start------------

[feizi]

@mapigit , Yes, currently, Skainet examples don't work with the Box 3 Hardware version yet. We will support it this week.

[feizi]

Thank you for the question. Now Skainet has supported Box-3. Please try again.
The commit: 8824d83

[mapigit]

@feizi, could you please also add "|| CONFIG_ESP32_S3_BOX_3_BOARD" in main.c? E.g. in the english speech recognition example in lines 170 and 154. Thanks!

[mapigit]

Hi there,

if I select "multiple wakewords" in the english speech recognition example and then "Alexa" as well as "Hi, ESP!", only "Alexa" will be recognized as a wakeword by the Box3. If I keep "multiple wakewords", deselect "Alexa" and select only "Hi, ESP!", then "Hi, ESP!" works. Is this a bug? I'd like to be able to use bothe wakewords alternatively in the same program.

Thanks, cheers,
Martin

[feizi]

Hi @mapigit ,

if I select "multiple wakewords" in the english speech recognition example and then "Alexa" as well as "Hi, ESP!", only "Alexa" will be recognized as a wakeword by the Box3. If I keep "multiple wakewords", deselect "Alexa" and select only "Hi, ESP!", then "Hi, ESP!" works. Is this a bug? I'd like to be able to use bothe wakewords alternatively in the same program.

It is not a bug. If you keep "multiple wakewords", you also need choose two or more wake words, like this:

[mapigit]

Hi @feizi
that's exactly what I did. After selecting "multiple wakewords" as well as both wakewords, still only "Alexa" works.
Thanks, cheers,
Martin

[sun-xiangyu]

You still need to bind the model name of the WakeNet in the AFE config.
Please refer to this example

    afe_config.wakenet_model_name = wn_name;
    afe_config.wakenet_model_name_2 = wn_name_2;