Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Voice assistant setup fails with 'announcement pipeline's file reader' / ffmpeg_proxy error #288

Open
wildegnux opened this issue Jan 13, 2025 · 4 comments

Comments

@wildegnux
Copy link

I have home assistant running in docker. The Voice PE is successfully added to HA as an ESPHome device, but when trying to set up the voice assistant part get the error " The voice assistant is unable to connect to Home Assistant". The link to documentation in the error points to the local network URL being incorrect, that is not the case here, the URL is valid, plain http and can be reached from the network the Voice PE is on. Logs indicate no connection issues, hower they do indicate issues with the FLAC file it (should?) get from HA during the setup:

[23:46:46][D][nabu_media_player.pipeline:173]: Reading FLAC file type
[23:46:51][E][nabu_media_player:305]: The announcement pipeline's file reader encountered an error.

Trying to query the same URL as the voice PE gives me a 200 response code but 0 byte body.

Since the documentation for voice PE all seems to assume you're running HAOS in some form pressed on and made sure to have the wyoming setup finished as well, thinking something could be missing there. So i have wyoming with piper, whisper and openwakeword set up. In Settings -> Voice Assistants I have a voice assistant set up using my local piper/whisper and that works fine when interacting with it through the web ui. Still, after that being set up I get the same error when trying to set up the voice PE. What am I missing here??

Error:
Screenshot 2025-01-12 001610
Voice Assistant config:
Screenshot 2025-01-12 001630
Device logs:

[23:46:42][D][esp-idf:000][update_task]: I (44032) esp-x509-crt-bundle: Certificate validated
[23:46:42]
[23:46:43][D][update:011][update_task]: 'Home Assistant Voice 096425' - Publishing:
[23:46:43][D][update:012][update_task]:   Current Version: 25.1.0
[23:46:43][D][update:015][update_task]:   Latest Version: 25.1.0
[23:46:43][D][update:018][update_task]:   Firmware URL: https://firmware.esphome.io/home-assistant-voice-pe/home-assistant-voice/25.1.0/home-assistant-voice-esp32s3.ota.bin
[23:46:43][D][update:021][update_task]:   Title: Nabu Casa.Home Assistant Voice PE
[23:46:43]* Bugfix: Avoid crashing when started without internet access
[23:46:43][D][update:026][update_task]:   Release URL: https://github.com/esphome/home-assistant-voice-pe/releases/tag/25.1.0
[23:46:46][D][light:036]: 'voice_assistant_leds' Setting:
[23:46:46][D][light:047]:   State: ON
[23:46:46][D][light:051]:   Brightness: 66%
[23:46:46][D][light:109]:   Effect: 'Replying'
[23:46:46][D][media_player:080]: 'Media Player' - Setting
[23:46:46][D][media_player:087]:   Media URL: http://192.168.1.67:8123/api/esphome/ffmpeg_proxy/a444e7685385ea130f4b9516893f4876/TfnRrLE851vzulEUND8_kg.flac
[23:46:46][D][media_player:093]:  Announcement: yes
[23:46:46][D][voice_assistant:515]: State changed from IDLE to STREAMING_RESPONSE
[23:46:46][D][voice_assistant:522]: Desired state set to STREAMING_RESPONSE
[23:46:46][D][ring_buffer:034]: Created ring buffer with size 48000
[23:46:46][D][ring_buffer:034]: Created ring buffer with size 48000
[23:46:46][D][ring_buffer:034]: Created ring buffer with size 65536
[23:46:46][D][ring_buffer:034]: Created ring buffer with size 65536
[23:46:46][D][power_supply:033]: Enabling power supply.
[23:46:46][D][nabu_media_player.pipeline:173]: Reading FLAC file type
[23:46:51][E][nabu_media_player:305]: The announcement pipeline's file reader encountered an error.
[23:46:51][D][voice_assistant:515]: State changed from STREAMING_RESPONSE to IDLE
[23:46:51][D][voice_assistant:522]: Desired state set to IDLE
[23:46:51][D][light:036]: 'voice_assistant_leds' Setting:
[23:46:51][D][light:047]:   State: OFF
[23:46:51][D][light:109]:   Effect: 'None'

ESPHome debug logs:

2025-01-11 23:46:40.733 DEBUG (MainThread) [aioesphomeapi.connection] home-assistant-voice-096425 @ 192.168.1.153: Sending UpdateCommandRequest: key: 2957754261
command: UPDATE_COMMAND_CHECK

2025-01-11 23:46:40.733 DEBUG (MainThread) [aioesphomeapi._frame_helper.base] home-assistant-voice-096425 @ 192.168.1.153: Sending frame: [0007760d95bf4bb01002]
2025-01-11 23:46:42.401 DEBUG (MainThread) [aioesphomeapi.connection] home-assistant-voice-096425 @ 192.168.1.153: Got message of type UpdateStateResponse: key: 2957754261
current_version: "25.1.0"
latest_version: "25.1.0"
title: "Nabu Casa.Home Assistant Voice PE"
release_summary: "## What\'s Changed\r\n* Bugfix: Avoid crashing when started without internet access"
release_url: "https://github.com/esphome/home-assistant-voice-pe/releases/tag/25.1.0"

2025-01-11 23:46:45.746 DEBUG (MainThread) [homeassistant.components.esphome.assist_satellite] Waiting for announcement to finished (message=, media_id=http://192.168.1.67:8123/api/assist_satellite/connection_test/fa6e7472fbdd6ad51a51eea2203e5828?authSig=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiIyNTc0OGZjYjI2NTc0NWViOWJjNmM3YjAwZWYxZWZmMiIsInBhdGgiOiIvYXBpL2Fzc2lzdF9zYXRlbGxpdGUvY29ubmVjdGlvbl90ZXN0L2ZhNmU3NDcyZmJkZDZhZDUxYTUxZWVhMjIwM2U1ODI4IiwicGFyYW1zIjpbXSwiaWF0IjoxNzM2NjM1NjA1LCJleHAiOjE3MzY3MjIwMDV9.YSb2memzED_omymcssnYDccL4k6LZVK6xnH243JMkRo)
2025-01-11 23:46:45.746 DEBUG (MainThread) [homeassistant.components.esphome.ffmpeg_proxy] Media URL allowed by proxy: http://192.168.1.67:8123/api/assist_satellite/connection_test/fa6e7472fbdd6ad51a51eea2203e5828?authSig=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiIyNTc0OGZjYjI2NTc0NWViOWJjNmM3YjAwZWYxZWZmMiIsInBhdGgiOiIvYXBpL2Fzc2lzdF9zYXRlbGxpdGUvY29ubmVjdGlvbl90ZXN0L2ZhNmU3NDcyZmJkZDZhZDUxYTUxZWVhMjIwM2U1ODI4IiwicGFyYW1zIjpbXSwiaWF0IjoxNzM2NjM1NjA1LCJleHAiOjE3MzY3MjIwMDV9.YSb2memzED_omymcssnYDccL4k6LZVK6xnH243JMkRo
2025-01-11 23:46:45.746 DEBUG (MainThread) [aioesphomeapi.connection] home-assistant-voice-096425 @ 192.168.1.153: Sending VoiceAssistantAnnounceRequest: media_id: "http://192.168.1.67:8123/api/esphome/ffmpeg_proxy/a444e7685385ea130f4b9516893f4876/TfnRrLE851vzulEUND8_kg.flac"

2025-01-11 23:46:45.746 DEBUG (MainThread) [aioesphomeapi._frame_helper.base] home-assistant-voice-096425 @ 192.168.1.153: Sending frame: [0070770a6e687474703a2f2f3139322e3136382e312e36373a383132332f6170692f657370686f6d652f66666d7065675f70726f78792f61343434653736383533383565613133306634623935313638393366343837362f54666e52724c45383531767a756c45554e44385f6b672e666c6163]
2025-01-11 23:46:45.979 DEBUG (MainThread) [aioesphomeapi.connection] home-assistant-voice-096425 @ 192.168.1.153: Got message of type MediaPlayerStateResponse: key: 2232357057
state: MEDIA_PLAYER_STATE_PLAYING
volume: 0.499999881

2025-01-11 23:46:45.984 DEBUG (MainThread) [homeassistant.components.esphome.ffmpeg_proxy] /usr/bin/ffmpeg -i http://192.168.1.67:8123/api/assist_satellite/connection_test/fa6e7472fbdd6ad51a51eea2203e5828?authSig=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiIyNTc0OGZjYjI2NTc0NWViOWJjNmM3YjAwZWYxZWZmMiIsInBhdGgiOiIvYXBpL2Fzc2lzdF9zYXRlbGxpdGUvY29ubmVjdGlvbl90ZXN0L2ZhNmU3NDcyZmJkZDZhZDUxYTUxZWVhMjIwM2U1ODI4IiwicGFyYW1zIjpbXSwiaWF0IjoxNzM2NjM1NjA1LCJleHAiOjE3MzY3MjIwMDV9.YSb2memzED_omymcssnYDccL4k6LZVK6xnH243JMkRo -f flac -ar 48000 -ac 1 -sample_fmt s16 -map_metadata -1 -vn -nostats pipe:
2025-01-11 23:46:46.016 DEBUG (MainThread) [homeassistant.components.esphome.ffmpeg_proxy] ffmpeg[263] output: ffmpeg version 6.1.2 Copyright (c) 2000-2024 the FFmpeg developers
2025-01-11 23:46:46.016 DEBUG (MainThread) [homeassistant.components.esphome.ffmpeg_proxy] ffmpeg[263] output:   built with gcc 14.2.0 (Alpine 14.2.0)
2025-01-11 23:46:46.016 DEBUG (MainThread) [homeassistant.components.esphome.ffmpeg_proxy] ffmpeg[263] output:   configuration: --prefix=/usr --disable-librtmp --disable-lzma --disable-static --disable-stripping --enable-avfilter --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-libmp3lame --enable-libopenmpt --enable-libopus --enable-libplacebo --enable-libpulse --enable-librav1e --enable-librist --enable-libsoxr --enable-libsrt --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-lto=auto --enable-lv2 --enable-openssl --enable-pic --enable-postproc --enable-pthreads --enable-shared --enable-vaapi --enable-vdpau --enable-version3 --enable-vulkan --optflags=-O3 --enable-libjxl --enable-libsvtav1 --enable-libvpl
2025-01-11 23:46:46.016 DEBUG (MainThread) [homeassistant.components.esphome.ffmpeg_proxy] ffmpeg[263] output:   libavutil      58. 29.100 / 58. 29.100
2025-01-11 23:46:46.016 DEBUG (MainThread) [homeassistant.components.esphome.ffmpeg_proxy] ffmpeg[263] output:   libavcodec     60. 31.102 / 60. 31.102
2025-01-11 23:46:46.016 DEBUG (MainThread) [homeassistant.components.esphome.ffmpeg_proxy] ffmpeg[263] output:   libavformat    60. 16.100 / 60. 16.100
2025-01-11 23:46:46.016 DEBUG (MainThread) [homeassistant.components.esphome.ffmpeg_proxy] ffmpeg[263] output:   libavdevice    60.  3.100 / 60.  3.100
2025-01-11 23:46:46.016 DEBUG (MainThread) [homeassistant.components.esphome.ffmpeg_proxy] ffmpeg[263] output:   libavfilter     9. 12.100 /  9. 12.100
2025-01-11 23:46:46.016 DEBUG (MainThread) [homeassistant.components.esphome.ffmpeg_proxy] ffmpeg[263] output:   libswscale      7.  5.100 /  7.  5.100
2025-01-11 23:46:46.016 DEBUG (MainThread) [homeassistant.components.esphome.ffmpeg_proxy] ffmpeg[263] output:   libswresample   4. 12.100 /  4. 12.100
2025-01-11 23:46:46.016 DEBUG (MainThread) [homeassistant.components.esphome.ffmpeg_proxy] ffmpeg[263] output:   libpostproc    57.  3.100 / 57.  3.100
2025-01-11 23:46:50.990 DEBUG (MainThread) [homeassistant.components.esphome.ffmpeg_proxy] ffmpeg transcoding cancelled
2025-01-11 23:46:51.016 DEBUG (MainThread) [aioesphomeapi.connection] home-assistant-voice-096425 @ 192.168.1.153: Got message of type MediaPlayerStateResponse: key: 2232357057
state: MEDIA_PLAYER_STATE_IDLE
volume: 0.499999881

2025-01-11 23:46:51.061 DEBUG (MainThread) [aioesphomeapi.connection] home-assistant-voice-096425 @ 192.168.1.153: Got message of type VoiceAssistantAnnounceFinished: success: true

2025-01-11 23:47:13.611 DEBUG (MainThread) [aioesphomeapi.connection] home-assistant-voice-096425 @ 192.168.1.153: Sending PingRequest: 
2025-01-11 23:47:13.611 DEBUG (MainThread) [aioesphomeapi._frame_helper.base] home-assistant-voice-096425 @ 192.168.1.153: Sending frame: [000007]
2025-01-11 23:47:13.709 DEBUG (MainThread) [aioesphomeapi.connection] home-assistant-voice-096425 @ 192.168.1.153: Got message of type PingResponse: 
@sparky3387
Copy link

I hope I'm able to assist you or anyone else, I had the exact same problem with FFMPEG, the problem for me at least, was that home assistant sends the internal URL to the HAVPE, and this can be on any network not neccesarily the internal one

The HAVPE appears to connect in the following steps:

  1. Home assistant checks the updates for the HAVPE on its port 6053, so you need to ensure this can work
  2. Home assistant then sends the internal URL to the HAVPE
  3. The HAVPE connects to the internal URL, so whatever network your HAVPE is on needs to be able to connect to this
  4. This appears to trigger FFMPEG_PROXY as above, to also connect with the same internal URL from within Home Assistant itself, which in the served out to the HAVPE.

For me as the HAVPE was on a separate network to the home assistant server, I needed to ensure the internal URL was resolvable on both networks, but in your case I would check the network reliability between these as you are using IP addresses (i assume on the same subnet).

In my case (and I'm assuming others) I have home assistant running through an nginx proxy, that provides SSL through a wildcard certificate. To get this resolved I setup ADGuard, with a Domain Redirection filter, to serve up the externally resolvable address to the internal home assistant IP, then on top of this used a letsencrypt/nginx proxy so that the same https://myhomeassistantserver.com could resolve both to home assistant itself internally and when accessed via the already setup external nginx proxy.

If someone from Nabu Casa reads this, is it possible to specify the home assistant URL that the HAVPE connects to as an advanced option for the ones that need this?

@tommyjlong
Copy link

@sparky3387

  1. Home assistant then sends the internal URL to the HAVPE

I wanted to confirm that in the HA UI where one sets this "local network" URL, that you had the "Automatic" switch turned off?

The reason I ask, is that in my case, the UI has this confusing wording for the "Automatic" switch:

Image

I'm just not clear what "Use the configured network settings" means. I think it means if "Automatic" is turned Off, then HA will use the configured network settings in the box below it. However I talked with another user who said his configured URL didn't work for his HAVPE unless Automatic was set to On.

Thanks.

@sparky3387
Copy link

@sparky3387

  1. Home assistant then sends the internal URL to the HAVPE

I wanted to confirm that in the HA UI where one sets this "local network" URL, that you had the "Automatic" switch turned off?

Thats correct, as that is the only way I can get the secure URL set that gets sent to the HAVPE

I'm just not clear what "Use the configured network settings" means. I think it means if "Automatic" is turned Off, then HA will use the configured network settings in the box below it. However I talked with another user who said his configured URL didn't work for his HAVPE unless Automatic was set to On.

To see what URL (or other packets) are being sent you can install tcpdump using "Advanced SSH & Web Terminal" addon, i wrote my capture to a file and then opened it in wireshark, definitely recommend anyone doing this who is having network issues with this

@wildegnux
Copy link
Author

wildegnux commented Jan 18, 2025

4. This appears to trigger FFMPEG_PROXY as above, **to also connect with the same internal URL from within Home Assistant** itself, which in the served out to the HAVPE.

This appears to be my problem! HA in docker on a 172.x.x.x network cannot connect to the host 192.168.x.x IP. Need to figure out how I fix that...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants