2024.11.0 breaks Frigate camera streaming when WebRTC is disabled in Frigate integration configuration

[felipecrs]

The problem

The following shows up in my browser logs:

{
    "code": "webrtc_get_client_config_failed",
    "message": "Camera does not support WebRTC, frontend_stream_type=hls"
}

I believe this is a bug, because the 2024.11.0 release notes states that the former playback method would be used as fallback (which never happens, the player is just stuck).

What version of Home Assistant Core has the issue?

core-2024.11.0

What was the last working version of Home Assistant Core?

core-2024.10.0

What type of installation are you running?

Home Assistant OS

Integration causing the issue

No response

Link to integration documentation on our website

No response

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

• Refs Improve support for HA 2024.11.0 WebRTC streaming blakeblackshear/frigate-hass-integration#759

[home-assistant]

Hey there @home-assistant/core, mind taking a look at this issue as it has been labeled with an integration (go2rtc) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of go2rtc can trigger bot actions by commenting:

• @home-assistant close Closes the issue.
• @home-assistant rename Awesome new title Renames the issue.
• @home-assistant reopen Reopen the issue.
• @home-assistant unassign go2rtc Removes the current integration label and assignees on the issue, add the integration domain after the command.
• @home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue.
• @home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

---

go2rtc documentation
go2rtc source
_{^{(message by IssueLinks)}}

[home-assistant]

Hey there @home-assistant/core, mind taking a look at this issue as it has been labeled with an integration (camera) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of camera can trigger bot actions by commenting:

• @home-assistant close Closes the issue.
• @home-assistant rename Awesome new title Renames the issue.
• @home-assistant reopen Reopen the issue.
• @home-assistant unassign camera Removes the current integration label and assignees on the issue, add the integration domain after the command.
• @home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue.
• @home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

---

camera documentation
camera source
_{^{(message by IssueLinks)}}

[edenhaus]

Please provide diagnostic information and logs.

The error above shows that the Camera entity provided by frigate does not support WebRTC

[felipecrs]

Diagnostics of which integration? I also could not find any relevant logs.

The error above shows that the Camera entity provided by frigate does not support WebRTC

Yes, but as I said, the 2024.11.0 release notes mentions that Home Assistant will fallback to the old player when WebRTC does not work.

This bug report is about this fallback mechanism which isn't working.

That said, all it takes for Frigate integration to support WebRTC is to:

And then streaming cameras through HA works again. I'll retitle the issue to make this clearer.

[cypherbits]

Same here, but with webrtc disabled and stream stopped working.

[ohadlevy]

in my browser console, I can see an invalid URL being queried:

"/media-browser/browser/app,media-source://frigate/video,media-source://frigate/frigate/event-search/clips///////undefined"

[felipecrs]

@edenhaus you can give a try to the Frigate integration devcontainer:

https://github.com/blakeblackshear/frigate-hass-integration

It should come with both Frigate and Home Assistant pre-configured, which should allow you to catch the issue very easily.

[katschub88]

I've upgraded core to 2024.11.0 and after that the frigate card didn't work properly. Clips and recordings didn't play anymore.
On desktop, the console showed the error: this.popover.destroy is not a function
I've then rolled back to 2024.10.4 (ha core update --version=2024.10.4 ). After the command has finished, there was an error that go2rtc was missing and I could choose to remove it from the config or ignore the problem.
So I chose to fix the problem and started home assistant manually again (full reboot). After that the playback was working again.
So the problem was caused by core 2024.11.0. Maybe it has something to do with the go2rtc stuff?

[timota]

Same, after upgrade. When open camera stream in logs I have such error

2024-11-07 22:45:19.964 ERROR (MainThread) [homeassistant.components.websocket_api.http.connection] [140025229885648] Error handling message: Unknown error (unknown_error) USER from 192.168.1.30 (Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36)
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/decorators.py", line 28, in _handle_async_response
    await func(hass, connection, msg)
  File "/usr/src/homeassistant/homeassistant/components/camera/webrtc.py", line 263, in ws_webrtc_offer
    await camera.async_handle_async_webrtc_offer(offer, session_id, send_message)
  File "/usr/src/homeassistant/homeassistant/components/camera/__init__.py", line 634, in async_handle_async_webrtc_offer
    answer = await self.async_handle_web_rtc_offer(offer_sdp)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/frigate/camera.py", line 332, in async_handle_web_rtc_offer
    answer = await resp.json()
             ^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/aiohttp_client.py", line 80, in json
    return await super().json(*args, loads=loads, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/client_reqrep.py", line 1277, in json
    raise ContentTypeError(
aiohttp.client_exceptions.ContentTypeError: 500, message='Attempt to decode JSON with unexpected mimetype: text/plain; charset=utf-8', url='http://192.168.1.209:5000/api/go2rtc/webrtc?src=cam_back'

Also, my cameras stream in h265, supposed, webrtc should not work(I may be wrong). So, homeassistant should fallback to hls, but it does not and stream is frozen.

[timota]

I have managed to find the source of that error in go2rtc log file

github.com/AlexxIT/go2rtc/internal/webrtc/webrtc.go:239 > error="streams: codecs not matched: H265, AAC => VP8, VP9, H264, AV1, OPUS, G722, PCMU, PCMA, S16B, S16L"

definitely, webrtc not supported on my cameras (H265) and GO2RTC integration does not process it correctly and fails.
So, I have disabled WebRTC in Frigate integration and now see in browser console:

{
    "code": "webrtc_get_client_config_failed",
    "message": "Camera does not support WebRTC, frontend_stream_type=hls"
}

As it states in release notes - it should fallback to hls. Personally, I will consider it as a bug.

[tammeryousef1006]

even i enable webrtc for frigate still unable to load camera

[shpitz461]

The solution for now is to downgrade to 2024.10.x:

blakeblackshear/frigate-hass-integration#757 (comment)

[edenhaus]

The problem is that the custom component frigate is not using the Camera entity correctly. The core allowed this misuse until 2024.11. We discussed this issue internally and concluded that the custom component needs to be updated.

I tried to contact the maintainer but have yet to receive a response.
Core identifies whether a camera entity has native WebRTC support by checking whether the native WebRTC methods are implemented. Native WebRTC cameras don't support HLS, so the core will not fall back to HLS.
The issue is now that frigate is always implementing the native WebRTC methods even when the camera doesn't support WebRTC, as go2rtc is disabled in frigate (For core, it means that any frigate camera entity supports only WebRTC and therefore, the frontend will not fallback to HLS). Frigate is using the frontend_stream_type to manipulate the camera behavior, which was not intended. To avoid further issues, we deprecated the property, which will be removed in 6 months.

Please report the issue to the author of the custom component.
I will close the issue as no action is required by core.

[felipecrs]

@dermotduffy @NickM-27 ☝️

[dermotduffy]

I tried to contact the maintainer but have yet to receive a response.

Thanks @edenhaus , we'll work on this.

For future reference, how did you determine who to contact? I am suspecting we should update codeowners to include the most active maintainers. Is this what you consulted?

[tammeryousef1006]

Latest custom frigate component update solved my issue , noting that i enabled webrtc option at frigate componenet

…

On Mon, 25 Nov 2024, 11:10 am On Freund, ***@***.***> wrote: @edenhaus <https://github.com/edenhaus> Can you point me to where this was discussed? The fallback was implemented to allow things like cast to work. Frigate cameras support both webrtc and RTSP, so it seems inefficient to only serve one of them and rely on additional HA mechanisms to convert. Native WebRTC cameras don't support HLS, so the core will not fall back to HLS. That's true for Nest, but I don't think you can make this generalization, and in particular, Frigate cameras do support both. — Reply to this email directly, view it on GitHub <#129990 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AO3B3TDZWXSBTYDLAJFEEX32CLLOBAVCNFSM6AAAAABRJ3I3U6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOJXGE3TQMJTGI> . You are receiving this because you commented.Message ID: ***@***.***>

[edenhaus]

@dermotduffy I contacted blakeblackshear directly in Discord.

Native WebRTC cameras don't support HLS, so the core will not fall back to HLS.

That's true for Nest, but I don't think you can make this generalization, and in particular, Frigate cameras do support both.

This is currently a limitation of HA. The HLS stream component needs a stream source, and native WebRTC cameras only need the WebRTC functions. Therefore, HLS cannot work, and we cannot fallback to HLS.
If you want to add this to HA feel free to open a PR for it, but we shouldn't discuss this topic on this issue as it's unrelated

[OnFreund]

@edenhaus I tagged you on the arch discussion from back then,
In the meantime, I feel like this issue is getting an inefficient resolution - HA - > go2rtc (HA instance) -> go2rtc (frigate instance) -> frigate.
I'd like to better understand why the fallback was removed. Was there any discussion prior to that?

[edenhaus]

In the meantime, I feel like this issue is getting an inefficient resolution - HA - > go2rtc (HA instance) -> go2rtc (frigate instance) -> frigate.

This is incorrect! The go2rtc provider will only be used, when the camera entity is not supporting WebRTC. So the above-mentioned one is not possible.

I'd like to better understand why the fallback was removed.

There was no fallback before and we did not remove one. We did not add the fallback for WebRTC cameras as they don't need to provide a stream source, which is needed for HLS support.

Locking this issue as the latest comments are not related to this issue.