ONVIF Motion Events / Binary sensor not appearing on recent firmwares

[JurajNyiri]

Please keep the discussion in here strictly related to the "Binary sensor not appearing" - if the binary sensor appears and gets stuck, create a separate issue.

Tests

HA Version	Integration Version	Camera	HW	Firmware	State
2023.5.2	5.2.0	C200	HW 1.0	1.3.6 Build 230424 Rel.77225n(4555)	✅
2023.5.2	5.2.0	C200	HW 3.0	1.3.4 Build 230424 Rel.76394n(4555)	✅
2023.5.2	5.2.0	C200	HW 1.0	1.1.16 Build 211209 Rel.37726n(4555)	✅
2023.5.1	5.1.8	C200	HW 2.0	1.3.6 Build 230424 Rel.77225n(4555)	❌
2023.5.1	5.1.8	C200	HW 1.0	1.3.6 Build 230424 Rel.77225n(4555)	❌
2023.5.1	5.1.8	C200	HW 3.0	1.3.4 Build 230424 Rel.76394n(4555)	✅
2023.5.1	5.1.8	C200	HW 1.0	1.1.18	✅
2023.5.1	5.1.8	C200	HW 1.0	1.1.16	✅
2023.5.0	5.1.7	C200	HW 1.0	1.3.6 Build 230424 Rel.77225n(4555)	❌
2023.5.0	5.1.7	C200	HW 2.0	1.3.5 Build 230307 Rel.2897n(4555)	❗
2023.5.0	5.1.7	C200	N/A	1.3.5 Build 230307 Rel.2897n(4555)	❗
2023.5.0	5.1.7	C200	HW 3.0	1.3.4 Build 230424 Rel.76394n(4555)	❗
2023.5.0	5.1.7	C210	N/A	1.3.0 Build 220830 Rel.69909n(4555)	❗
2023.5.0	5.1.7	C200	HW 1.0	1.1.18	❗
2023.5.0	5.1.7	C200	HW 3.0	1.3.0	❗
2023.4.x	5.1.2	C200	N/A	1.3.5 Build 230307 Rel.2897n(4555)	❌
2023.4.x	5.1.2	C200	HW 3.0	1.3.0	✅
2023.4.x	5.1.2	C200	HW 1.0	1.1.18	✅

✅ - Works as expted
❗ - Inconsistent / getting stuck
❌ - Binary sensor does not appear at all

Description

• ONVIF events no longer work on firmware 1.3.4 and 1.3.5.
• The issue affects also https://www.home-assistant.io/integrations/onvif/
• Issue will most probably need to be fixed on cameras via firmware

Reproduction Steps

1. Update to latest firmware
2. Observe motion sensor no longer working

Expected behavior

Motion sensor working.

Next steps

Issue will need to be fixed either upstream, or via firmware update to camera.

[JurajNyiri]

Motion sensor stopped working recently after camera update?

When adding a new comment that it is also not working for you, please:

1. Follow FAQ first
2. Test and confirm it is working / not working with onvif integration
3. Enable debug logs and send relevant onvif / motion events / binary sensor part with your comment

logger:
  default: critical
  logs:
    custom_components.tapo_control: debug
    homeassistant.components.onvif: debug

FYI @silfax @Taomyn @Agimgrue created this issue to track the issue in one place.

[JurajNyiri]

Please test with https://github.com/JurajNyiri/HomeAssistant-Tapo-Control/releases/tag/5.1.0 .

[AndreiArdelean1]

It works with 5.1.0!

[silfax]

It works with 5.1.0!

What camera model? I have 2 x C200, FW 1.3.4 & 1.3.5 and the motion sensor is still 'unavailable' after updating the integration. Will take a look at the logs later today, on the phone right now. Thanks!

[AndreiArdelean1]

C225. Make sure HA is updated to the latest version. After updating the integration, restart both the container/computer and the camera.

[rodrigogbs]

Sill not working here with 4x C200, the motion sensor continue with unavailable status even after lots of reboots...

Home Assistant 2023.4.5
Supervisor 2023.04.0
Operating System 9.5
Frontend 20230411.1 - latest
Integration version: 5.1.0

[JurajNyiri]

Please follow steps above.

[mitzu2250]

Hello.

I have 2 different cameras:
1 C200 running on 1.3.5 Build 230307 Rel.6847n(4555) contains string 'hol' in it's name
1 C210 running on 1.3.0 Build 220830 Rel.69909n(4555) contains string 'living' in it's name

For some time, the motion sensor from C200 is not working. Usually when I had this issue, it was fixed after a restart of HA or reboot of the camera.
The C210 one is still working as expected.

I have enabled the debug on the two entities and got the attached log.
I cannot see any ERRORS, maybe you can spot something that it shouldn't be there.

home-assistant_2023-04-19T07-20-50.421Z.log

[JurajNyiri]

Thank you for your report and logs @mitzu2250 .

The important part of the logs is below:

2023-04-19 10:17:09.551 DEBUG (MainThread) [custom_components.tapo_control] Setting up subscription to motion sensor...
2023-04-19 10:17:09.551 DEBUG (MainThread) [custom_components.tapo_control] Initiating onvif.
2023-04-19 10:17:10.031 DEBUG (MainThread) [custom_components.tapo_control] {'device': <onvif.client.ONVIFCamera object at 0x7f00c5ddb8b0>, 'device_mgmt': <onvif.client.ONVIFService object at 0x7f00c5ddad10>}
2023-04-19 10:17:10.031 DEBUG (MainThread) [custom_components.tapo_control] setupOnvif - entry
2023-04-19 10:17:10.031 DEBUG (MainThread) [custom_components.tapo_control] Setting up onvif...
2023-04-19 10:17:10.031 DEBUG (MainThread) [custom_components.tapo_control] setupEvents - entry
2023-04-19 10:17:10.031 DEBUG (MainThread) [custom_components.tapo_control] Setting up events...
2023-04-19 10:17:10.910 DEBUG (MainThread) [custom_components.tapo_control] async_update_data - entry
2023-04-19 10:17:10.910 DEBUG (MainThread) [custom_components.tapo_control] Motion sensor or time sync is enabled.
2023-04-19 10:17:10.910 DEBUG (MainThread) [custom_components.tapo_control] Setting up subscription to motion sensor...
2023-04-19 10:17:10.910 DEBUG (MainThread) [custom_components.tapo_control] Initiating onvif.
2023-04-19 10:17:11.160 DEBUG (MainThread) [custom_components.tapo_control] {'device': <onvif.client.ONVIFCamera object at 0x7f00c6fddbd0>, 'device_mgmt': <onvif.client.ONVIFService object at 0x7f00c6fde6e0>}
2023-04-19 10:17:11.160 DEBUG (MainThread) [custom_components.tapo_control] setupOnvif - entry
2023-04-19 10:17:11.160 DEBUG (MainThread) [custom_components.tapo_control] Setting up onvif...
2023-04-19 10:17:11.160 DEBUG (MainThread) [custom_components.tapo_control] setupEvents - entry
2023-04-19 10:17:11.160 DEBUG (MainThread) [custom_components.tapo_control] Setting up events...
2023-04-19 10:17:15.275 DEBUG (MainThread) [custom_components.tapo_control] Events started.
2023-04-19 10:17:15.275 DEBUG (MainThread) [custom_components.tapo_control] Creating binary sensor entity.
2023-04-19 10:17:15.275 DEBUG (MainThread) [custom_components.tapo_control] Binary sensor creation for motion has been forwarded to component.

This shows us the difference in behaviour in C200 on 1.3.5 and C210 1.3.0, where C210 allows to be subscribed into (the second "setupEvents - entry" followed up by "Events started.") and C200 does not (no "Events started.").

This integration is using onvif from HA, so it will need to be fixed upstream (if there is anything to fix). It is also possible that there is a bug in the new firmware that bricked the onvif. At this point I would recommend:

1. Testing with official onvif integration, if it does not work, open issue in HA github for onvif integration
2. Downgrading firmware back to 1.3.0 or below which is confirmed to work.

[silfax]

Any advice/link on how the downgrade can be done? Couldn't find anything really useful. Thanks!

[JurajNyiri]

nervous-inhuman/tplink-tapo-c200-re#4 (comment)
Discord: https://discord.com/channels/763866796603342848/763867553348976680/1092165428526207037 (or, ideally, ask TPLink if it is an issue they caused, they should provide fw links).

[mitzu2250]

Just checked with the main ONVIF integration. I can confirm that for my C200 camera, the "Camera Living Cell Motion Detection" entity doesn't pop up when motion is happening but it works good on C210.

I'll open a issue on ONVIF side to see if this can be solved there or it's a TP-Link issue.

[thomasramm]

With the latest updates (Alpha Channel), the motion sensor starts working with my C320WS (latest Firmware: 1.0.4 Build 221031 Rel.40874n).
I'm currently getting some new error messages that I haven't had before, but it still works, so not a big deal for me - and i'm on the alpha version.

The error messages i get are:

    if not hass.data[DOMAIN][config_entry.entry_id]["events"].started:
AttributeError: 'bool' object has no attribute 'started'

The second error is:

    hass.data[DOMAIN][entry.entry_id]["events"] = EventManager(
TypeError: EventManager.__init__() missing 1 required positional argument: 'name'

I don't have any error messages before the last updates.

Home Assistant 2023.5.0.dev20230424
Supervisor 2023.04.1
Operating System 11.0.dev20230420
Frontend 20230424.0.dev - latest
Tapo: Cameras Control 5.1.2

[JurajNyiri]

New version for testing is available at https://github.com/JurajNyiri/HomeAssistant-Tapo-Control/releases/tag/5.1.3. Only usable with DEV version of HA or 2023.05.0 and higher if you are reading this from the future.

[JurajNyiri]

@thomasramm just released new version, could you please test?

[thomasramm]

Tested Version 5.1.3: no error messages!
Make some reboots, motion detections with saving snapshots, try the live stream and change some settings. Everything works well.

I Forget to write, that the binary sensor "TapoCamera Motion" is shown in the system, but did not work (a message says it is not available and i should remove it). But the Onvif "Cell Motion detection" is working, so this is not a problem.
(I have add the Tapo Integration and the Onvif Integration for the same cam. Both Entities are shown together on the same device).

Yesterday i get 500 motion detections, it works stable at my installation. I'm very happy :-)

Tested with this configuration:
Home Assistant 2023.5.0.dev20230425
Supervisor 2023.04.1
Operating System 11.0.dev20230424
Frontend 20230425.0.dev - latest

[JurajNyiri]

It should be working just from this integration. It is possible it is not working for you because you are hitting it twice at the same time. Try removing onvif and using just Tapo, it should expose motion sensor.
Test for at least an hour or so, camera is really unstable when ending and creating new onvif sessions (at least my c200 is).
If not working please enable debug logs and attach here.

[silfax]

2023.4 here. Do I have to wait until I will update to 2023.5? Thanks.

[JurajNyiri]

New version is usable with DEV version of HA or 2023.5.0 and higher if you are reading this from the future.
So if you want to use stable version and you don't have time machine, yes. 😁

Edit: Also of significant note, issue is now marked on HA as device issue, so you might need to wait for tplink to release new firmware. New HA onvif release fixes just some issues, not all.

[thomasramm]

It should be working just from this integration. It is possible it is not working for you because you are hitting it twice at the same time. Try removing onvif and using just Tapo, it should expose motion sensor. Test for at least an hour or so, camera is really unstable when ending and creating new onvif sessions (at least my c200 is). If not working please enable debug logs and attach here.

you are right, i remove the onvif integration, after some reboots or after a longer time (doesn't know which was the reason) the Tapo Motion Sensor starts working.
Thanks to your support and your hard work on this integration.

Thomas

[orwoods]

@thomasramm Hi! Could you help me please?
I have this configuration:

Home Assistant 2023.5.0.dev20230426
Supervisor 2023.04.1
Frontend 20230426.0.dev - latest

but HACS doesn't install Tapo: Cameras Control integration.
When I trying to (re)install integration I see next error message in the log file:

2023-04-26 22:07:55.614 ERROR (MainThread) [homeassistant.components.websocket_api.http.connection] [281472534378960] Error handling message: Unknown error (unknown_error) Ivan from 127.0.0.1 (Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Home Assistant/2023.4 (io.robbie.HomeAssistant; build:2023.460; macOS 13.3.1) Mobile/HomeAssistant, like Safari)
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/decorators.py", line 26, in _handle_async_response
    await func(hass, connection, msg)
  File "/config/custom_components/hacs/websocket/repository.py", line 216, in hacs_repository_download
    await repository.async_install()
  File "/config/custom_components/hacs/repositories/base.py", line 899, in async_install
    await self.async_install_repository()
  File "/config/custom_components/hacs/repositories/base.py", line 939, in async_install_repository
    raise HacsException("The version of Home Assistant is not compatible with this version")
custom_components.hacs.exceptions.HacsException: The version of Home Assistant is not compatible with this version

[JurajNyiri]

Follow manual installation steps.

[JurajNyiri]

@Agimgrue thank you for the report, but please use the latest version when reporting issues.

[Agimgrue]

@Agimgrue thank you for the report, but please use the latest version when reporting issues.

Sorry, I don't see that there was an update.

Updated to 5.1.8 version and binary sensor becomes unavailable.

Home Assistant 2023.5.1
Supervisor 2023.04.1
Operating System 10.1
Frontend 20230503.2 - latest

Tapo: Cameras Control
5.1.8

Tapo C200
HW 1.0
FW 1.3.5 Build 230307 Rel.6847n(4555)

I'll try to downgrade to Tapo: Cameras Control 5.1.7

[JurajNyiri]

Make sure to always restart camera between HA restarts.

[fightforlife]

I had this issue once today where multiple sensors where shown but none where available.
For me it was fixed by closing all streams accessing the camera and then reinstalling the integration.

[JurajNyiri]

Issue has now been fixed in 5.2.0.

[MunkeyBalls]

Strange, I had it working again on 5.1.8 (C200 3.0 1.3.4 & C210 2.0 1.3.1) and just updated to 5.2.0 and motion sensor is not showing up anymore.

HA is updated to 2023.5.2

2023-05-06 00:52:28.072 ERROR (MainThread) [custom_components.tapo_control] [initOnvifEvents] Initiating onvif connection failed.

[JurajNyiri]

@MunkeyBalls are you running HA with certificate on HTTPS?

[MunkeyBalls]

I load the HA webui through a https proxy, should I try re-add the integration without HTTPS?

[JurajNyiri]

What is your internal HA url? Does your HA instance work on http://<your ip>:8123?

[MunkeyBalls]

I should be able to connect directly. I have a local DNS record and connect via that for ease of use (https://home.lan)

[JurajNyiri]

Does your HA instance work on http://<your ip>:8123?

[MunkeyBalls]

Yes, I can connect via http://ip:8123
I just reverted back to 5.1.8 and everything is working again.

[JurajNyiri]

Make sure you have correct IP set for Home Assistant. Turn on Advanced Mode under /profile. Go to /config/network and under Network Adapter verify correct IP is shown for the device. If it is not correct, under Home Assistant URL uncheck Automatic next to Local Network and set it to http://<IP address>:8123. DO NOT USE HTTPS.

[MunkeyBalls]

Could having the camera on a separate VLAN which cannot reach HA nor the internet have anything to do with it?

[JurajNyiri]

Yes, in order for webhooks to work camera needs to reach HA and you need to have correct (non-https) internal reachable URL set.

[MunkeyBalls]

Mystery solved then :P

I guess I'm going to have to set some firewall rules for the camera to be able to reach HA.
5.1.8 seems to be working fine for me however, maybe make it possible to choose the old method of fetching the sensor/events?

[JurajNyiri]

5.1.8 should be avoided, it uses really old onvif integration and bundles both onvif-zeep and old onvif integration. I am working on a better solution.

[MunkeyBalls]

I'll do some tweaking on the internal network tomorrow and give 5.2.0 another spin. Thanks for resolving this issue so quickly, amazing how quickly you and bdraco pushed out those new versions.

[JurajNyiri]

@MunkeyBalls you can try with 5.2.1, if it detects usage of https it disables webhooks. If you can reach HA through http:IP:8123 and you have a http internal URL (read FAQ), and fix vlan so that camera can reach it, you should be able to use webhooks to the fullest potential.

If it detects you are actually using https, it automatically disables webhook, which is not an issue on all cameras except 1.3.6 which has broken pullpoint.

[MunkeyBalls]

Thanks! I'll give it a try tomorrow. Nothing good is going to come from me messing around with my network this late in the night 😄

[JurajNyiri]

I am going to copy&paste part of readme/faq here for awareness;

• If you have webhooks enabled, and your Home Assistant internal URL is reachable on HTTP, make sure camera can reach it.
• Make sure you have correct IP set for Home Assistant. Turn on Advanced Mode under /profile. Go to /config/network and under Network Adapter verify correct IP is shown for the device. If it is not correct, under Home Assistant URL uncheck Automatic next to Local Network and set it to http://<some IP address>:8123. DO NOT USE HTTPS.
• Certain camera firmwares have pullpoint broken, with only webhooks working. If you are not able to run webhooks because of above (https, or vlan setup), binary sensor will never show up.

[fightforlife]

Working fine for me, only the entity name changed. From ...._motion to ..._motion_alarm

[aIexus]

@JurajNyiri
Hi!
As for 5.2.0 and HA 2023.5.2 - two motion sensors, but with different names:

• binary_sensor.tapo_camera_cell_motion_detection
• binary_sensor.tapo_camera_motion_alarm

I understand 'alarm' one, but what is 'cell motion detection'?

[JurajNyiri]

That works as expected, both are reported by camera. "Cell motion detection" is the one we always had. I do not know the purppse of motion alarm, it is something tapo added recently.

[romaruss]

Home Assistant 2023.5.2 Supervisor 2023.04.1 Operating System 10.1 Frontend 20230503.3 - latest
Integration Version 5.2.2
Camera C200
HW 1.0
Firmware 1.3.6 Build 230424 Rel 77225n(4555)

Motion Sensor still not exists
in preview version sensor become stuck so i deleted it (as suggested by the frontend).
Now i've updated integration and core but sensor don't appear.
I tryed removing integration, restart HA and camera.
Maybe is there something i can do for make sensor become available again?

[fightforlife]

I noticed the sensor not being available directly after installing the integration.
The two motion sensors became available when they detected the first movement.
So movement detection needs to be enabled and you need to move once in front of the camera.

[romaruss]

I noticed the sensor not being available directly after installing the integration. The two motion sensors became available when they detected the first movement. So movement detection needs to be enabled and you need to move once in front of the camera.

tryed to do.
i also activate cloud account and webhooks (i think correctly)
Here is the debug log
home-assistant_tapo_control_2023-05-07T22-13-48.783Z.log