ONVIF: TP-Link Tapo Camera Motion not working after firmware upgrade (port 1024 is unresponsive in newer firmware, webhook messages not delivered) #91661

mitzu2250 · 2023-04-19T10:05:32Z

Working firmware table : JurajNyiri/HomeAssistant-Tapo-Control#312 (comment)

The problem

Hello.

I have 2 different cameras:
1 C200 running on 1.3.5 Build 230307 Rel.6847n(4555) contains string 'hol' in it's name
1 C210 running on 1.3.0 Build 220830 Rel.69909n(4555) contains string 'living' in it's name

For some time, the motion sensor from C200 is not working. Usually when I had this issue, it was fixed after a restart of HA or reboot of the camera.
The C210 one is still working as expected.

I have enabled the debug on the two entities and got the attached log.
I cannot see any ERRORS, maybe you can spot something that it shouldn't be there.

This has already been discussed in the Tapo Control issues : JurajNyiri/HomeAssistant-Tapo-Control#312

What version of Home Assistant Core has the issue?

core-2023.4.5

What was the last working version of Home Assistant Core?

core-2023.4.5

What type of installation are you running?

Home Assistant OS

Integration causing the issue

ONVIF

Link to integration documentation on our website

https://www.home-assistant.io/integrations/onvif/

Diagnostics information

home-assistant_2023-04-19T10-04-13.732Z.log

Example YAML snippet

No response

Anything in the logs that might be useful for us?

Please check https://github.com/JurajNyiri/HomeAssistant-Tapo-Control/files/11270254/home-assistant_2023-04-19T07-20-50.421Z.log for more logs.

Additional information

Might not be ONVIF fault, maybe it's a bad firmware from TP-Link?

home-assistant · 2023-04-19T10:05:37Z

Hey there @hunterjm, mind taking a look at this issue as it has been labeled with an integration (onvif) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of onvif can trigger bot actions by commenting:

@home-assistant close Closes the issue.
@home-assistant rename Awesome new title Renames the issue.
@home-assistant reopen Reopen the issue.
@home-assistant unassign onvif Removes the current integration label and assignees on the issue, add the integration domain after the command.

_{^{(message by CodeOwnersMention)}}

onvif documentation
onvif source
_{^{(message by IssueLinks)}}

bdraco · 2023-04-19T10:09:57Z

Make sure the custom component is up to date as older versions pin the onvif library to the wrong version

If that’s not it try unplugging the camera for a minute and plugging it back in. Reasoning for this detailed here #91485

bdraco · 2023-04-19T10:11:36Z

#37545 (comment)

mitzu2250 · 2023-04-19T10:27:35Z

Thanks for the replies.
I can confirm that the custom component is up to date.
Tried multiple troubleshooting methods, restarted HA, rebooted camera, unplugged / plugged it, nothing works.

As mentioned, the only difference between the camera that is working and the one that doesn't is the firmware version so the problem might be actually by TP-Link Tapo, but not sure if something can be done on ONVIF side also.

Can you please let me know how can I set up the onvif custom integration after copying the files to custom_components/onvif ? I only get the original one while searching for Integrations.

I found this in the logs though:
2023-04-19 13:24:18.145 ERROR (SyncWorker_2) [homeassistant.loader] The custom integration 'onvif' does not have a version key in the manifest file and was blocked from loading. See https://developers.home-assistant.io/blog/2021/01/29/custom-integration-changes#versions for more details

xhemp · 2023-04-19T11:58:26Z

This looks to be tied to the firmware version, I have a C210 with firmware 1.3.3 and my motion doesn't work.
I added a version: 1 to the manifest.json just to be able to load ONVIF as a custom_module (as per bdraco's comment) but motion is still not working

Edit: Interesting, I can now see errors are gone and I can see the motion working!

mitzu2250 · 2023-04-19T12:09:59Z

I turned on the Diagnostics of the faulty camera and extracted this log that was generated while I was moving in front of the camera.
I see a lot of errors but not sure if any of these are relevant but this one looks relevant:
[__tev__CreatePullPointSubscription:5994]: the amount of MD threads is up to limit :3

diagnose_log.log

If needed, I can run a full diagnostic on the camera, if you think this would tell something.

bdraco · 2023-04-19T19:37:15Z

I forgot to add the version key to the manifest when I made it a custom component. Please try pulling again

mkdir -p /config/custom_components
cd /config/custom_components
git clone https://github.com/bdraco/onvif
cd /config/custom_components/onvif
git pull
git log -1

Restart

The git log -1 should should 7c65f84f1269b814bf9b61eba116eaf04d0e13cb

AndreiArdelean1 · 2023-04-20T06:51:08Z

Motion detection works reliably with this pull re. But, I get this error:

2023-04-20 09:39:09.611 DEBUG (MainThread) [custom_components.onvif] Tapo_Camera_8A28: Device does not support notification service or too many subscriptions: illegal status line: bytearray(b'POST /onvif/service\x00HTTP/1.1\x00\x00Host\x00 192.168.0.129:2020\x00\x00Accept\x00 */*\x00\x00Accept-Encoding\x00 gzip, deflate, br\x00\x00Connection\x00 keep-alive\x00\x00User-Agent\x00 ZeHTTP/1.1 500 Internal Server Error')

Looks somewhat related (both have 500 Internal Server Error) with #85902.

Here are the full logs

mitzu2250 · 2023-04-20T06:52:27Z

I tried this, had it run over night.
On my side the motion detection for the new firmware still doesn't work and the one which works gets stuck on 'detected'.

Logs attached:
home-assistant_2023-04-20T06-48-51.857Z.log

Will try some HA/camera restarts to see if this changes.

bdraco · 2023-04-20T06:58:24Z

2023-04-20 09:39:09.611 DEBUG (MainThread) [custom_components.onvif] Tapo_Camera_8A28: Device does not support notification service or too many subscriptions: illegal status line: bytearray(b'POST /onvif/service\x00HTTP/1.1\x00\x00Host\x00 192.168.0.129:2020\x00\x00Accept\x00 */*\x00\x00Accept-Encoding\x00 gzip, deflate, br\x00\x00Connection\x00 keep-alive\x00\x00User-Agent\x00 ZeHTTP/1.1 500 Internal Server Error')

Thats not good. It looks like there might be a bug in the firmware

bdraco · 2023-04-20T06:59:15Z

I tried this, had it run over night. On my side the motion detection for the new firmware still doesn't work and the one which works gets stuck on 'detected'.

Logs attached: home-assistant_2023-04-20T06-48-51.857Z.log

Will try some HA/camera restarts to see if this changes.

Anything interesting in the camera diagnostics ? Its almost like the camera is crashing and restarting

mitzu2250 · 2023-04-20T07:25:56Z

Performed the following steps:

Stopped HA entirely (shut down)
Unplugged both cameras and then waited some minutes, plugged back
Started HA

First observation: the 'Starting ONVIF, not everything will be available until it is finished.' message was there for a while.
Had to restart again because the message 'Home Assistant is starting, not everything will be available until it is finished.' didn't go away for a while.
ONVIF motion sensor on the camera which was working before isn't detecting anything.
Nothing in the HA logs. home-assistant_2023-04-20T07-15-33.419Z.log

After restart:
The following integrations and platforms could not be set up:
onvif ([Show logs](http://homeassistant.local:8123/config/logs?filter=
But nothing is displayed in the logs. home-assistant_2023-04-20T07-19-33.838Z.log
onvif))

Motion sensor is still unavailable for the new firmware, old one works good.
Attaching the broken camera diagnose.
diagnose_log.log

At this point I tend to believe that it's a firmware issue that cannot be fixed on our side.

bdraco · 2023-04-20T07:29:15Z

At this point I tend to believe that it's a firmware issue that cannot be fixed on our side.

Based on the above, I agree.

bdraco · 2023-04-20T07:31:03Z

2023-04-20 09:39:09.611 DEBUG (MainThread) [custom_components.onvif] Tapo_Camera_8A28: Device does not support notification service or too many subscriptions: illegal status line: bytearray(b'POST /onvif/service\x00HTTP/1.1\x00\x00Host\x00 192.168.0.129:2020\x00\x00Accept\x00 */*\x00\x00Accept-Encoding\x00 gzip, deflate, br\x00\x00Connection\x00 keep-alive\x00\x00User-Agent\x00 ZeHTTP/1.1 500 Internal Server Error')

Thats not good. It looks like there might be a bug in the firmware

I updated the firmware on my C320WS and now I have the same problem. If I find a workaround, I'll post it, but I think we have to wait for them to fix the firmware at this point.

mitzu2250 · 2023-04-20T07:54:01Z

I have sent an email to local TP-Link support email support.ro@tp-link.com

I'll get back if I get any replies.

orwoods · 2023-04-20T08:29:40Z

I have the same problem with my C320WS. Maybe you need some help with the investigation?

mitzu2250 · 2023-04-28T13:07:51Z

From my emails to TP-Link, the Tapo developers are stating that there is no change on the ONVIF part of the cameras in the latest 1.3.x firmware.
Looks like there was a fix found in this topic : JurajNyiri/HomeAssistant-Tapo-Control#312

Given this, looks like it's an issue on Home Assistant side, as the ONVIF is working with other software like iSpy as per these instructions on the official TP-Link website https://www.tp-link.com/en/support/faq/2680/

bdraco · 2023-04-28T13:14:29Z

Thats strange since all the 320s I have on the newer firmware throw 500 internal errors and returns back data that looks like its been corrupted in memory.

bdraco · 2023-04-28T13:16:18Z

Its certainly possible that the memory corruption has nothing to do with the onvif implementation in the camera and its happening deeper in the stack.

JurajNyiri · 2023-04-28T13:19:08Z

Looks like there was a fix found in this topic : JurajNyiri/HomeAssistant-Tapo-Control#312

If the issue is not in firmware, it will need to be fixed here -- in HA onvif. My last updates just implement the new onvif version introduced in the latest dev versions. If you would like to help further, enable debug logs for onvif with latest DEV version of HA and post them here. Make sure to disable tapo integration and use only onvif, to prevent noise or wrong implementation in Tapo custom integration.

bdraco · 2023-04-28T13:23:17Z

Also events do work fine for me on the 320s with pull point. Its only the notification subscriptions (webhooks) that throw the 500 errors.

bdraco · 2023-04-28T13:24:34Z

I don't have a 200 to test with so its possible there is another issue but without logs I can only tell you what I see with the 320

MunkeyBalls · 2023-04-30T07:41:52Z

I don't have a 200 to test with so its possible there is another issue but without logs I can only tell you what I see with the 320

I've got a C200 a since yesterday and was stupid enough to update it out of the box to the latest firmware: 1.3.4 Build 230424 Rel.76394n(4555). I tried the custom Tapo component and ran into the same issue as others, motion detection not working.

To maybe help debug the issue I updated to the dev branch of homeassistant, removed the tapo component and added the official ONVIF component. After a few minutes a binary_sensor.camera_cell_motion_detection sensor pops up, but it looks like it's stuck in "Detected" state.

Here's a few snippets from the logs:

2023-04-30 09:26:20.234 DEBUG (MainThread) [homeassistant.components.onvif] catfinder: Failed to unsubscribe PullPoint subscription; This is normal if the device restarted: Device sent empty error
2023-04-30 09:26:20.234 DEBUG (MainThread) [homeassistant.components.onvif] catfinder: Unsubscribing from webhook
2023-04-30 09:26:20.286 DEBUG (MainThread) [zeep.transports] HTTP Response from http://192.168.24.159:2020/onvif/service (status: 200):
2023-04-30 09:26:20.287 DEBUG (MainThread) [onvif] Closing service ('subscription', 'PullPointSubscription') with http://192.168.24.159:1024/event-1024_1024
2023-04-30 09:26:20.287 DEBUG (MainThread) [onvif] Creating service ('subscription', 'PullPointSubscription') with http://192.168.24.159:1025/event-1025_1025
2023-04-30 09:26:20.287 DEBUG (MainThread) [onvif] Closing service ('pullpoint', 'PullPointSubscription') with http://192.168.24.159:1024/event-1024_1024
2023-04-30 09:26:20.288 DEBUG (MainThread) [onvif] Creating service ('pullpoint', 'PullPointSubscription') with http://192.168.24.159:1025/event-1025_1025
2023-04-30 09:26:20.308 DEBUG (MainThread) [homeassistant.components.onvif] catfinder: Unregistering webhook onvif_32d119873a67ddd1423f17ebcc3f2852
2023-04-30 09:26:20.312 DEBUG (MainThread) [onvif] Creating service ('devicemgmt', None) with http://catfinder.lan:2020/onvif/device_service
2023-04-30 09:26:20.314 DEBUG (MainThread) [zeep.transports] HTTP Post to http://catfinder.lan:2020/onvif/device_service:
2023-04-30 09:26:20.351 DEBUG (MainThread) [zeep.transports] HTTP Response from http://catfinder.lan:2020/onvif/device_service (status: 200):
2023-04-30 09:26:20.354 DEBUG (MainThread) [homeassistant.components.onvif] catfinder: Setting up the ONVIF device management service
2023-04-30 09:26:20.354 DEBUG (MainThread) [homeassistant.components.onvif] catfinder: Retrieving current device date/time
2023-04-30 09:26:20.355 DEBUG (MainThread) [zeep.transports] HTTP Post to http://catfinder.lan:2020/onvif/device_service:
2023-04-30 09:26:20.356 DEBUG (MainThread) [homeassistant.components.onvif] catfinder: SetSynchronizationPoint: None
2023-04-30 09:26:20.382 DEBUG (MainThread) [zeep.transports] HTTP Response from http://catfinder.lan:2020/onvif/device_service (status: 200):
2023-04-30 09:26:20.383 DEBUG (MainThread) [homeassistant.components.onvif] catfinder: Device time: {
2023-04-30 09:26:20.384 DEBUG (MainThread) [homeassistant.components.onvif] catfinder: Device date/time: 2023-04-24 14:45:40+00:00 | System date/time: 2023-04-30 07:26:20.354528+00:00
2023-04-30 09:26:20.384 WARNING (MainThread) [homeassistant.components.onvif] The date/time on catfinder (UTC) is '2023-04-24 14:45:40+00:00', which is different from the system '2023-04-30 07:26:20.354528+00:00', this could lead to authentication issues
2023-04-30 09:26:20.385 DEBUG (MainThread) [zeep.transports] HTTP Post to http://catfinder.lan:2020/onvif/device_service:
2023-04-30 09:26:20.404 DEBUG (MainThread) [zeep.transports] HTTP Response from http://catfinder.lan:2020/onvif/device_service (status: 200):
2023-04-30 09:26:20.406 DEBUG (MainThread) [zeep.transports] HTTP Post to http://catfinder.lan:2020/onvif/device_service:
2023-04-30 09:26:20.426 DEBUG (MainThread) [zeep.transports] HTTP Response from http://catfinder.lan:2020/onvif/device_service (status: 200):
2023-04-30 09:26:20.427 DEBUG (MainThread) [homeassistant.components.onvif] Camera catfinder info = DeviceInfo(manufacturer='tp-link', model='Tapo C200', fw_version='1.3.4 Build 230424 Rel.76394n(4555)', serial_number='', mac='')
2023-04-30 09:26:20.427 DEBUG (MainThread) [onvif] Creating service ('media', None) with http://192.168.24.159:2020/onvif/service
2023-04-30 09:26:20.429 DEBUG (MainThread) [zeep.transports] HTTP Post to http://192.168.24.159:2020/onvif/service
2023-04-30 09:26:20.435 DEBUG (MainThread) [zeep.transports] HTTP Response from http://192.168.24.159:2020/onvif/service (status: 200):
2023-04-30 09:26:20.436 DEBUG (MainThread) [onvif] Creating service ('imaging', None) with http://192.168.24.159:2020/onvif/service
2023-04-30 09:26:20.437 DEBUG (MainThread) [homeassistant.components.onvif] catfinder: Starting PullPoint manager
2023-04-30 09:26:20.437 DEBUG (MainThread) [onvif] Creating service ('events', None) with http://192.168.24.159:2020/onvif/service
2023-04-30 09:26:20.438 DEBUG (MainThread) [zeep.transports] HTTP Post to http://192.168.24.159:2020/onvif/service:
2023-04-30 09:26:20.452 DEBUG (MainThread) [zeep.transports] HTTP Response from http://192.168.24.159:2020/onvif/service (status: 200):
2023-04-30 09:26:20.453 DEBUG (MainThread) [onvif] Creating service ('subscription', 'PullPointSubscription') with http://192.168.24.159:1026/event-1026_1026
2023-04-30 09:26:20.468 DEBUG (MainThread) [homeassistant.components.onvif] catfinder: SetSynchronizationPoint: None
2023-04-30 09:26:20.469 DEBUG (MainThread) [homeassistant.components.onvif] catfinder: Starting webhook manager
2023-04-30 09:26:20.469 DEBUG (MainThread) [homeassistant.components.onvif] catfinder: Registering webhook: onvif_32d119873a67ddd1423f17ebcc3f2852
2023-04-30 09:26:20.469 DEBUG (MainThread) [homeassistant.components.onvif] catfinder: Registered webhook: onvif_32d119873a67ddd1423f17ebcc3f2852
2023-04-30 09:26:20.469 DEBUG (MainThread) [homeassistant.components.onvif] catfinder: Creating webhook subscription
2023-04-30 09:26:20.469 DEBUG (MainThread) [onvif] Creating service ('notification', None) with http://192.168.24.159:2020/onvif/service
2023-04-30 09:26:20.471 DEBUG (MainThread) [zeep.transports] HTTP Post to http://192.168.24.159:2020/onvif/service:
2023-04-30 09:26:20.594 DEBUG (MainThread) [zeep.transports] HTTP Response from http://192.168.24.159:2020/onvif/service (status: 200):
2023-04-30 09:26:20.595 DEBUG (MainThread) [onvif] Creating service ('pullpoint', 'NotificationConsumer') with http://192.168.24.159:2020/event-0_2020
2023-04-30 09:26:20.595 DEBUG (MainThread) [onvif] Creating service ('subscription', 'NotificationConsumer') with http://192.168.24.159:2020/event-0_2020
2023-04-30 09:26:20.617 DEBUG (MainThread) [homeassistant.components.onvif] catfinder: Webhook subscription created
2023-04-30 09:26:20.617 DEBUG (MainThread) [homeassistant.components.onvif] Camera catfinder capabilities = Capabilities(snapshot=False, events=True, ptz=True, imaging=True)
2023-04-30 09:26:20.618 DEBUG (MainThread) [zeep.transports] HTTP Post to http://192.168.24.159:2020/onvif/service:
2023-04-30 09:26:20.671 DEBUG (MainThread) [zeep.transports] HTTP Response from http://192.168.24.159:2020/onvif/service (status: 200):
2023-04-30 09:26:20.678 DEBUG (MainThread) [onvif] Creating service ('ptz', None) with http://192.168.24.159:2020/onvif/service
2023-04-30 09:26:20.679 DEBUG (MainThread) [zeep.transports] HTTP Post to http://192.168.24.159:2020/onvif/service:
2023-04-30 09:26:20.712 DEBUG (MainThread) [zeep.transports] HTTP Response from http://192.168.24.159:2020/onvif/service (status: 200):
2023-04-30 09:26:20.715 DEBUG (MainThread) [zeep.transports] HTTP Post to http://192.168.24.159:2020/onvif/service:
2023-04-30 09:26:20.736 DEBUG (MainThread) [zeep.transports] HTTP Response from http://192.168.24.159:2020/onvif/service (status: 200):
2023-04-30 09:26:20.738 DEBUG (MainThread) [homeassistant.components.onvif] Camera catfinder profiles = [Profile(index=0, token='profile_1', name='mainStream', video=Video(encoding='H264', resolution=Resolution(width=1920, height=1080)), ptz=PTZ(continuous=True, relative=True, absolute=True, presets=['1', '2', '3', '4', '5', '6', '7', '8']), video_source_token='raw_vs1'), Profile(index=1, token='profile_2', name='minorStream', video=Video(encoding='H264', resolution=Resolution(width=640, height=360)), ptz=PTZ(continuous=True, relative=True, absolute=True, presets=['1', '2', '3', '4', '5', '6', '7', '8']), video_source_token='raw_vs1')]
2023-04-30 09:26:21.489 DEBUG (MainThread) [homeassistant.components.onvif] catfinder: Pulling PullPoint messages timeout=0:01:00 limit=100
2023-04-30 09:27:22.022 DEBUG (MainThread) [homeassistant.components.onvif] catfinder: continuous PullMessages: no events
2023-04-30 09:27:22.773 DEBUG (MainThread) [homeassistant.components.onvif] catfinder: Pulling PullPoint messages timeout=0:01:00 limit=100
2023-04-30 09:27:50.635 DEBUG (MainThread) [homeassistant.components.onvif] catfinder: Renewed Webhook subscription
2023-04-30 09:28:16.800 DEBUG (MainThread) [homeassistant.components.onvif] catfinder: continuous PullMessages: 1 event(s)
2023-04-30 09:28:16.805 DEBUG (MainThread) [homeassistant.components.onvif] catfinder: Renewed PullPoint subscription
2023-04-30 09:28:17.551 DEBUG (MainThread) [homeassistant.components.onvif] catfinder: Pulling PullPoint messages timeout=0:01:00 limit=100
2023-04-30 09:28:43.993 DEBUG (MainThread) [zeep.transports] HTTP Post to http://192.168.24.159:2020/onvif/service:
2023-04-30 09:28:44.011 DEBUG (MainThread) [zeep.transports] HTTP Response from http://192.168.24.159:2020/onvif/service (status: 200):
2023-04-30 09:29:47.558 DEBUG (MainThread) [homeassistant.components.onvif] catfinder: Failed to fetch PullPoint subscription messages: Device sent empty error
2023-04-30 09:29:47.558 DEBUG (MainThread) [homeassistant.components.onvif] catfinder: PullPoint renew already in progress
2023-04-30 09:29:48.309 DEBUG (MainThread) [homeassistant.components.onvif] catfinder: Pulling PullPoint messages timeout=0:01:00 limit=100
2023-04-30 09:08:13.527 DEBUG (MainThread) [onvif] Creating service ('subscription', 'NotificationConsumer') with http://192.168.24.159:2020/event-0_2020

I removed the SOAP messages from the logs, let me know if you need more info.

bdraco · 2023-04-30T15:02:53Z

Tapo C200
by tp-link
Firmware: 1.3.3 Build 230228 Rel.79740n(5553)

I bought a C200 and it has the above firmware and it worked out of box with 2023.5.0b6

MunkeyBalls · 2023-04-30T15:14:26Z

Tapo C200 by tp-link Firmware: 1.3.3 Build 230228 Rel.79740n(5553)

I bought a C200 and it has the above firmware and it worked out of box with 2023.5.0b6

You don't have a update pending for 1.3.4? If so, could you check if that breaks it?

JurajNyiri · 2023-05-04T14:09:44Z

Open sanity checks:

jnyiri@Jurajs-MacBook-Air ~ % nmap -p 443 192.168.100.70
Starting Nmap 7.93 ( https://nmap.org ) at 2023-05-04 16:06 CEST
Nmap scan report for 192.168.100.70
Host is up (0.078s latency).

PORT    STATE SERVICE
443/tcp open  https

Nmap done: 1 IP address (1 host up) scanned in 0.14 seconds

jnyiri@Jurajs-MacBook-Air ~ % nmap -p 8800 192.168.100.70
Starting Nmap 7.93 ( https://nmap.org ) at 2023-05-04 16:06 CEST
Nmap scan report for 192.168.100.70
Host is up (0.0082s latency).

PORT     STATE SERVICE
8800/tcp open  sunwebadmin

Nmap done: 1 IP address (1 host up) scanned in 0.08 seconds

jnyiri@Jurajs-MacBook-Air ~ % nmap -p 2020 192.168.100.70
Starting Nmap 7.93 ( https://nmap.org ) at 2023-05-04 16:06 CEST
Nmap scan report for 192.168.100.70
Host is up (0.0068s latency).

PORT     STATE SERVICE
2020/tcp open  xinupageserver

Nmap done: 1 IP address (1 host up) scanned in 0.06 seconds

Close sanity checks:

jnyiri@Jurajs-MacBook-Air ~ % nmap -p 1234 192.168.100.70
Starting Nmap 7.93 ( https://nmap.org ) at 2023-05-04 16:10 CEST
Nmap scan report for 192.168.100.70
Host is up (0.0093s latency).

PORT     STATE    SERVICE
1234/tcp filtered hotline

Nmap done: 1 IP address (1 host up) scanned in 0.26 seconds

jnyiri@Jurajs-MacBook-Air ~ % nmap -p 5678 192.168.100.70
Starting Nmap 7.93 ( https://nmap.org ) at 2023-05-04 16:10 CEST
Nmap scan report for 192.168.100.70
Host is up (0.013s latency).

PORT     STATE    SERVICE
5678/tcp filtered rrac

Nmap done: 1 IP address (1 host up) scanned in 0.27 seconds

jnyiri@Jurajs-MacBook-Air ~ % nmap -p 0987 192.168.100.70
Starting Nmap 7.93 ( https://nmap.org ) at 2023-05-04 16:10 CEST
Nmap scan report for 192.168.100.70
Host is up (0.024s latency).

PORT    STATE    SERVICE
987/tcp filtered unknown

Nmap done: 1 IP address (1 host up) scanned in 0.42 seconds

1024:

jnyiri@Jurajs-MacBook-Air ~ % nmap -p 1024 192.168.100.70
Starting Nmap 7.93 ( https://nmap.org ) at 2023-05-04 16:06 CEST
Nmap scan report for 192.168.100.70
Host is up (0.074s latency).

PORT     STATE    SERVICE
1024/tcp filtered kdm

Nmap done: 1 IP address (1 host up) scanned in 0.89 seconds

So I would say it is closed.

Let me know if I should run something else.

bdraco · 2023-05-04T14:29:15Z

Working camera

 % nmap -p 1024 192.168.214.156
Starting Nmap 7.93 ( https://nmap.org ) at 2023-05-04 09:28 CDT
Nmap scan report for C200_99A9B8.koston.org (192.168.214.156)
Host is up (0.0021s latency).

PORT     STATE SERVICE
1024/tcp open  kdm

So its pretty clear that they broke the service on port 1024 in the newer firmware

JurajNyiri · 2023-05-04T14:32:09Z

The sensor is working on C200 HW3.0 on 1.3.4;

And it has a CLOSED response for me compared to OPEN on yours and FILTERED on 1.3.6 HW 1.0

jnyiri@Jurajs-MacBook-Air ~ % nmap -p 1024 192.168.100.105
Starting Nmap 7.93 ( https://nmap.org ) at 2023-05-04 16:30 CEST
Nmap scan report for 192.168.100.105
Host is up (0.094s latency).

PORT     STATE  SERVICE
1024/tcp closed kdm

Nmap done: 1 IP address (1 host up) scanned in 0.15 seconds

What does this mean? Is the filtered the issue?

1.3.4 HW 3.0:

Sanity check OPEN:
443: OPEN
8800: OPEN
2020: OPEN

Sanity check CLOSE:
1234: CLOSED
5678: CLOSED
987: CLOSED

PORT 1234:
1024: CLOSED

1.3.6 HW 1.0:

On 1.3.4 HW 3.0:

Sanity check OPEN:
443: OPEN
8800: OPEN
2020: OPEN

Sanity check CLOSE:
1234: FILTERED
5678: FILTERED
987: FILTERED

PORT 1234:
1024: FILTERED

bdraco · 2023-05-04T14:44:08Z

At this point its clear we can't fix this with a code change in the lib or HA side, and its something tplink will have to fix in the firmware.

JurajNyiri · 2023-05-04T14:45:35Z

Given that the port 1024 is closed on 1.3.4 and it still works...
Is it possible the camera gets stuck on a call to "filtered" port 1024 and never gets a response thus leading into loop? Could we ignore any communication other than on port 2020?

Just thinking out loud here, I do not know how onvif actually works under the hood.

bdraco · 2023-05-04T14:46:43Z

Given that the port 1024 is closed on 1.3.4 and it still works... Is it possible the camera gets stuck on a call to "filtered" port 1024 and never gets a response thus leading into loop? Could we ignore any communication other than on port 2020?

Just thinking out loud here, I do not know how onvif actually works under the hood.

We really can't do anything about it. Tplink has to fix it. Its clearly broken too deep in the internals.

@JurajNyiri Unrelated to this, but I'm going to make some changes onvif library to fix some I/O in the event loop. They will require awaiting in more places.

I'm bdraco#6479 on discord if you need assistance processing the changes once they get published.

JurajNyiri · 2023-05-04T15:11:56Z

@bdraco

I have set up iSpy (AgentDVR) that tplink is recommending to use. After setting up both cameras (1.3.6 and 1.3.4) they both have working events - when there is a movement they start blinking instantly.

bdraco · 2023-05-04T15:16:43Z

@bdraco

I have set up iSpy (AgentDVR) that tplink is recommending to use. After setting up both cameras (1.3.6 and 1.3.4) they both have working events - when there is a movement they start blinking instantly.

Can you get a wireshark dump on how iSpy is setting up the PullPoint / notify subscription?

We might be able to change to do it that way if there isn't a risk of breaking other cameras.

JurajNyiri · 2023-05-04T15:26:45Z

I got it. I can share privately through Discord, I have added you to friends.

bdraco · 2023-05-04T15:46:57Z

To close the loop on our discord conversation iSpy/Agent DVR doesn't appear to be using events and is doing local processing so thats a dead end.

bdraco · 2023-05-04T16:20:10Z

AFAICT the newer firmware errors on pull point subscription, the webhook subscription appears to be successful, but events never get delivered back to HA.

While the onvif integration tries both ways to get the events, neither work on the new firmware.

bdraco · 2023-05-04T22:47:29Z

I received the 1.3.4 update on the C200 camera with hardware version 3 and everything is still working here with PR #92551. Tested with Pull Point and webhooks

JurajNyiri · 2023-05-05T16:51:35Z

Combination of #92551, #92627 and #92629 makes the latest firmware 1.3.6 work. 🎉

bdraco · 2023-05-05T16:52:52Z

While PullPoint is still broken on 1.3.6 firmware, the webhooks should work after #92627

IMPORTANT: You need to make sure your internal HA url is not HTTPS for the webhooks to work.

bdraco · 2023-05-05T21:08:59Z

The above PRs are in 2023.5.2

mitzu2250 · 2023-05-06T13:58:07Z

For me, since the last updates (2023.5.0 -> 2023.5.1 -> 2023.5.2), none of the two cameras are working anymore.
While the C200 is still broken with the 1.3.6 firmware, now also the C210 is not working anymore.

These are the sensors for C200:

These are the sensors for C210:

For further information, I'm attaching the log:
home-assistant_2023-05-06T13-11-09.243Z.log

I have removed and re-added both cameras in HA.
Restarted HA, cameras, now I only get these:

C200: No motion sensor:

C210: Detection cell but sometimes stuck on "Detected", sometimes "Unavailable"

Enabled debugging and this is what I get:
home-assistant_2023-05-06T13-56-58.245Z.log

JurajNyiri · 2023-05-06T15:00:44Z

@mitzu2250 This issue is for onvif integration, logs posted are for tapo_control custom integration. Disable tapo_control, add camera through onvif integration and enable logs like so:

logger:
  default: error
  logs:
    #custom_components.tapo_control: warn
    custom_components.onvif: debug
    homeassistant.components.onvif: debug
    zeep: debug
    httpx: debug
    onvif: debug

Make sure to specify also firmware of cameras and create, or find and update 2 issues which are specifying the same thing you are experincing, since these are 2 different problems.
Once it is fixed in onvif, it will also be improved and fixed in tapo_control CC as a result.

Optionally, go and read the release notes, or FAQ in the custom integration, both contain information to help you solve your issue. You most probably need to specify internal URL or fix network firewall rules if you are using vlans so that your camera can reach HA. Depending on camera firmwares, nonworking webhooks can cause even the working pullpoint to get stuck like on your C210. I assume your C200 is running 1.3.6 which has broken pullpoint, with only webhooks working - thus you will need to make sure your webhooks actually work.

mitzu2250 · 2023-05-06T19:20:48Z

Hello,

Attached the log:
home-assistant_2023-05-06T19-15-40.203Z.log

The ...201 camera is a C200 with 1.3.6 Build 230424 Rel.77225n(4555)
The ...202 camera is a C210 with 1.3.0 Build 220830 Rel.69909n(4555)

Information in release notes and FAQ didn't solved the issue. The internal URL is ok in the network settings and I don't use vlans, camera is in the same network as the Home Assistant VM.
Also I use http so this shouldn't be the issue.
The only thing is that I'm using Home Assistant inside a VMWare Workstation VM but this wasn't an issue before and not sure why it should be now.
Do you see something relevant in the logs that might help me get the motion sensors back?

JurajNyiri · 2023-05-06T19:57:08Z

The webhooks are not working & the pullpoint on 1.3.6 is timing out because of broken firmware as expected and the C210 gets stuck because it cannot handle both webhook and pullpoint subscription at the same time.

Is your HA instance available at and responds at http://192.168.100.211:8123/ ? Make absolutely sure it does and that camera can reach it.

mitzu2250 · 2023-05-06T23:07:00Z

Looks like something is wrong indeed with my network settings.

While I try to access http://192.168.100.211:8123/ from host computer (where VMware is installed) it works.
If I try to access from another device in the network, it doesn't respond.

It's really strange because with my current network/router settings everything except this works.
With this test I have realized that my Companion App is using the external address all the time...

While writing this I tried one more thing, at some point I selected Static ip configuration and added manually the ip.
Even though it was the correct one, 192.168.100.211, after I selected Automatic, the camera started working and I have the sensors working again!

Thanks for the hint!

I'm not sure why I did this in the past, hope I didn't break anything doing this ...

home-assistant bot added the integration: onvif label Apr 19, 2023

home-assistant bot assigned hunterjm Apr 19, 2023

This was referenced Apr 21, 2023

XM530AI AZISHN 1080P 2.0MP Wifi IP Camera live stream not working #91678

Closed

ONVIF integration problem with Hikvision DS-2CD2432F-I camera - "Failed to set up" #91471

Closed

bdraco added the problem in device Issue lies within the device, such firmware, software, or user customization/config label Apr 23, 2023

bdraco changed the title ~~ONVIF: TP-Link Tapo Camera Motion not working after firmware upgrade~~ ONVIF: TP-Link Tapo Camera Motion not working after firmware upgrade (port 1024 is unresponsive in newer firmware) May 4, 2023

bdraco mentioned this issue May 4, 2023

Improve reliability of ONVIF subscription renewals #92551

Merged

20 tasks

TA2k mentioned this issue May 5, 2023

Hallo, Tapo Cam C200 haben Firmwareupdate erhalten. Nun gibt es den Fehler FetchError: request to https://192.168.100.55/ failed, reason: socket hang up TA2k/ioBroker.tapo#6

Open

JurajNyiri mentioned this issue May 5, 2023

Tapo Cameras: Motion sensor gets stuck #92604

Closed

bdraco mentioned this issue May 5, 2023

Fix missing ONVIF events when switching from PullPoint to webhooks #92627

Merged

20 tasks

bdraco closed this as completed May 5, 2023

orrpan mentioned this issue May 15, 2023

Tapo c200 onvif motion problem on new fw koush/scrypted#796

Closed

martin-webv mentioned this issue Jun 4, 2023

Homekit motion notification stops working koush/scrypted#836

Closed

github-actions bot locked and limited conversation to collaborators Jun 6, 2023

ONVIF: TP-Link Tapo Camera Motion not working after firmware upgrade (port 1024 is unresponsive in newer firmware, webhook messages not delivered) #91661

ONVIF: TP-Link Tapo Camera Motion not working after firmware upgrade (port 1024 is unresponsive in newer firmware, webhook messages not delivered) #91661

Comments

mitzu2250 commented Apr 19, 2023 • edited by bdraco Loading

The problem

What version of Home Assistant Core has the issue?

What was the last working version of Home Assistant Core?

What type of installation are you running?

Integration causing the issue

Link to integration documentation on our website

Diagnostics information

Example YAML snippet

Anything in the logs that might be useful for us?

Additional information

home-assistant bot commented Apr 19, 2023

bdraco commented Apr 19, 2023

bdraco commented Apr 19, 2023

mitzu2250 commented Apr 19, 2023 • edited Loading

xhemp commented Apr 19, 2023 • edited Loading

mitzu2250 commented Apr 19, 2023

bdraco commented Apr 19, 2023

AndreiArdelean1 commented Apr 20, 2023

mitzu2250 commented Apr 20, 2023

bdraco commented Apr 20, 2023

bdraco commented Apr 20, 2023 • edited Loading

mitzu2250 commented Apr 20, 2023

bdraco commented Apr 20, 2023

bdraco commented Apr 20, 2023 • edited Loading

mitzu2250 commented Apr 20, 2023

orwoods commented Apr 20, 2023

mitzu2250 commented Apr 28, 2023 • edited Loading

bdraco commented Apr 28, 2023

bdraco commented Apr 28, 2023

JurajNyiri commented Apr 28, 2023 • edited Loading

bdraco commented Apr 28, 2023

bdraco commented Apr 28, 2023 • edited Loading

MunkeyBalls commented Apr 30, 2023

bdraco commented Apr 30, 2023

MunkeyBalls commented Apr 30, 2023

JurajNyiri commented May 4, 2023 • edited Loading

bdraco commented May 4, 2023

JurajNyiri commented May 4, 2023 • edited Loading

1.3.4 HW 3.0:

1.3.6 HW 1.0:

bdraco commented May 4, 2023

JurajNyiri commented May 4, 2023

bdraco commented May 4, 2023

JurajNyiri commented May 4, 2023 • edited Loading

bdraco commented May 4, 2023

JurajNyiri commented May 4, 2023 • edited Loading

bdraco commented May 4, 2023

bdraco commented May 4, 2023

bdraco commented May 4, 2023 • edited Loading

JurajNyiri commented May 5, 2023

bdraco commented May 5, 2023 • edited Loading

bdraco commented May 5, 2023

mitzu2250 commented May 6, 2023

JurajNyiri commented May 6, 2023 • edited Loading

mitzu2250 commented May 6, 2023

JurajNyiri commented May 6, 2023 • edited Loading

mitzu2250 commented May 6, 2023 • edited Loading

mitzu2250 commented Apr 19, 2023 •

edited by bdraco

Loading

mitzu2250 commented Apr 19, 2023 •

edited

Loading

xhemp commented Apr 19, 2023 •

edited

Loading

bdraco commented Apr 20, 2023 •

edited

Loading

bdraco commented Apr 20, 2023 •

edited

Loading

mitzu2250 commented Apr 28, 2023 •

edited

Loading

JurajNyiri commented Apr 28, 2023 •

edited

Loading

bdraco commented Apr 28, 2023 •

edited

Loading

JurajNyiri commented May 4, 2023 •

edited

Loading

JurajNyiri commented May 4, 2023 •

edited

Loading

JurajNyiri commented May 4, 2023 •

edited

Loading

JurajNyiri commented May 4, 2023 •

edited

Loading

bdraco commented May 4, 2023 •

edited

Loading

bdraco commented May 5, 2023 •

edited

Loading

JurajNyiri commented May 6, 2023 •

edited

Loading

JurajNyiri commented May 6, 2023 •

edited

Loading

mitzu2250 commented May 6, 2023 •

edited

Loading