Failed to fetch ONVIF PullPoint subscription messages with IESS NVR (rebranded TVT) #90916

coserotondo · 2023-04-06T09:29:16Z

The problem

I've a ONVIF compatible NVR (IESS that is a rebranding of TVT) that prior to 2023.4.0 gave no error but only exposed the video streams (no motion detection sensors). With 2023.4.0 the integration added several sensors including motion sensors but now I've the log filled. The full DEBUG log is added to diagnostic data.

What version of Home Assistant Core has the issue?

core-2023.4.0

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

ONVIF

Link to integration documentation on our website

https://www.home-assistant.io/integrations/onvif

Diagnostics information

config_entry-onvif-61dde3b5abff48c5bd7f14162ec9d2f8.json.txt
home-assistant_onvif_2023-04-06T09-06-11.392Z.log.zip

Example YAML snippet

No response

Anything in the logs that might be useful for us?

Logger: homeassistant.components.onvif
Source: components/onvif/event.py:211
Integration: ONVIF (documentation, issues)
First occurred: 08:51:54 (122 occurrences)
Last logged: 11:24:18

Failed to fetch ONVIF PullPoint subscription messages for '00:18:AE:B4:0C:05': Action Failed

Additional information

In 2023.4.0 is the first time that finally I have motion sensor enabled

home-assistant · 2023-04-06T09:29:21Z

Hey there @hunterjm, mind taking a look at this issue as it has been labeled with an integration (onvif) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of onvif can trigger bot actions by commenting:

@home-assistant close Closes the issue.
@home-assistant rename Awesome new title Renames the issue.
@home-assistant reopen Reopen the issue.
@home-assistant unassign onvif Removes the current integration label and assignees on the issue, add the integration domain after the command.

_{^{(message by CodeOwnersMention)}}

onvif documentation
onvif source
_{^{(message by IssueLinks)}}

bdraco · 2023-04-18T06:43:50Z

It may be that your cameras / device doesn't support PullPoint. The spec only requires Notifications (webhooks) or PullPoint

#91485 will add support for webhooks

coserotondo · 2023-04-18T07:11:09Z

It may be that your cameras / device doesn't support PullPoint. The spec only requires Notifications (webhooks) or PullPoint

#91485 will add support for webhooks

Before 2023.4 no motion sensors were added, with 2023.4 I have the motion sensors, now, with 2023.5 all sensors are unavailable.

bdraco · 2023-04-18T07:17:21Z

It may be that your cameras / device doesn't support PullPoint. The spec only requires Notifications (webhooks) or PullPoint
#91485 will add support for webhooks

Before 2023.4 no motion sensors were added, with 2023.4 I have the motion sensors, now, with 2023.5 all sensors are unavailable.

Try power cycling the camera and than restart HA. There is currently a problem where subscriptions are held for too long so the camera can run out of slots when upgrading/restarting HASS which isn't fixed until #91485 gets merged and released

coserotondo · 2023-04-18T10:02:47Z

Tried also to remove and add the integration again but now the motion sensors are missing. Il giorno mar 18 apr 2023 alle ore 09:17 J. Nick Koston ***@***.***> ha scritto:

…

It may be that your cameras / device doesn't support PullPoint. The spec only requires Notifications (webhooks) or PullPoint #91485 will add support for webhooks Before 2023.4 no motion sensors were added, with 2023.4 I have the motion sensors, now, with 2023.5 all sensors are unavailable. Try power cycling the camera and than restart HA — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

bdraco · 2023-04-19T00:14:20Z

#37545 (comment)

bdraco · 2023-05-05T21:09:21Z

Can you try 2023.5.2?

coserotondo · 2023-05-06T06:06:54Z

Motion sensors came back also with 5.1 but even in 5.2, once detected, the sensors remain in ‘detected’ status.

…

On Fri, 5 May 2023 at 23:09, J. Nick Koston ***@***.***> wrote: Can you try 2023.5.2? — Reply to this email directly, view it on GitHub <#90916 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A5CAQBF7HODMV3DXXXHP5NTXEVUAZANCNFSM6AAAAAAWVE7KPA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

bdraco · 2023-05-06T13:53:25Z

Can you enable debug logs for onvif homeassistant.components.onvif and zeep (https://www.home-assistant.io/integrations/logger/)

# Example configuration.yaml entry
logger:
  default: info
  logs:
    homeassistant.components.onvif: debug
    onvif: debug
    zeep: debug

Restart HA

Reload the config entry and than post the log

bdraco · 2023-05-06T17:08:15Z

It looks like the log was cut off. Can you send the timestamps after 2023-05-06 17:43:34.788?

coserotondo · 2023-05-06T20:52:40Z

Re-attached new log compressed. [home-assistant_2023-05-06T16-22-21.590Z.log.zip](https://github.com/home-assistant/core/files/11414850/home-assistant_2023-05-06T16-22-21.590Z.log.zip) I made a check with the NVR logs and the webhooks received: I found that 1/2 second after the motion is detected at the NVR, HA receives the event. Then a few seconds later there is a new event (but no log from the NVR). Il giorno sab 6 mag 2023 alle ore 19:08 J. Nick Koston < ***@***.***> ha scritto:

…

It looks like the log was cut off. Can you send the timestamps after 2023-05-06 17:43:34.788? — Reply to this email directly, view it on GitHub <#90916 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A5CAQBFA3PT5N6XZCSW47TDXE2AQXANCNFSM6AAAAAAWVE7KPA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

bdraco · 2023-05-07T13:30:35Z

s:Header>\n\t<s:Body>\t\t<wsnt:RenewResponse>\t\t\t<wsnt:TerminationTime>2023-05-06T16:28:27Z</wsnt:TerminationTime>\n\t\t\t<wsnt:CurrentTime>2023-05-06T16:18:27Z</wsnt:CurrentTime>\n\t\t</wsnt:RenewResponse>\n\t</s:Body>\n</s:Envelope>\n'
2023-05-06 18:18:28.671 DEBUG (MainThread) [homeassistant.components.onvif] NVR: Renewed Webhook subscription
2023-05-06 18:19:03.514 DEBUG (MainThread) [homeassistant.components.onvif] NVR: Processed webhook onvif_9e69192f2db05e864b49c8ec225dd2ac with 1 event(s)
2023-05-06 18:19:23.846 DEBUG (MainThread) [homeassistant.components.onvif] NVR: Processed webhook onvif_9e69192f2db05e864b49c8ec225dd2ac with 1 event(s)
2023-05-06 18:20:29.966 DEBUG (MainThread) [homeassistant.components.onvif] NVR: Processed webhook onvif_9e69192f2db05e864b49c8ec225dd2ac with 1 event(s)
2023-05-06 18:20:50.504 DEBUG (MainThread) [homeassistant.components.onvif] NVR: Processed webhook onvif_9e69192f2db05e864b49c8ec225dd2ac with 1 event(s)

So it is getting the webhooks

Thats good.

bdraco · 2023-05-07T13:31:15Z

Can you try the updated version with onvif-zeep-async==3.1.1?

cd /config
curl -o- -sSL https://gist.githubusercontent.com/bdraco/43f8043cb04b9838383fd71353e99b18/raw/core_integration_pr | bash /dev/stdin -d onvif -p 92711
cat /config/custom_components/onvif/manifest.json

Verify that the version displayed above is "onvif-zeep-async==3.1.1" and NOT 2.1.1

Restart HA

Test

Delete /config/custom_components/onvif and restart after testing is completed to restore the core version

coserotondo · 2023-05-07T17:37:34Z

I'm now with the custom components I can confirm "onvif-zeep-async==3.1.1". Atteched the new log. The behaviour is the same: detected ok but stay forever. I noticed in the logs that it seems with this version not using webhooks anymore.
home-assistant_2023-05-07T17-29-47.081Z.log.zip

bdraco · 2023-05-07T18:33:09Z

interesting
lots of request to get the snapshot uri

b'<?xml version=\'1.0\' encoding=\'utf-8\'?>\n<soap-env:Envelope xmlns:soap-env="http://www.w3.org/2003/05/soap-envelope"><soap-env:Header xmlns:wsa="http://www.w3.org/2005/08/addressing"><wsa:Action>http://www.onvif.org/ver10/media/wsdl/GetSnapshotUri</wsa:Action><wsa:MessageID>urn:uuid:fa330a22-6661-4095-b090-2d4a3e3da890</wsa:MessageID><wsa:To>http://192.168.210.3:80/onvif/Media1</wsa:To><wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"><wsse:UsernameToken><wsse:Username>homeassistant</wsse:Username><wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordDigest">lrjWtebizg4MU7wHUZpTN7OMpUc=</wsse:Password><wsse:Nonce EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary">fg72REB/0CpOsik38TGNdA==</wsse:Nonce><wsu:Created xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">2023-05-07T17:23:48+00:00</wsu:Created></wsse:UsernameToken></wsse:Security></soap-env:Header><soap-env:Body><ns0:GetSnapshotUri xmlns:ns0="http://www.onvif.org/ver10/media/wsdl"><ns0:ProfileToken>Profile_2_0</ns0:ProfileToken></ns0:GetSnapshotUri></soap-env:Body></soap-env:Envelope>'
2023-05-07 19:23:48.399 DEBUG (MainThread) [zeep.transports] HTTP Response from http://192.168.210.3:80/onvif/Media1 (status: 200):
b'<?xml version="1.0" encoding="UTF-8"?>\n<s:Envelope xmlns:ad="http://www.w3.org/2005/08/addressing" xmlns:addis="http://schemas.xmlsoap.org/ws/2004/08/addressing" xmlns:adw="http://www.w3.org/2006/05/addressing/wsdl" xmlns:ana="http://www.onvif.org/ver20/analytics/wsdl" xmlns:dio="http://www.onvif.org/ver10/deviceIO/wsdl" xmlns:dis="http://schemas.xmlsoap.org/ws/2005/04/discovery" xmlns:et="http://www.hikvision.com/2011/event/topics" xmlns:http="http://schemas.xmlsoap.org/wsdl/http" xmlns:img="http://www.onvif.org/ver20/imaging/wsdl" xmlns:nt="http://www.onvif.org/ver10/network/wsdl" xmlns:ptzw="http://www.onvif.org/ver20/ptz/wsdl" xmlns:s="http://www.w3.org/2003/05/soap-envelope" xmlns:sto="http://www.onvif.org/ver10/storage/wsdl" xmlns:tds="http://www.onvif.org/ver10/device/wsdl" xmlns:ter="http://www.onvif.org/ver10/error" xmlns:tev="http://www.onvif.org/ver10/events/wsdl" xmlns:tns1="http://www.onvif.org/ver10/topics" xmlns:tr2="http://www.onvif.org/ver20/media/wsdl" xmlns:trc="http://www.onvif.org/ver10/recording/wsdl" xmlns:trp="http://www.onvif.org/ver10/replay/wsdl" xmlns:trt="http://www.onvif.org/ver10/media/wsdl" xmlns:trv="http://www.onvif.org/ver10/receiver/wsdl" xmlns:tse="http://www.onvif.org/ver10/search/wsdl" xmlns:tt="http://www.onvif.org/ver10/schema" xmlns:wsa5="http://www.w3.org/2005/08/addressing" xmlns:wsdl="http://schemas.xmlsoap.org/wsdl" xmlns:wsnt="http://docs.oasis-open.org/wsn/b-2" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wstop="http://docs.oasis-open.org/wsn/t-1" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" xmlns:xsd="http://www.w3.org/2001/XMLSchema">\t<s:Header></s:Header>\n\t<s:Body>\t\t<trt:GetSnapshotUriResponse>\t\t\t<trt:MediaUri>\t\t\t\t<tt:Uri>http://192.168.210.3:80/onvif/snapshot?2_0</tt:Uri>\n\t\t\t\t<tt:InvalidAfterConnect>false</tt:InvalidAfterConnect>\n\t\t\t\t<tt:InvalidAfterReboot>false</tt:InvalidAfterReboot>\n\t\t\t\t<tt:Timeout>PT60S</tt:Timeout>\n\t\t\t</trt:MediaUri>\n\t\t</trt:GetSnapshotUriResponse>\n\t</s:Body>\n</s:Envelope>\n'

bdraco · 2023-05-07T18:34:29Z

That looks ok as its only doing it once per camera

bdraco · 2023-05-07T18:37:03Z

<wsnt:CurrentTime>2023-05-07T17:22:50Z</wsnt:CurrentTime>\n\t\t\t<wsnt:TerminationTime>2023-05-07T17:32:50Z</wsnt:TerminationTime>\

Its accepting the subscription

bdraco · 2023-05-07T18:37:46Z

pull point looks good as well
<wsnt:CurrentTime>2023-05-07T17:22:50Z</wsnt:CurrentTime>\n\t\t\t<wsnt:TerminationTime>2023-05-07T17:32:50Z</wsnt:TerminationTime>

bdraco · 2023-05-07T18:39:15Z

I don't see it even trying to pull the events though

bdraco · 2023-05-07T18:52:42Z

I made another change to the PR (but not the library). It should force it to pull

cd /config
curl -o- -sSL https://gist.githubusercontent.com/bdraco/43f8043cb04b9838383fd71353e99b18/raw/core_integration_pr | bash /dev/stdin -d onvif -p 92711
cat /config/custom_components/onvif/manifest.json

Verify that the version displayed above is "onvif-zeep-async==3.1.1" and NOT 2.1.1

Restart HA

Test

Delete /config/custom_components/onvif and restart after testing is completed to restore the core version

bdraco · 2023-05-07T19:19:22Z

I just made another fix so if you have run the above already, please do it again and restart again

coserotondo · 2023-05-07T21:06:09Z

I did it, same behaviour, sensors remains in 'detected' mode. Just a question: what should be the right behaviour? I mean, once the NVR sends the motion event, should the NVR/Camera send a 'clear' event? Why not immediately clear after the event?
home-assistant_2023-05-07T20-53-04.267Z.log.zip

bdraco · 2023-05-07T21:11:16Z

It usually clears about 30 seconds after depending on how they implemented it.

Some NVRs don't actually implement the clear event which might be the case here.

bdraco · 2023-05-07T21:13:29Z

The subscription setup looks good..

2023-05-07 22:47:22.879 DEBUG (MainThread) [onvif] 192.168.210.3: Checking for broken relative timestamps: expected_interval: 0:10:00, current_time: 2023-05-07 20:47:22+00:00, termination_time: 2023-05-07 20:57:22+00:00
2023-05-07 22:47:22.880 DEBUG (MainThread) [onvif] 192.168.210.3: Relative timestamps OK: expected interval: 0:10:00, actual interval: 0:10:00
2023-05-07 22:47:22.881 DEBUG (MainThread) [onvif] 192.168.210.3: Renew notification subscription in 480.0 seconds
2023-05-07 22:47:22.881 DEBUG (MainThread) [onvif] 192.168.210.3: Start the notification manager

I made another change to the PR and the library onvif-zeep-async==3.1.2.

cd /config
curl -o- -sSL https://gist.githubusercontent.com/bdraco/43f8043cb04b9838383fd71353e99b18/raw/core_integration_pr | bash /dev/stdin -d onvif -p 92711
cat /config/custom_components/onvif/manifest.json

Verify that the version displayed above is "onvif-zeep-async==3.1.2" and NOT 2.1.1

Restart HA

Test

Delete /config/custom_components/onvif and restart after testing is completed to restore the core version

coserotondo · 2023-05-07T22:04:17Z

Attached DEBUG with latest. Same behaviour. Maybe no clear event at all from NVR. If it is the case, how to clear the sensor?
home-assistant_2023-05-07T21-54-34.112Z.log.zip

bdraco · 2023-05-07T22:13:20Z

I don't actually see Home Assistant trying to pull the event messages in the log but it may just be that the log was cut off too soon.

If it is the case, how to clear the sensor?

The NVR has to send a false event to clear the sensor. Some NVRs have to be configured to enable events

bdraco · 2023-05-07T22:26:39Z

Do you have a link to the NVR's docs/where you got it from/where to buy it?

coserotondo · 2023-05-07T23:21:21Z

Unfortunately it is only in italian and not updated. In any case there is no explicit option to enable events but to enable ONVIF and this is what I did adding a specific user for HA. Infact after this I was able to add the NVR with the ONVIF integration. As it sends the motion event I don't think there should be anything else to enable (otherwise neither the motion event had to work).

issue-triage-workflows · 2023-08-06T00:18:36Z

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.
Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍
This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

home-assistant bot added the integration: onvif label Apr 6, 2023

home-assistant bot assigned hunterjm Apr 6, 2023

This comment was marked as off-topic.

Sign in to view

bdraco changed the title ~~Failed to fetch ONVIF PullPoint subscription messages~~ Failed to fetch ONVIF PullPoint subscription messages with IESS NVR (rebranded TVT) Apr 19, 2023

bdraco added the problem in device Issue lies within the device, such firmware, software, or user customization/config label Apr 23, 2023

bdraco added the waiting-for-reply label May 5, 2023

issue-triage-workflows bot added the stale label Aug 6, 2023

issue-triage-workflows bot closed this as not planned Won't fix, can't repro, duplicate, stale Aug 13, 2023

github-actions bot locked and limited conversation to collaborators Sep 12, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Failed to fetch ONVIF PullPoint subscription messages with IESS NVR (rebranded TVT) #90916

Failed to fetch ONVIF PullPoint subscription messages with IESS NVR (rebranded TVT) #90916

coserotondo commented Apr 6, 2023

home-assistant bot commented Apr 6, 2023

This comment was marked as off-topic.

bdraco commented Apr 18, 2023

coserotondo commented Apr 18, 2023

bdraco commented Apr 18, 2023 •

edited

Loading

coserotondo commented Apr 18, 2023 via email

bdraco commented Apr 19, 2023

bdraco commented May 5, 2023

coserotondo commented May 6, 2023 via email

bdraco commented May 6, 2023

bdraco commented May 6, 2023

coserotondo commented May 6, 2023 via email •

edited

Loading

bdraco commented May 7, 2023

bdraco commented May 7, 2023

coserotondo commented May 7, 2023 •

edited

Loading

bdraco commented May 7, 2023

bdraco commented May 7, 2023

bdraco commented May 7, 2023

bdraco commented May 7, 2023

bdraco commented May 7, 2023

bdraco commented May 7, 2023

bdraco commented May 7, 2023

coserotondo commented May 7, 2023

bdraco commented May 7, 2023

bdraco commented May 7, 2023

coserotondo commented May 7, 2023

bdraco commented May 7, 2023

bdraco commented May 7, 2023

coserotondo commented May 7, 2023

issue-triage-workflows bot commented Aug 6, 2023

Failed to fetch ONVIF PullPoint subscription messages with IESS NVR (rebranded TVT) #90916

Failed to fetch ONVIF PullPoint subscription messages with IESS NVR (rebranded TVT) #90916

Comments

coserotondo commented Apr 6, 2023

The problem

What version of Home Assistant Core has the issue?

What was the last working version of Home Assistant Core?

What type of installation are you running?

Integration causing the issue

Link to integration documentation on our website

Diagnostics information

Example YAML snippet

Anything in the logs that might be useful for us?

Additional information

home-assistant bot commented Apr 6, 2023

This comment was marked as off-topic.

bdraco commented Apr 18, 2023

coserotondo commented Apr 18, 2023

bdraco commented Apr 18, 2023 • edited Loading

coserotondo commented Apr 18, 2023 via email

bdraco commented Apr 19, 2023

bdraco commented May 5, 2023

coserotondo commented May 6, 2023 via email

bdraco commented May 6, 2023

bdraco commented May 6, 2023

coserotondo commented May 6, 2023 via email • edited Loading

bdraco commented May 7, 2023

bdraco commented May 7, 2023

coserotondo commented May 7, 2023 • edited Loading

bdraco commented May 7, 2023

bdraco commented May 7, 2023

bdraco commented May 7, 2023

bdraco commented May 7, 2023

bdraco commented May 7, 2023

bdraco commented May 7, 2023

bdraco commented May 7, 2023

coserotondo commented May 7, 2023

bdraco commented May 7, 2023

bdraco commented May 7, 2023

coserotondo commented May 7, 2023

bdraco commented May 7, 2023

bdraco commented May 7, 2023

coserotondo commented May 7, 2023

issue-triage-workflows bot commented Aug 6, 2023

bdraco commented Apr 18, 2023 •

edited

Loading

coserotondo commented May 6, 2023 via email •

edited

Loading

coserotondo commented May 7, 2023 •

edited

Loading